immersive media

The Earth is Watching

treat by Julia Zanin de Paula

PROTOTYPE EXPERIENCES LLC AND ROTO

"Any sufficiently advanced technology is indistinguishable from magic."

Arthur C. Clarke

intro

I grew up obsessed with fairytales and folklore. Storytelling has always helped me cope with and understand deeper issues in life using the forms of monsters, weird imagery, and fantastical elements. The opportunity to bring to life such serious matters on a project that ties the magical and peculiar to scientific evidence, climate information, and real change is beyond amazing, largely unprecedented, and is in tune with my aesthetic, outlook, and experience. There is a real and exciting opportunity here to touch people through this video, in addition to leaving a mark on the world at an amazing museum and initiative.

In the depths of the Amazon forest of Brazil, there is a legend of a boy with bright orange hair, and his feet are turned backward, this legend is named “Curupira.”

Curupira uses its backward feet to create footprints that lead to its starting point, thus making hunters and travelers confused. He preys upon poachers and hunters who greedily take more than they need of the forest, also attacking those who hunt animals in ways contrary to the balance of nature.

Across countries and cultures, there are many different versions of the legend of this creature of the forest, protector of Mother Nature. How beautiful that different cultures have recognized the primal need to create such caretakers of our planet. It is something sacred. Peruvians call it Shapishico, the Slavic call it Leshy, the Greek called it Gaia, and nowadays we call it Greta Thunberg! All over the world, humans have craved this one-off savior and protector that would bring justice and harmony to nature. It’s the human way to weave myth and magical realism and heroism into our stories and narrative. This is why I love bringing these magical elements into my work.

This is also why I feel this film will benefit from mythical and magical elements that speak beyond “reality” and science, in order to inspire interest, to seed people’s minds, and to tickle them into thinking deeper about climate change and into recognizing their role in it.

We must come to terms with nature. We’re challenged as mankind has never been challenged before to prove our maturity and our mastery, not of nature, but of ourselves.

Rachel Carson

Our magical elements and metaphors will only be effective for our purposes when used judiciously. They can only serve, touch, and enchant audiences when they are combined with well thought out real-life imagery. A movie about flying creatures having fun in the sky won’t be as powerful as one about humans who learn they can fly after taking a journey and discovering their wings. The Point here is that we need to work on a film story that will take our audience through a grounded, relatable narrative, that is occasionally interrupted - or surprised - by magical elements and metaphors.

What you are about to read here is the tone and the mood I want to bring to this piece, rather than a script. The script can only be built after further understanding the museum and its themes and mission. So please take away a general idea of what my brain is connecting to, but know I will diligently study and come up with ideas that fit the concepts needed once we have those more clearly outlined.

I feel we must avoid the didactic nature and other clichés of the plethora of expected educational and “message” videos. We should resist such fashionably dystopian scenarios as an ocean filled with many plastic bags or depictions of tortoises with their nostrils filled with straws. My initial response here is to steer away from these loaded images and touch people more imaginatively and deeply, without shocking or depressing them, but rather by amazing them into being inspired to ponder the more positive outcomes of our vast human potential. That's the beauty of myth and fantastical elements, they are ubiquitous and familiar to humans on all levels, and we will use them to build and convey a story with a modern and authentic message – that won’t feel like a “message.”

I propose the ambitious goal of creating a film where science and art coexist. Where the right and left brain are engaged in a symbiosis greater than the sum of their traditionally divided hemispheres. Where we bring our patrons an experience that sends them back into the world informed and emotional and excited and, hopefully, even a bit transformed. We should think of this as a call to action that springs from within, rather than a lesson that checks the expected boxes.

I truly believe that trusting the audience to fill in the gaps of what the experience signifies will make for a unique, moving, enjoyable, hopefully repeatable and definitely unforgettable experience for them. How to communicate the seemingly obvious yet complex issues of Climate Change in one film to be viewed by all strata of education, age, economic background, and political leaning?

One time-tested way is through the use of myth, fairytale, and archetypes, things most of us have experience with since our earliest times in life. Almost a "universal background" we share.

bridge to mind

I’ve been thinking a lot during this week about how to make a video more interactive in a fun and innovative way. And I’m completely open to exploring different things. What is brewing in my mind right now, and exciting me, is the possibility of using augmented reality to expand the universe we are showing on screen.

Imagine a virtual world superimposing our video and the room itself when one of our museum attendees is grabbing their phone to record the room. In my mind, we’ll make such a breathtaking video that people will want to shoot Instagram and TikTok content right in front of it. Like Bloomtanica in NYC and other unique and beautiful experiences, they can't help but take some photos within our created world – to become a shared part of it all.

So, after embracing our cinematic visuals, sometimes sensory and graphical, the audience takes out their phone and points the camera to the screen. And boom, even more content blooms onto their camera screen. No two people will experience it the same way, and it will be different the next time they visit, and the time after that. There is an opportunity here to use AR throughout the museum as well. I'm open to give my ideas on it!

“The cosmos is within us. We are made of star-stuff. We are a way for the universe to know itself.”

Carl Sagan

the journey of emotions

Our experience will come up after all lights in the room have dimmed, changing the mood radically. The film begins and we see a journey. Beautiful images of people in extraordinary places, exploring nature, appreciating life. We can mix it with textural and sensory shots of liquids/ plants/ molecules/fractals/explosions of color/various perspectives of the Earth...

In the background, as a voice-over, we hear the voices of scientists, explorers, and philosophers, all dwelling on the myriad observations through time of the state of our planet. The voices could contrast with what we are seeing, a very negative recollection of the state of humankind while we see peace and movement and thriving humanity. I love the works of Richard Dawkins about biology or someone like James Baldwin. I would like to create a chorus mixing in scientists with free thinkers, stepping up the conversation to a deeper, meaningful level. We must bring in the classic Einstein and Marie Currie, but also touch on more modern ideas of the world/ society. Again, we wish here to engage the whole brain – the entire person – emotionally and intellectually as one.

Sensory is key here.

- in the AR world, we could transform some of those dooming voices into life in select scenes. The perfect woods a person is walking through is interlaced with energy cables, glowing in colors out of step with nature. A beautiful lake is filled with weird black gunk and fire. Some “technology" weeds are growing inside a family’s library, perhaps even dividing the family. We could explore going as far as putting some augmented reality masks and outfits on people the camera is pointed at. Like the person itself is covered in gunk or webs of internet cables and tendrils. Possibilities are endless with how we can add to this mythic/magical cautionary tale of our journey.

After sitting with these stunning images and sensory sounds, we are gradually transported to a world of uniqueness and surreal and, most importantly – Hope and Positivity. Our characters start flying, and we see cars and cities being overtaken by flowers. One of our character's hands can transform into pages of a book, which can transform into butterflies. The voices of scientists and thinkers suddenly shut up, and we realize we can now sit in silence and absorb the beauty of the images, and realize the validity each of our own thoughts and voices brings to this cosmic chorus.

Our AR world could now positively add to these images, we see more people flying. Flowers now grow everywhere, even onto the people the audience points their phone cameras to.

Now our imagery begins to make its way back to “reality,” but this reality is suddenly a little less “natural”. We see technology expanding people’s lifestyles. Electric cars, remotely driving themselves in a kind of kaleidoscopic ballet, wind turbines spinning together like synchronized pinwheels, capturing the breath of the wind to “sing” a magical, ethereal song, green energy that appears as fractals of every possible shade of the color green…you name it. In voice-over, we hear the positive changes humans have made to transform the environment. Ozone layer closing, oceans getting cleaner, less plastic consumption, industry using recycled materials, the end of glacial melting, wars over fossil fuels ending and an unprecedented time of peace ushering in a New Renaissance… Any messages of Hope as the engine for achieving such goals belong here.

In the AR world, we see the technological world giving space to exist in harmony with newly thriving nature. Trees growing, calm, and clear water is on the floor of our space. Clear beautiful skies are projected into the ceiling, unimagined flora and fauna between the two.

This concept can expand and be added on, there are so many possibilities. We can take the audience through an emotional journey (fear, strangeness, hopefulness) with images and sound on the screen. The key here is to mix myth, symbolism, and metaphors with real and truthful information. And, if the audience is open to it, we would ADD to that journey with augmented reality. I haven’t seen AR being used in this scale before and it would be a very interesting challenge to make such a huge interactive piece.

sound around

Another extremely exciting thing about this project is how we can build a narrative through sound. With the array of high-quality speakers throughout the room, there are endless possibilities for us to use music, sound effects, and voices as tools to ignite and emphasize the emotions of the film journey. Our detailed audio mix employing Dolby Atmos and/or other immersive sound fields, whether swirling in cacophony or pinpointed for effect, will use its own powers of suggestion and emotion to amplify our visuals.

If we have a flock of birds passing through the screen, for example, their sounds can follow the path they are taking through the screen. From right to left/ bottom to top. The method of how sound will surround and embrace our audience will be an important step of the process and one I want to make sure I'm involved in to ensure a truly immersive experience for those watching. Immersiveness must be a part of the presentation to add to the experience even if the viewer should opt out of AR. The way to make sure a film is immersive is through spatial sound design.

peculiar pictures

I've been so lucky to have directed and created many peculiarly beautiful projects. The images below are part of my portfolio of work.

For our project, I'd mix my technical knowledge of the industry and my creativity to bring another peculiarly beautiful project to life.

With a mix of capturing specific images the right way (high-resolution cinema cameras), purchasing highly specific and fitting stock footage, and working in post-production with comp artists and animators to create metaphorical universes, we will create amazing work together.

This is also an opportunity to create and go beyond creatively if we can implement AR into the experience.

We will assemble an amazing group of artists and filmmakers to shape our cohesive, positive, and surreal experience.

I truly appreciate the opportunity to share my ideas and my work with you. Thank you for reaching out and reading through.