From March 2018 I started leading a research project funded by Debora Arango Higher Education School. The project aimed to create a bespoke process for the creation of an immersive story.
For six months I led and worked side by side with a team of professionals from the audiovisual industry i.e audio specialist, camera crew, producer and actors.
The first plan was to create a 3D environment and placing real-life footage of performing actors. So we started shooting actors against a green screen and testing the resulting video inside of very basic 3D environments.
After testing the chromed characters inside of both Unity and Blender, I decided that the most efficient course of action due to the complexity of this project was to produce a Monoscopic 360 video with ambisonic sound instead of building a VR app.
We spent a good amount of working hours trying to figure out the best way to place characters in front of a moving camera. One of the biggest challenges was consistently matching lighting schemes on both ends (physical and digital), Another big challenge was to figure out how to place and reveal characters in the 3D environment preventing them to reveal their flatness once the viewer’s point of view changed.
We managed to overcome all these obstacles and ended up introducing a total of 5 characters, including two that directly interacted with the viewer at a very close range.
Regarding the 3D environment, we discover that buying an existing 3D environment was not an option as we wanted it to reflect specific architectonic features from our culture and the models available online have very different ones, so apparently, depicting a typical Colombian city hasn’t been of interest for 3D artists.
So I decided to start building the entire fragment of the city from scratch.
The first models were quite basic and this was pretty useful as the script evolved a lot and we ended up changing the story and characters more than 3 times.
I also started testing two different approaches for building creation, the first one explored the process of building an entire facade by extruding some features of a frontal photo of an existing building.
The process proved to add a quite realistic look to the scene but, at the same time, added a lot of size to the textures used in the project and demanded a lot of retouching e.g removing cables from the facades or getting rid of shiny materials on doors and windows.
The second option was to build all the locations from scratch using conventional 3D modelling and texturing techniques.
In the end, I decided to mostly do every asset from scratch to have enough flexibility to design the 3D environment, in total, it took me over three months to build a fragment of typical Colombian city spread across a 5 by 5 array of blocks.
It is worth mentioning that I did use google maps to navigate the cities of Medellín and Envigado to get inspired and also to get the footage I needed to replicate architectural features of two emblematic buildings.
