AOIN is a creative agency based in San Francisco, USA whose roots are with Film/TV Production, creating digital content and immersive experiential installations for brands, agencies, entertainers and events. As early adopters of MR, we delved into how Berto Mora, Lead R&D/Producer and Kevin Zhu, Senior Designer at AOIN see xR shaping the industry and what the future has in store for their AR and MR workflows...
How did you first get into Augmented and Mixed Reality?
Berto Mora: I got into AR/MR about 8 years ago when 360 video was becoming a thing. Originally I went to film school to study Motion graphic and CG to create match moving effects for film, but at the time real-time graphics were really not a thing, so I lost interest. Recently about 3 years ago I learned about disguise and Notch and the possibility of real time content generation using their tool set and here I am learning these new workflows they have been developing.
Kevin Zhu: I actually got interested in Augmented and Mixed Reality because of full immersion Virtual Reality. I remember seeing VR headsets enter the market and thought it was just a temporary gimmick. The resolution was unbearable, GPUs weren’t powerful enough for a smooth experience, and there was also a lack of content being developed. I started to change my tune as tools and games came online through the Steam marketplace, like Tilt Brush by Google, and the Brookhaven Experiment. It wasn’t until local artists created stunning motion graphics in this medium* that I really opened my eyes to the new possibilities of tools like VR, 360° video, and ultimately AR / MR.
What excites you about the Mixed Reality creative opportunities?
BM: The most exciting thing for me is being able to merge virtual worlds with the physical word. This is something that I always wanted to achieve and it is possible with these new workflows and technologies.
KZ: I am most excited about access to a new medium that breaks the traditional boundaries of a screen or surface. Instead of thinking in flat pixels as a content designer, the canvas is now extended into 3D space and truly becomes environmental.
What it is that is driving the market and your business to adopt to Mixed Reality workflows?
BM: In order to drive this market and to get interest into the MR workflows you have to be able to prove that it's possible with testing and R&D. We have convinced ourselves that MR is now possible to deploy in 2019. These possibilities are driving us to adopt the workflow and implement it within our work here at All of it now.
KZ: The broadcast market is one of the biggest drivers in terms of investment and deployment. This is especially true in the United States, where the television networks are some of the largest in the world. More and more broadcast shows are adopting these workflows to stay competitive and fresh from an entertainment perspective.
What is different in the workflow from what you have used before?
BM: The difference in this workflow for me is the real-time aspect. Everything else has been possible for years, but today we have better technology at a prosumer and consumer level that allows these workflows to be workflows to be more accessible to studios and agencies like All of it Now.
KZ: The difference in this workflow for me is the real-time aspect. Everything else has been possible for years, but today we have better technology at a prosumer and consumer level that allows these workflows to be coming more popular and available to studios and agencies like All of it Now.
Does it allow you to be more effective?
BM: At All of it Now we try to adapt to new technologies as soon as they are available - that's what gives a edge from other agencies. This allow us to experiment with these new workflows and integrate them within our work. The best part about all this is being able to blow people's minds.
KZ: Mixed Reality allows us to be more effective in creating experiential illusions. The fact that it is still a developing and emerging field works to our benefit when entering industries that have used the same traditional techniques for a long time. Instead of trying to compete against massive agencies that exist in the traditional workflows, we are able to join the industry from a new perspective and integrate as a niche visual component.
Where do you see the future of MR going with All of it Now? Will it change the way you approach future projects? If so how?
BM: I think if the the idea of MR gets adopted All of it Now will keep implementing the workflow. Since having a background in film and graphics I see that these new workflows are reviving the the film industry and giving it a reboot. Instead of having practical effects we are now entering a era where we can produce real-time digital effects to enhance the on set in camera experience.
KZ: I think the biggest change to come with MR is accessibility. As the tools and workflow are developed and adopted further, we expect a shift of focus away from troubleshooting technical workflow. For us as a design agency, this means staying technologically agnostic by exploring any and all MR tools, while fostering artistic growth in our community by sharing our resources and knowledge.
What type of projects will you use MR for?
BM: Recently we have seen MR being popular in Broadcast TV shows and being requested heavily in a festival environment where MR is becoming more popular. We have done shows like the Macy's day parade in VR, CS GO Championships live, and America's Got Talent to name a few.
KZ: Any and all projects. We try to stay open minded.
What applications of MR have you seen that you take inspiration from?
BM: A lot of mobile AR is really where I get a lot of inspiration from. They have mastered the space of AR and they're the ones really pioneering this technology. I tend to study their workflow and apply those techniques within the workflow disguise has provided to achieve these effects.
KZ: Lux Machina: They are doing some amazing work with MR to slowly kill off the green screen. The use of MR in film and hollywood to bring an element of VFX onto the set from post production is a game changer in how directors and crews can work and engage with the environment.
BOX by Bot & Dolly: A pivotal source of inspiration for us as an agency. It mixes many of the mediums and content styles that we engage with on a daily basis, and executes the whole process so well. In general, just a great example of creating an experience that is truly engaging.
Tycho Practical Effects by GMUNK, Conor Grebel, and others: This one is the least relevant to MR, but is an inspirational reminder for myself that it’s all smoke and mirrors at the end of the day. How the effects are created is not priority - it’s always about the end effect and experience you are trying to craft for the viewer.
How has MR enhanced your projects?
BM: MR has enhanced our projects virtually. It has allowed us to extend our physical world into the virtual world and be able to provide a never before seen experiences and come up with new ways to enhance our story.
KZ: Mixed Reality design is about creating environments and systems for a team of people to engage with and explore for the audience. It’s great to be able to add something new to the experience without replacing technologies and workflows already in place. Learning to work together with larger teams we don’t normally interact with has been a fantastic way to forge relationships and produce a product that is greater than the sum of its parts.