O1 - To understand the creative and technological requirements, creative uses and impact of Virtual Production for film, video and mixed reality environments by exchanging information between scientists and creative professionals on the current technological state of the art and advanced practice; proposing guidelines, scenarios and scope ; setting up a network of creative and technical experts to complement the project’s partner User Group; exploring scenarios and planning evaluations, with creative professionals from the film, TV and artistic communities; carrying out a series of short production-based experiments using the technologies as they evolve throughout the project and using the feedback to guide R&D.
O2 - To research and develop performance and scene capture methods for Virtual Production that can support the real-time integration of elements by producing a final camera track automatically, on set , real-time short-range depth map generation, developing a depth- and dynamic-range enhanced omnidirectional capture system to capture information about dynamic scene lighting, and locations for combination with virtual sets, and real-time high-quality 3D reconstruction on large sets.
O3 - To research and develop methods of combining captured performances, video, CGI, lighting designs and Visual Effects and viewing them with real-time render by algorithms for real-time ray tracing, lighting simulation and depth compositing; real-time compositing of a fixed visual effects stack ; algorithms for layered depth extraction; software modules for realtime ray tracing and advanced lighting simulation ; a framework for 2D compositing operations in a configurable graph ; algorithms for automated depth-based matte extraction ; real-time demonstrators for rendering and lighting ; prototypes for a real-time configurable 2D compositing pipeline and real-time compositing with depth ; and multi-view fusion of dynamic omnidirectional video.
O4 - To research and develop methods of designing, directing, navigating, editing and modifying Virtual Productions in a collaborative environment by specification and interface design for Virtual Production Editing Tools; a first prototype for virtual set editing and lighting ; a demonstrator for online set, animation and lighting editing ; a specification and tools for holographic live performance and installation art .
O5 - To test and evaluate the technological performance, efficiency and creative impact of the Demonstrator in Experimental Virtual Productions for a range of contexts by preparing an integration plan and APIs ; carrying out initial formative trials and producing a prototype Dreamspace Live View System; producing final prototypes of the integrated Dreamspace toolsets and performance environment ; carrying out and evaluating technology field trials in experimental production.
O6 - To showcase and disseminate the results to creative professionals in the film, TV and artistic media communities, and to scientists and media technology developers, and prepare the exploitation of the project results by establishing a project identity and operational plan; presenting and publishing results in showcases, workshops, roadshows, conference sessions, electronic media and journals; preparing and updating dissemination and exploitation plans.