Advanced visual computing for digital media production

Abstract

This Ph.D. thesis delves into the realm of digital art creation and virtual reality (VR), exploring efficient and realistic 2D, 3D, and 4D asset creation to revolutionize the way of digital media production. The research encompasses three core works, each targeting specific challenges in 2D, 3D, and 4D art creation to propose novel solutions that enhance the precision and efficiency of digital media production.

This work introduces SmartSmudge, a dynamic digital painting algorithm developed in response to the cumbersome process of color smudging and painting in traditional digital painting software. SmartSmudge was designed to facilitate dynamic brush adjustments and real-time region selection, enabling users to effortlessly and efficiently achieve desired realistic shading effects during 2D art creation. The tool's effectiveness in creating natural and efficient shading effects is confirmed through both quantitative and qualitative evaluations.

In the second work, the focus shifts to 3D accurate and robust sketching and modeling within VR environments. Recognizing the limitations of mid-air drawing, which often results in inaccurate sketches due to the complexity of geometry relationships among strokes, HandPainter is developed to allow for seamless, precise, and robust freehand 2D and 3D sketching without additional devices for 3D shape generations. Versatile experiments and evaluations highlight HandPainter sketching performance in creating more accurate and realistic 3D sketches efficiently over conventional methods.

The final work addresses the challenges of high-fidelity and real-time 4D asset generation in VR and AR experiments. The proposed VR-GS system introduces an efficient and robust algorithm for realistic 4D motion synthesis, utilizing physical dynamics-aware cages and a two-level embedding strategy for rigid body and soft body simulations with different rigidities. This system ensures real-time both physics-based and geometry-based generative dynamics, not only significantly improving the simulation and reconstruction efficiency and performance but also paving the way for novel forms of 4D digital media production.

Collectively, these contributions signify a leap forward in the realm of digital art creation, leveraging cutting-edge visual computing technologies to offer realistic, efficient, and robust solutions for the generation of 2D, 3D, and 4D digital assets. This thesis marks a pivotal step forward in the fields of digital media production and manipulation.