Generative-AI- and Optical-Flow-Based Aspect Ratio Enhancement of Videos

The global ultra-wide display market is growing rapidly due to widespread consumer adoption. However, existing content, especially on TV screens, is mainly designed for lower aspect ratios like 4:3. As TVs shift towards wider formats, there's a need to develop a method to transform legacy content and adapt to varying screen sizes. This shift holds the potential to revolutionize the content viewing experience for gamers, content creators, and streaming enthusiasts. While delivering high-quality visual content for dynamic aspect ratios remains a challenge, recent advancements in Deep Learning show promise in addressing these problems with generative approaches. Our contribution begins with a survey of prior video completion efforts, providing a foundational backdrop. We then introduce our novel solution, combining optical flow methodologies with generative latent diffusion models. These models, conditioned on an initial prompt and evolving video frame, refine content generation. We validate our approach on the DAVIS dataset, demonstrating its efficacy and robustness. In summary, our study pioneers advancements in content generation for ultra-wide displays.