In the past console generation, there has been a significant increase in fidelity. In the past ten years, developers like Naughty Dog and Sony Santa Monica have transitioned from painstakingly hand-drawing facial and movement animation to creating games and characters entirely through performance capture. So it should be no surprise that Sony has applied for a patent for yet another innovative performance-capturing system. The patented technique, submitted on April 1, 2022, by Sony Interactive Entertainment LLC, uses mesh information from a human actor. He has been photographed in three dimensions to record the depth of the face. This polygonal mesh is often hand-made by animators utilizing the vertices dots on a mocap actor’s face as a reference source. It is used in game development to create precise human models. Currently, most triple-A videogame production uses this technique, including the forthcoming Spider-Man 2 from Insomniac. While employing a combination of 3D and 2D cameras to train an artificial intelligence program that can precisely record facial expressions and construct mesh data without human involvement, this system would automatically generate mesh data.
This invention utilizes artificial intelligence (AI) to create tasks that previously required a lot of manual effort, which aligns with industry trends. The usage of AI in the video game business has significantly increased recently. Tasks range from interpolating frames for smoother gameplay to upscaling texture resolution. In truth, using AI in triple-A games isn’t new to the gaming community. Recently, Rockstar Games faced criticism for employing AI upscaling in their GTA Trilogy remake, which resulted in some unpleasant spelling errors on various items. As this patent explicitly mentions, the use of AI, in this case, is intended to lessen the effort and contain the exorbitant expenses of facial animation. The method is highly time-consuming and expensive, requiring hundreds of animators to create thousands of blend shape and joint files to replicate video sequences performances.
