The animated short film “Dog and Boy” recently launched on Netflix has quickly gained over 2 million views and 20 million+ views on a single tweet. The scene art of the short film is generated using AI, rather than drawn by humans, which is a great step forward for the animation industry. The Hangzhou Wujie AI painting platform has accumulated millions of users by providing low-threshold, multi-type AI painting experience. With the open source of AI large models, AI can already be combined with animation production in a deeper way, from drawing to coloring to finishing. AI animation production is still not perfect, but it is developing at an astonishing speed and is expected to help solve the long-standing problem of insufficient production capacity in the animation industry.
Google has recently released two AI models, Phenaki and Imagen Video, that are capable of generating videos from a single cue word or description. Phenaki is a Text-to-Video model that can generate up to two minutes of video in minutes, while Imagen Video is based on the diffusion model and is characterized by a high resolution while being able to understand different art styles and 3D structures. Both models have been praised for their ability to create realistic videos with story-telling elements. For example, Phenaki can generate a video of a teddy bear diving, slowly surfacing, walking onto the beach, and then walking by a bonfire. Imagen Video can create videos with high resolution and a variety of art styles and 3D structures. Both models are revolutionizing the field of video generation and are set to be widely used in the near future.
The animation industry has always been a labor-intensive industry, with a large demand for animation works on streaming media platforms, but insufficient production capacity has long affected its development. Tanaka Hiroshi, the photography supervisor of “Dog and Boy”, said that with the help of AI, producers can be liberated and everyone has more time to focus on creative work. AI can generate sketches and complete coloring for animation scenes, allowing human painters to act as “sorters” and select the pictures they want from a large number of pictures generated by AI. AI-assisted animation can help reduce labor intensity, improve production efficiency, and meet the market demand for animation works.
AI can significantly improve animation production capacity in terms of cost and efficiency. AI can reduce the time for animation drawing by one-fifth of the original time, and generate 100 pictures (accuracy 4k) within 3 minutes. AI can also reduce labor costs, with each 4k-precision picture costing no more than 20 cents. Although AI can streamline the art team, the combination of AI and animation is not yet perfect.
AI technology has improved animation production, but the scope involved is still limited. AI is currently unable to complete tasks such as lighting, AI frame supplementation, and dynamic special effects. Lighting is especially difficult to integrate with characters without wearing them out. AI is also trying to solve the automatic generation of inbetweens to make the action look smoother. AI technology has improved animation production in individual production links such as sketching, modification, and coloring, but manual assistance is still required in the later stage.
AI technology has made great progress in the animation industry, allowing for faster and more efficient production. However, AI still has difficulty in generating in-between pictures and dynamic special effects. For example, when drawing the coherent action of Monkey King swinging a stick, there may be broken strokes between the lines generated by AI, which need to be filled manually. Similarly, AI is also difficult to achieve perfectly in animation when it comes to dynamic special effects, such as motion blur blur operation from the beginning of drawing. To address these issues, industry experts have proposed a solution using ControlNet and T2I-Adapter technology, which is expected to be better resolved in three months.
AI has made great progress in understanding human language, but it still has difficulty in understanding the abstract concepts in some animations. For example, in the anime “”One Piece””, the protagonist Luffy has the ability to extend his arms and legs to attack the enemy. AI has difficulty understanding this exaggerated action, and needs more clear instructions from the user to understand, such as the height of a person, the length of the arm extension, and the clenched fist. AI is still learning and improving its understanding of human language and animation, and with more research and development, it will be able to better understand these abstract concepts.
AI has the potential to revolutionize the animation industry, but there are still some unfavorable factors that limit its ability to directly generate high-quality animations. According to Zhao Jiecheng, AI can currently be used for year-end animations that require a large amount of content, or animations that already have the first season. Unbounded AI has proprietary entries and plug-ins that are being optimized, and more testing with animation practitioners is needed to turn the theory into reality. AI could be a great tool for animation production companies that are ready to start doing more content, as it can help to increase accuracy and reduce the amount of time and effort needed to create animations.
Integrating AI into animation production is not only a technological transformation, but also a thinking transformation. At the beginning of 2022, there was a wave of resistance to AI overseas, with many artists boycotting AI paintings. However, Zhao Jiecheng believes that the wave of AIGC has come and AI will not completely replace painters. Traditional painters should establish an understanding of AI as soon as possible and conduct model training. An example of this is the fan poster of “”Ghost Slayer: Blade”” generated by Unbounded AI at the beginning of this month, which sparked heated discussions among fans due to its beauty and restoration of Kamado Tanjiro’s design. AI integration into animation production is an inevitable trend, and traditional painters should embrace it and use it to their advantage.
Unbounded AI has developed a test model to restore the characters of Ganlu Temple Mili from the animation series. However, the restoration of the characters is relatively worse due to the lack of training materials and the model not being released yet. The model works by integrating physical features of characters into one entry, so that users only need to input the keyword of the character to generate an image. This model can help improve the accuracy of the character image generation, but the image still needs to be retouched manually.
AI painting is a new technology that uses machine learning algorithms to generate art. It has the potential to revolutionize the animation industry by allowing directors to quickly test ideas and allowing art executives to quickly produce paintings that satisfy the director’s needs. Zhao Jiecheng believes that AI can help art demanders clarify their vision and creative intentions. AI painting can also be used to quickly produce sequels of popular works, such as “Hunter x Hunter”, allowing fans to get their desired content faster. The key to distinguishing “”painting skills”” among AI painters in the future may depend on how well they polish their models. AI painting has the potential to revolutionize the animation industry and help art demanders clarify their vision and creative intentions.
AI is transforming the traditional animation production process, allowing producers to create new types and styles of products at a lower cost. Pixar’s “Toy Story” was the first to use 3D animation, a digital transformation of traditional stop-motion animation. Similarly, Japanese 2D hand-drawn animation has been transformed into digital production due to the use of computer technology. AI is also transforming the animation production process in terms of technology, cost, and thinking. AI may allow producers to experiment with new ideas and styles of animation at a lower cost, creating a new wave of animation production.
