T2C: Text-guided 4D Cloth Generation

Zhipeng Yu1, Zimeng Zhao1, Yanxi Du1, Yuzhou Zheng1, Binghui Zuo1, Yangang Wang1
1SouthEast University
Interpolate start reference image.

Clothed 4D humans generated by our T2C from free-form user descriptions. The transition of human skin from light to dark indicates a change in time.

Abstract

In the age of AIGC, the creation process is increasingly automated. Generating vivid characters with clothing and motions according to scripts or novels is no exception. Unfortunately, the diversity of fabric topologies, the complexity of fabric layering, and the flexibility of fabric motion make most approaches only applicable to motion generation for characters in undressing or tight-fitting clothing. This paper introduces a novel approach named \textbf{T2C}, which employs a multilayered clothing representation and a physics-based clothing animation paradigm to generate text-controlled Clothed 4D Humans, expanding the boundaries of the aforementioned issues. The hierarchical representation of clothing utilizes Fourier spherical mapping to define the geometric information of garments within a standard pose space, mapping it onto several two-dimensional frequency domain subspaces.

The motion of clothing in tandem with the human body is realized through a hybrid forward dynamic solution, where the internal virtual mechanic's parameters driving the clothing are learned from text features. A series of qualitative and quantitative experiments reveal that T2C can generate dynamic clothing with a sense of layering, realistic details, and rich textures.

Video