The VAE component of TripoSF, a new model by Tripo (https://www.tripo3d.ai) for high-resolution 3D shape modeling.
The core idea is SparseFlex, a sparse voxel representation based on Flexicubes. Instead of dense grids, it only uses voxels near the surface. This massively reduces memory, enabling reconstruction up to 1024³ resolution directly supervised by differentiable rendering (avoids detail loss from watertight preprocessing).
Key features enabled by SparseFlex + the training approach:
- High Resolution: Up to 1024³ output.
- Arbitrary Topology: Naturally handles open surfaces (like cloth, plants) and complex internal structures without needing watertight meshes.
- Interior Reconstruction: The "Frustum-Aware Sectional Voxel Training" allows reconstructing internal details using only rendering loss by rendering from inside the object during training.
TripoSF – 1024³ 3D VAE Using SparseFlex for Arbitrary Topology · HackerTrans
Key features enabled by SparseFlex + the training approach: - High Resolution: Up to 1024³ output. - Arbitrary Topology: Naturally handles open surfaces (like cloth, plants) and complex internal structures without needing watertight meshes. - Interior Reconstruction: The "Frustum-Aware Sectional Voxel Training" allows reconstructing internal details using only rendering loss by rendering from inside the object during training.