NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints, differentiable camera API, differentiable lighting with spherical harmonics and spherical gaussians, powerful quadtree acceleration structure called Structured Point Clouds, interactive 3D visualizer for jupyter notebooks, convenient batched mesh container and more. Visit the Kaolin Library Documentation to get started!

Note that Kaolin library is part of the larger NVIDIA Kaolin effort for 3D deep learning.

Starting with v0.12.0, Kaolin supports installation with wheels:

# Replace TORCH_VERSION and CUDA_VERSION with your torch / cuda versions
pip install kaolin==0.18.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-{TORCH_VERSION}_cu{CUDA_VERSION}.html

For example, to install kaolin 0.18.0 over torch 2.8.0 and cuda 12.8:

pip install kaolin==0.18.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.8.0_cu128.html

In this version we added collisions to our physics module.

We also added a conversion from 3D gaussians to voxelgrids, that we are using in our Densifying function to improve the accuracy.

We have a new tutorial notebook showing physics simulation with mesh and gaussiansin a single scene and rendering using 3DGRUT.

Check our updated tutorials:

• Use Simplicit's Easy API to Simulate a Mesh
• Simulate a Gaussian Splat

Flexicubes license got changed to Apache v2.0, which is making it usable for commercial purposes. See the new location.

See change logs for details.

Please review our contribution guidelines.

• NVIDIA Kaolin Wisp:
  • Use Camera API, Structured Point Clouds and its rendering capabilities
• gradSim: Differentiable simulation for system identification and visuomotor control:
  • Use DIB-R rasterizer, obj loader and timelapse
• Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer:
  • Use Kaolin's DIB-R rasterizer, camera functions and Timelapse for 3D checkpoints.
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Surfaces:
  • Use SPC conversions and ray-tracing, yielding 30x memory and 3x training time reduction.
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction:
  • Use Kaolin's DefTet volumetric renderer, tetrahedral losses, camera_functions, mesh operators and conversions, ShapeNet dataset, point_to_mesh_distance and sided_distance.
• Text2Mesh:
  • Use Kaolin's rendering functions, camera functions, and obj and off importers.
• Flexible Isosurface Extraction for Gradient-Based Mesh Optimization (FlexiCubes) :
  • Use Flexicube class, obj loader, turntable visualizer
• SATR:
  • Use Kaolin's rendering functions, lighting functions, camera functions, and obj/off importers.

Most of Kaolin's repository is under Apache v2.0 license, except under kaolin/non_commercial which is under NSCL license restricted to non commercial usage for research and evaluation purposes.

Default kaolin import includes Apache-licensed components:

import kaolin

The non-commercial components need to be explicitly imported as:

import kaolin.non_commercial

FlexiCubes is now under Apache-v2 here, the old version is maintained for backward compatibility

If you are using Kaolin library for your research, please cite:

@software{KaolinLibrary,
      author = {Fuji Tsang, Clement and Shugrina, Maria and Lafleche, Jean Francois and Perel, Or and Loop, Charles and Takikawa, Towaki and Modi, Vismay and Zook, Alexander and Wang, Jiehan and Chen, Wenzheng and Shen, Tianchang and Gao, Jun and Jatavallabhula, Krishna Murthy and Smith, Edward and Rozantsev, Artem and Fidler, Sanja and State, Gavriel and Gorski, Jason and Xiang, Tommy and Li, Jianing and Li, Michael and Lebaredian, Rev},
      title = {Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research},
      date = {2024-11-20},
      version = {0.18.0},
      url={\url{https://github.com/NVIDIAGameWorks/kaolin}}
}

Current Team:

• Technical Lead: Clement Fuji Tsang
• Manager: Maria (Masha) Shugrina
• Charles Loop
• Vismay Modi
• Or Perel
• Alexander Zook

Other Majors Contributors:

• Wenzheng Chen
• Sanja Fidler
• Jun Gao
• Jason Gorski
• Jean-Francois Lafleche
• Rev Lebaredian
• Jianing Li
• Michael Li
• Krishna Murthy Jatavallabhula
• Artem Rozantsev
• Tianchang (Frank) Shen
• Edward Smith
• Gavriel State
• Towaki Takikawa
• Jiehan Wang
• Tommy Xiang