A collaboration friendly studio for NeRFs

• Quickstart
• Learn more
• Supported Features

About

What is different between original repo and this fork?

1. This fork has ns-process-data splatfacto
2. This fork uses all images for training. But, eval images is splitted by fraction or interval. Default settings is fraction. Eval images is 0,1 of all images.

Why add this ns-process-data splatfacto?

It mimics python3 convert.py -s /path/to/input/folder from Original Inria 3DGS

What is special things about python3 convert.py?

1. original ns-process-data images make "images" in output folder become smaller despite has same resolution, it hurts quality.
2. It uses exhausive_matcher instead of vocab_tree_matcher in ns-process-data images
3. it uses hyperparameter of ba_global_function_tolerance = 0.000001
4. It uses colmap image_undistorter

So what I get if I use that?

I got extra quality boost, about 0.3 - 0,7 dB in PSNR evaluation of all images. I have tested it on different dataset.

What is different between this implementation and python3 convert.py.?

1. The original convert.py gives result of colmap format which does not have transforms.json and sparse_pc.json. This implementation creates that files so it compatible with NerfstudioDataparser.
2. The original convert.py uses magick command for resizing images. This repo uses OpenCV for resizing images.

Why uses all training images?

It mimics INRIA 3DGS which uses all images for training data.

Will it be overfitting?

Yes, thats okay for me. Overfitting is not problem because we don't generate entirely new scene. But we must maximize the existing scene quality.

How about quality?

It increases from 28,7 dB to 29,2 dB PSNR (extra 0,5 dB) for apartement eyeful tower 1k JPEGs dataset. Another dataset is not yet tested. Hopefully in short of time.

You can NOT compare the metrics since you have trained all images included eval images.

Yes it will be biased since eval images already leaks into training dataset. But this repo is intended for END USER which does not care about PSNR, SSIM, etc. They care only the end product.

Is it compatible with original Nerfstudio?

If you resume the training from original nerfstudio's checkpoint = No, it will be error because different count of training dataset.
If you train from ns-process-data images/videos/odm/realitycapture/metashape etc from original nerfstudio = Yes, it is compatible

INSTALLATION

# make sure you uninstall the previous gsplat and nerfstudio
pip uninstall gsplat nerfstudio

# install it from pip
pip install git+https://github.com/nerfstudio-project/[email protected]
pip install git+https://github.com/ichsan2895/[email protected]

Tested on Torch 2.0.1+cu118 and Python 3.10 in Ubuntu 22.04 LTS