Abstract

3D scene registration is a fundamental problem in computer vision that seeks the best 6-DoF alignment between two scenes. This problem was extensively investigated in the case of point clouds and meshes, but there has been relatively limited work regarding Neural Radiance Fields (NeRF). In this paper, we consider the problem of rigid registration between two NeRFs when the position of the original cameras is not given. Our key novelty is the introduction of Viewshed Fields (VF), an implicit function that determines, for each 3D point, how likely it is to be viewed by the original cameras. We demonstrate how VF can help in the various stages of NeRF registration, with an extensive evaluation showing that VF-NeRF achieves SOTA results on various datasets with different capturing approaches such as LLFF and Objaverese.

VF-NeRF Optimization process

Visualization of the VF-NeRF registration optimization process on the Trex scene from LLFF dataset:

Qualitative Results

Visualization of the VF-NeRF registration results on our captured Table scene and the Horns scene from LLFF dataset:

VF-NeRF Based Point Clouds

A demonstration of our VF-based point clouds, generated using the normalizing-flows invert direction:

Viewshed Fields Visualization

Visualization of the viewshed fields on the Trex scene from LLFF dataset, note the effect of view direction on the VF:

Quantitative Results

We compare our method to previous NeRF registration and point-cloud registration methods. We show here the results over Objaverse dataset as described in the paper:

Citation


The website template was borrowed from Dor Verbin.