Rendered images on the UVS dataset using U2NeRF

Abstract

Underwater images suffer from colour shifts, low contrast, and haziness due to light absorption, refraction, scattering and restoring these images has warranted much attention. In this work, we present Unsupervised Underwater Neural Radiance Field (U2NeRF), a transformer-based architecture that learns to render and restore novel views conditioned on multi-view geometry simultaneously. Due to the absence of supervision, we attempt to implicitly bake restoring capabilities onto the NeRF pipeline and disentangle the predicted color into several components - scene radiance, direct transmission map, backscatter transmission map, and global background light, and when combined reconstruct the underwater image in a self-supervised manner. In addition, we release an Underwater View Synthesis (UVS) dataset consisting of 12 underwater scenes, containing both synthetically-generated and real-world data. Our experiments demonstrate that when optimized on a single scene, U2NeRF outperforms several baselines by as much as LPIPS 11%↓, UIQM 5%↑, UCIQE 4%↑ (on average) and showcases improved rendering and restoration capabilities. The multi-view inductive prior enables U2NeRF to be trained on multiple scenes, and successfully transfer to unseen scenes with or without per-scene fine-tuning.

Overview of U2NeRF

Single Scene Rendering Results

Qualitative comparison for single-scene rendering. U2NeRF recovers and restores underwater images with greater quality, comparable and even better than non-rendering baselines UIESS, UPIFM.

Physical Interpretation of U2NeRF

Visualisations of the predicted image components (scene radiance, transmission maps, global light).