LandShop

Weihao Yuan1, Zhe Sheng1, Yisheng He1, Weichao Shen1, Yang Gao1, Kejie Qiu1, Xiaodong Gu1, Chao Xu2, Xiaodan Ye2, Junfei Zhang2, Zilong Dong, Liefeng Bo
Institute for Intelligent Computing (通义实验室), Alibaba Group
1AI Algorithm   2TIDE Rendering

Abstract

MY ALT TEXT

LandShop is a Large-scale Neural Digital Twin System, which enables Photorealistic Reconstruction, LOD Textured Mesh Reconstruction, Neural Editing, and Neural Segmentation.

1. Large-scale Photorealistic Reconstruction

We build a large-scale photorealistic reconstruction system based on large-scale SfM and large-scale neural rendering technologies.

2. Large-scale LOD Textured Mesh Reconstruction

Although the neural rendering could reconstruct a photo-realistic 3D scene, it is hard to be embedded into the traditional industries, e.g., oblique photography. These traditional industries have grown for several decades, and are already stable with the mesh as representation. To link to them, we add the geometry constraint to the neural reconstruction, and then extract the neural fields to a traditional mesh. Afterwards, we develop the large-scale texture mapping and level-of-details (LOD) technologies, which together enable our framework to be embedded into the traditional rendering engine. We also develop a rendering engine, named UniTIDE, for efficient large-scale rendering.

3. Large-scale Neural Editing

Based on the reconstructed 3D scene, we edit this scene to different seasons, time, weather, and so on.

For more technical details and more editing effects, please refer to our paper:
Freditor: High-Fidelity and Transferable NeRF Editing by Frequency Decomposition



4. Large-scale Neural Segmentation

Based on the reconstructed 3D scene or an arbitrary mesh of a 3D scene, we segment this scene to different areas.