Loading
Research · Thread 03
Mirrors and glass capture
The open problem Gaussian Splatting still hasn't solved for high-end interiors.
Splatting reconstructs surfaces from photographs, but a mirror or a floor-to-ceiling glass wall is a reflection rather than a surface, so pipelines either ghost the reflected room into the wrong volume or hollow out the geometry behind the glass. In luxury real estate, where mirrors and glass walls are everywhere, that is the bottleneck rather than an edge case. The primary bet is handling it when the scene is captured rather than in the renderer, though a render-side reflection branch is in progress too and has reached a first convergent result on a real reflective showroom scan.
Findings
The reflection problem, unsolved
A mirror is a reflection, not a surface, splatting can't reconstruct it. Pipelines ghost the reflected room, or hollow out the wall behind it. In luxury interiors, that's most of the room.
Mirrors, glass and chrome are the dominant artefact class on hard scans.
A symmetry trick for mirrors
MirrorGaussian models the real and mirror worlds at once, exploiting the symmetry across the mirror plane. Promising on paper. Not yet plug-and-play for production capture.
The primary bet sits at the photogrammetry layer, not the rasteriser alone.
Mitigate before COLMAP
The primary fix is capture-side: mask-and-inpaint before COLMAP, dual-pass shoots, controlled lighting. Solve it where the bad data enters. A render-side reflection branch is in progress alongside.
The primary fix moves upstream of reconstruction.
Paper to shipped
Reference
MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror Reflections (Liu, Tang, Cheng, Yang, Xiong, Ma, ECCV 2024)A leading attempt at handling mirrors in 3DGS by jointly modelling the real and reflected geometry. Promising on benchmarks; not yet plug-and-play for production capture.
View abstract
The key insight is that a mirrored scene is symmetric to the real scene with respect to the mirror plane. Building on this, the authors introduce a dual-rendering strategy that models the real-world and the mirror-world simultaneously, with an intersection-aware mirror Gaussian formulation that handles partial reflections. Demonstrated on real-world scenes containing mirrors with state-of-the-art reconstruction quality vs. prior 3DGS variants.
In progress
Capture-side mitigation pipelineInside the benchmark, the per-engine failure analysis names mirrors / glass / chrome as the dominant artefact class on the Hard tier. The primary mitigation candidates are at the photogrammetry layer (mask + inpaint pre-COLMAP, dual-pass capture, controlled-lighting subsets); in parallel, a render-side reflection branch has its first convergent result on a real reflective showroom scan.