Chapter 01 of 04
The brief
A scanned home shows one light only: the light of the hour it was scanned.
A buyer cannot see the living room at sunset. This is a month of trying to change that: per-Gaussian relighting driven by a real solar ephemeris, so the sun crossing the scene moves the way the sun actually moves, plus three other approaches that failed and the numbers that closed them. You can drag the sun across the live scan below. What you cannot do is remove the shadow that was already on the wall, and most of the work here is about why.
A Gaussian splat is built from photographs, so it inherits their light. The scan holds radiance: the colour a surface had under the lamps and sky that happened to be there. Relighting wants the other thing, albedo, the colour a surface would be under any light at all. Nothing in the capture separates the two, and every method tried here ran into that in a different way.
The practical consequence is easy to state and hard to get around. Multiply the captured colour by a new sun and the room is lit twice: once by the light that was there and once by the light you just added. The shadow that fell across the wall at capture time stays on that wall at every hour you dial up, because it is not a shadow any more, it is paint.
The reason to care is commercial rather than academic. A scan of a residence sold at a fixed hour is a scan that shows badly at every other hour, and the fix a client asks for first is the sun.

Chapter 02 of 04
The problem
Take a 3D scan whose lighting is frozen at the hour it was captured, and let someone move the sun across it in a browser without the room falling apart.
Chapter 03 of 04
The build
- 3D & Spatial
- Relit millions of Gaussians individually from the scan's own geometry, rather than lighting a proxy mesh and hoping the splats followed.
- Research
- Framed the question so it could come back negative, and it did: three of four method families failed, each closed by a number rather than by a hunch, and each logged as do-not-retry with the reason.
- Frontend Engineering
- Solved the ordering problem that kept per-Gaussian normals out of the web viewer, so the demo on this page is the technique and not a mock-up of it.
- Data Pipelines
- Decoded the proprietary capture format from its published spec, since nothing off the shelf would read it into the engine.
Build notes. How it came together and the decisions on the record.
Chapter 04 of 04
What it did
The loop holds end to end: a real capture, a real sun, real-time relighting, in an ordinary browser with no install.
The limits are real too. A scan records radiance, not albedo, so the light that was in the room at capture time is already in the data and relighting adds to it rather than replacing it. Inverse rendering was abandoned after two retrains (the scores are under Decisions). Learned delighting failed a cross-view consistency gate at a median albedo standard deviation of 0.223, and produced, in the same pass, the single frame in the whole corpus with the baked cast shadow genuinely removed. Two real-time cast-shadow approaches failed from opposite directions, one reading occlusion off a proxy mesh too coarse to carry it and one off per-Gaussian normals too noisy to trust, so the shipped default is a movable light with cast shadow off. PlayCanvas has since shipped that same proxy-mesh route as a documented engine feature and names the same limit in its own guidance, which is the useful kind of confirmation: the constraint is in the technique, not in this implementation of it.
What ships is a mood system that is honest about being one: a real sun, a real per-Gaussian response, and no claim to have removed light that is already in the data. The negative results are the more useful half. Two abandoned retrains, a delighting gate failed at 0.223, and two cast-shadow approaches that failed from opposite directions together say something a working demo cannot, which is where the boundary actually sits and what it would take to move it. The next move is a recapture under flat light and a run against the standard relighting benchmarks, so these numbers can be compared with published work instead of only with each other. The method to test against is no longer the one abandoned here: relighting has since moved feed-forward, learning the change of lighting across many scenes rather than solving for it inside one, which is a direct answer to why the retrains failed.
- 17.7M
- Largest capturethe outdoor site, in one continuous scan; the demo above is a smaller interior
- Real sun
- Light sourcesolar ephemeris, so azimuth and altitude both move
Research origin.
[ Relighting baked light ]
A scan records the light that was there. Changing it later is a different problem from rendering it.
[ 3D Gaussian Splatting ]
Turning ordinary photographs of a real place into a 3D scene you can walk through in a browser.




