Suppose several photographs show the same scene from different viewpoints. Novel-view synthesis asks what a camera would see from a position between those photographs. 3D Gaussian Splatting approaches this problem with an explicit collection of adjustable scene primitives.
What Is Stored in a Gaussian?
In the original 3DGS method, a primitive has a position, a spatial covariance, opacity, and view-dependent color represented with spherical harmonics. Its footprint can stretch and rotate: the representation is more expressive than a set of identical points. It is also different from a triangle mesh; a Gaussian does not define a hard surface boundary.
A useful mental model is a small translucent cloud. Its contribution is strongest near its center and falls off away from it. Many overlapping clouds can approximate visible scene structure. That analogy explains the rendering behavior, but does not imply that the recovered clouds are a physically correct model of the photographed material.
From Photographs to a Renderable Scene
The original method combines initialization from calibrated cameras and sparse points with optimization and density control. A simplified outline is:
- Initialize Gaussians from the sparse reconstruction.
- Project them into a training camera and render an image.
- Compare that rendering with the corresponding photograph and update parameters.
- Adjust the population of Gaussians as optimization proceeds.
The visibility-aware renderer projects the primitives to image-space footprints and composites their contributions in depth order. The representation and renderer work together; a collection of Gaussians by itself does not guarantee fast rendering.
A Pixel Is a Weighted Combination
Consider a pixel covered by a red foreground splat and a green background splat. Give them effective alpha values of 0.5 and 0.25 at that pixel. The red contribution is 0.5. Only half the light remains available after the red splat, so the green contribution is 0.5 × 0.25 = 0.125. With a black background, the final RGB value is [0.5, 0.125, 0].
# Teaching example: effective alpha values at one pixel.
# Splats are already ordered from front to back.
splats = [([1.0, 0.0, 0.0], 0.50),
([0.0, 1.0, 0.0], 0.25)]
color = [0.0, 0.0, 0.0]
transmittance = 1.0
for rgb, alpha in splats:
for channel in range(3):
color[channel] += transmittance * alpha * rgb[channel]
transmittance *= 1.0 - alpha
# Black background: color = [0.5, 0.125, 0.0]
# For another background, add transmittance * background.
This short example starts after projection and sorting; it is not a 3DGS renderer. In an actual renderer, effective alpha varies across a splat's footprint. The ordering matters too: placing green in front gives a different result. This is why visibility handling is central to the method rather than an optional cosmetic step.
Design a Small, Honest Experiment
Start with a static scene and overlapping photographs. Hold out some camera views before training. Keep those held-out images out of both optimization and parameter selection. Otherwise, a good-looking reconstruction can simply demonstrate that the model fitted the images it already saw.
After installing the dependencies from the official implementation and preparing a supported dataset, its evaluation workflow can be organized as follows. Replace the example paths with your own directories:
python train.py -s /path/to/scene -m output/scene --eval
python render.py -m output/scene
python metrics.py -m output/scene
The --eval flag enables the implementation's training/test split. Record the repository revision, image resolution, GPU, training settings, and number of Gaussians. Compare held-out quality alongside model size and rendering time. Use the same evaluation cameras when comparing two settings, and distinguish training memory from inference memory.
Where the Simple Story Breaks Down
Missing viewpoints leave parts of a scene weakly constrained. Moving objects can also make photographs inconsistent with a single static reconstruction. Inspect difficult regions directly: a scene-wide score can hide an isolated floating primitive, an unstable reflection, or a thin structure that disappears from a new angle.
Image scale is another variable. Mip-Splatting studies artifacts that appear when the sampling rate changes, such as when camera distance or focal length changes, and introduces filtering to address them. A result that looks good at the training scale should therefore also be checked at other viewing scales.
For an optimization such as quantization, first establish a reproducible baseline. Change one setting at a time, then inspect where quality was lost and what resource was saved. A smaller file is useful, but a convincing claim also needs to explain its effect on the rendered result and the actual execution cost.