I'm still highly skeptical of gaussian splatting as anything more than a demo. The files are too large. The steak sandwich is 12meg (as just one example)
There was a guassain splat based Matterport port clone at least year's siggraph. To view a 2 bedroom apartment required streaming 1.5gig
Cool demo
How much of the huge file size is because you need tons of splats to simulate a hard surface? Conceptually the splats seems flawed because gaussians don't have hard edges - they literally go to infinity in all directions, just at vanishingly small densities. So practically everybody cuts them off at 3 sigma or something, which covers 99.7% of the volume. But real-world objects have hard edges, and splats don't.
Would the format work better if you made that cut-off at something like 1 sigma instead? Then instead of these blurry blobs you'd effectively be rendering ovals with hard edges. I speculate out loud that maybe you could get a better render with fewer hard-edged ovals than tons of blurry blobs.
It's an interesting idea, and with spark you could test this by adjusting the parameter maxStdDev to control how far out it draws the splat.
I agree with you though that in general 3DGS is a worse representation for hard, flat, synthetic things with hard edges. But in the flip side, I would argue it's a better representation for many organic, real-world things, like imagine fur or hair or leaves on a tree... These are things that can render beautifully photo realistically in a way that would require much, much more complex polygon geometry and texturing and careful sorting and blending of semi-transparent texels. This is one reason why 3DGS has become so popular in scanning and 3D reconstruction.. you just get much better results with smaller file sizes. When 3DGS first appeared, everyone was shocked by how photorealistic you could render things in real time on a mobile device!
But one final thought I want to add: with Spark it's not an either/or. You can have BOTH in the same Three.js scene and they will blend together perfectly via the Z-buffer. So you can scan the world around you and render it with 3DGS, and then insert your hard-edged robot character polygon meshes right into that world, and get the best of both!
Cool - thanks for explaining that. I totally see how each has its place.
I imagine it's pretty complex to take the raw scan data and generate 3dgs. Are these algorithms simple & standard, or do they take a fair amount of tuning & tweaking to do a good job? Adapting these to work well with hard-edge ovals seems like it would take some work, and a lot more work to get them to output a mix of ovals & fuzzy blobs. But if you could do that, I agree the combination would be amazingly expressive.
There are a lot of tools to do this easily today, for free! Take a look at Postshot, or Brush. You can literally take a video with your mobile phone, toss it in Postshot, and a few minutes later you have a photorealistic 3DGS model you can use in Spark!
3DGS is still a rapidly evolving research field, but the "baseline" is pretty much standard these days.
There was an interesting preprint (and code release) recently on triangle splatting, you may find this interesting:
The SOGS compression technique works well. You can get 1M Gaussians with full spherical harmonics in about 14MB. There's a good article about it on the PlayCanvas blog:
https://blog.playcanvas.com/playcanvas-adopts-sogs-for-20x-3...
Large file sizes are mostly to store spherical harmonics coefficients which is a fixable problem.
Thanks! Notice 12MB steak sandwich is the biggest of them all. Rest are < 10MB and several of those very compelling in the 1-3MB range (e.g: Iberico Sandwich 1MB, Clams and Caviar 1.8MB).
Fancier compression methods are coming (e.g SOGS). This is 30MB!
https://vincentwoo.com/3d/sutro_tower/