Show HN: Spark, An advanced 3D Gaussian Splatting renderer for Three.js

Thanks. We have to definitely add an FAQ

There's a community of people passionate about scanning all short stuff with handheld devices, drones... Tipatat let us generously use his food scans for the demo. I also enjoy kotohibi flower scans: https://superspl.at/user?id=kotohibi

Edit: typos

And the transfer size for that level of detail isn't that bad, either - only around 80MB. (Not being sarcastic, it's really neat.)

BabylonJS and the OP's own Aframe [1] seem to have similar licenses, similar number of Github stars and forks, although Aframe seems newer and more game / VR focused.

How do Babylon, Aframe, Three.js, and PlayCanvas [2] compare from those that have used them?

IIUC, PlayCanvas is the most mature, featureful, and performant, but it's commercial. Babylon is the featureful 3D engine, whereas Three.js is fairly raw. Though it has some nice stuff for animation, textures, etc., you're really building your own kit.

Any good experiences (or bad) with any of these?

OP, your demo is rock solid! What's the pitch for Aframe?

How do you see the "gaussian splat" future panning out? Will these be useful for more than visualizations and "digital twins" (in the industrial setting)? Will we be editing them and animating them at any point in the near future? Or to rephrase, when (or will) they be useful for the creative and gaming fields?

[1] https://github.com/aframevr/aframe

[2] https://playcanvas.com/

Number of splats in the scene and distribution have an impact on performance. Probably in your case you turned the camera in a direction with less splats. There's definitely work to do to deliver consistent performance. We'll probably look into an LOD system next.

Thanks! Notice 12MB steak sandwich is the biggest of them all. Rest are < 10MB and several of those very compelling in the 1-3MB range (e.g: Iberico Sandwich 1MB, Clams and Caviar 1.8MB).

Fancier compression methods are coming (e.g SOGS). This is 30MB!

https://vincentwoo.com/3d/sutro_tower/

The SOGS compression technique works well. You can get 1M Gaussians with full spherical harmonics in about 14MB. There's a good article about it on the PlayCanvas blog:

https://blog.playcanvas.com/playcanvas-adopts-sogs-for-20x-3...

Large file sizes are mostly to store spherical harmonics coefficients which is a fixable problem.

With any demo / example in particular?

You definitely could prototype something like that. Would be really cool to see.

It does work in VR! We'll have a demo available soon.

Don't forget SPARC: https://en.wikipedia.org/wiki/SPARC

PartCAD has a library of 3D parts which might be useful for testing a different renderer.

The OCP CAD viewer could optionally use this Gaussian splatting renderer (because it's also already built on Three.js)

A-Frame integration coming soon!

Strange. I also have an iPhone I haven’t seen any issues. Are you using a different browser than Safari or any particular configuration?

Yes, Spark does instanced rendering of quads, one covering each Gaussian splat. The sorting is done by 1) calculating sort distance for every splat on the GPU, 2) reading it back to the CPU as float16s, 3) doing a 1-pass bucket sort to get an ordering of all the splats from back to front.

On most newer devices the sorting can happen pretty much every frame with approx 1 frame latency, and runs in parallel on a Web Worker. So the sorting itself has minimal performance impact, and because of that Spark can do fully dynamic 3DGS where every splat can move independently each frame!

On some older Android devices it can be a few frames worth of latency, and in that case you could say it's amortized over a few frames. But since it all happens in parallel there's no real impact to the overall rendering performance. I expect for most devices the sorting in Spark is mostly a solved problem, especially with increasing memory bandwidth and shared CPU-GPU memory.

Thanks! It works for both! An under-explored area is converting into splats assets created in “traditional” ways (e.g blender). Better visual results in some scenarios (high freq detail). See the furry logo in the homepage carousel.

Nothing top of mind like that. Check out people's scans at https://superspl.at/ Anything there should also render in Spark.

We have a template for R3F you might find useful:

https://github.com/sparkjsdev/spark-react-r3f

obj is traditional geometry (vertices, triangles). gaussian splats is a different way to represent 3D information (simplifying. it's a point cloud where each point it's an ellipsoid with view dependent color)

The WebGL API is based on the OpenGL ES standard, which jettisoned a lot of the procedural pipeline calls that made it easy to write CPU-bound 3D logic.

The tradeoff is initial complexity (your "hello world" for WebGL showing one object will include a shader and priming data arrays for that shader), but as consequence of design the API sort of forces more computation into the GPU layer, so the fact JavaScript is driving it matters very little.

THREE.js adds a nice layer of abstraction atop that metal.

Just WebGL2

We're definitely looking at it. No specific plans yet

Yes. We have demos working already. Those 3D gaussian videos (or 4D that some people call) are really big so we're figuring out what's the best way to distribute and make it a great experience.

Yes. I collaborated with one of the devs at World Labs on this. The goal is to explore new rendering techniques and popularize adoption of 3D gaussian splatting. There's no product associated with it.

We renamed due to a name collision with another renderer / tool.