Comment by ndiddy

That's a valid, but very tiny, use case.

After all, if you can tell in advance that you might hit the limits of SQLite, you'd simply start with postgresql on day one, not with a new unproven DB vendor with a product that has been through the trial by fire of existing DBs.

So the usecase is: I started with SQLite, but now I have too many terrabytes to fit on one server? That seems.. very uncommon.

And since moving it out of process, and even to another network, is going to make it much much much slower. You're going to need a rewrite anyway

Thanks. Serves me right for commenting without reading the article.

Yeah that's something I actually use quite a bit!

Man, I've seen the SQL Metabase emits, it's not great. Like, doing a massive join across 10 tables and selecting all the columns from all the tables - to only return the average of one column from one table.

I don't understand you reply here. Database startups have always had the consistent issue of cloud providers providing managed solutions without contributing back. It is why many moved to or use the AGPLv3 and why there was the whole SSPL controversy in the first place. Running a successful open source database startup is not trivial. None of this applies to SQLite.

1. Only if you modify it. There is a free test suit, and You can license the non-free test suit.

2. Compare to the test in Turso, the test in Turso is just kids toy.

Yep, exactly this.

It's painful having to switch to another language to talk to the database, and ORMs are the worst kind of leaky abstractions. With Rust, we've finally got a systems language that's expressive enough to do a really good job with the API to an embedded database.

The only thing that's really missing is language support for properly ergonomic Cap'n Proto support - Swift has stuff in this vein already. That'd mean serializable ~native types with no serialization/deserialization overhead, and it's applicable to a lot of things; Swift developed the support so they could do proper dynamically linked libraries (including handling version skew).

If I might plug my project yet again (as if I don't do that enough :) - bcachefs has a high quality embedded database at its core, and one of the dreams has always been to turn that into a real general purpose database. Much of the remaining stuff for making it truly general purpose is stuff that we're going to want sooner or later for the filesystem anyways, and while it's all C today I've done a ton of work on refactoring and modernizing the codebase to hopefully make a Rust conversion tractable, in the not too distant future.

(Basically, with the cleanup attribute in modern C, you can do pseudo RAII that's good enough to eliminate goto error handling in most code. That's been the big obstacle to transitioning a C codebase to be "close enough" to what the Rust version would look like to make the conversion mostly syntactic, not a rewrite, and that work is mostly done in bcachefs).

The database project is very pie in the sky, but if the project gets big enough (it's been growing, slowly but steadily), that's the dream. One of them, anyways.

A big obstacle towards codebases that we can continue to understand, maintain and continue to improve over the next 100 years is giant monorepos, and anything we can do to split giant monorepos apart into smaller, cleaner reusable components is pure gold.

I vaguely remember a crate doing a RocksDB kind of thing?