Comment by silotis
Comment by silotis 2 days ago
This depends heavily on what problem domain you're talking about. For example, a DBMS is necessarily going to shuffle a lot of data into and out of memory.
Comment by silotis 2 days ago
This depends heavily on what problem domain you're talking about. For example, a DBMS is necessarily going to shuffle a lot of data into and out of memory.
I was thinking less about the DB data itself and more about temporary allocations that have to be made per-request. The same is true for most server software. Even if arenas are used to reduce the number of allocations you're still doing a lot more memory management than a typical cryptographic benchmark.
Most databases do almost no memory management at runtime, at least not in any conventional sense. They mostly just DMA disk into and out of a fixed set of buffers. Objects don't have a conventional lifetime.
It depends. Consider DuckDB or another heavily vectorized columnar DB: there's a big part of the system (SQL parser, storage chunk manager, etc.) that's not especially performance-sensitive and a set of tiny, fast kernels that do things like predicate-push-down-based full table scans, ART lookups, and hash table creation for merge joins. DuckDB is a huge pile of C++. I don't see a RIIR taking off before AGI.
But you know what might work?
Take current DuckDB, compile it with Fil-C, and use a new escape hatch to call out to the tiny unsafe kernels that do vectorized high-speed columnar data operations on fixed memory areas that the buffers safe code set up on behalf of the unsafe kernels. That's how it'd probably work if DuckDB were implemented in Rust today, and it's how it could be made to work with Fil-C without a major rewrite.
Granted, this model would require Fil-C's author to become somewhat less dogmatic about having no escape hatches at all whatsoever, but I suspect he'll un-harden his heart as his work gains adoption and legitimate use-cases for an FFI/escape hatch appear.
> DuckDB is a huge pile of C++. I don't see a RIIR taking off before AGI.
While I'm not a big fan of rewriting things, all of DuckDB has been written in the last 10 years. Surely a rewrite with the benefit of hindsight could reach equivalent functionality in less than 10 years?
the sqlite RIIR is going quite well: https://turso.tech/blog/beyond-the-single-writer-limitation-...
(sqlite is quite a bit smaller than DuckDB tho)
I am professional database developer. We do not do what you are thinking we are doing. ;)