> DuckDB is a huge pile of C++. I don't see a RIIR taking off before AGI.
While I'm not a big fan of rewriting things, all of DuckDB has been written in the last 10 years. Surely a rewrite with the benefit of hindsight could reach equivalent functionality in less than 10 years?
I'm trying to make Turso load some data, but it is so slow that even several months are not enough to load the dataset: https://github.com/ClickHouse/ClickBench/issues/336
for one, duckdb includes all of sqlite (and many other dependencies). it knows how to do things like efficiently query over parquet files in s3. it's expansive - a swiss army knife for working with data wherever it's at.
sqlite is a "self contained system" depending on no external software except c standard library for target os:
> A minimal build of SQLite requires just these routines from the standard C library:
> memcmp(), memcpy(), memmove(), memset(), strcmp(), strlen(), strncmp()
> Most builds also use the system memory allocation routines:
> malloc(), realloc(), free()
> Default builds of SQLite contain appropriate VFS objects for talking to the underlying operating system, and those VFS objects will contain operating system calls such as open(), read(), write(), fsync(), and so forth
Quoting from the appropriately named https://sqlite.org/selfcontained.html
as a very rough and unfair estimate between the two project's source, sqlite is about 8% the size of duckdb:
$ pwd
/Users/jitl/src/duckdb/src
$ sloc .
---------- Result ------------
Physical : 418092
Source : 317274
Comment : 50113
Single-line comment : 46187
Block comment : 3926
Mixed : 4415
Empty block comment : 588
Empty : 55708
To Do : 136
Number of files read : 2611
----------------------------
$ cd ~/Downloads/sqlite-amalgamation-3500400/
$ sloc .
---------- Result ------------
Physical : 34742
Source : 25801
Comment : 8110
Single-line comment : 1
Block comment : 8109
Mixed : 1257
Empty block comment : 1
Empty : 2089
To Do : 5
Number of files read : 2
----------------------------
the sqlite RIIR is going quite well: https://turso.tech/blog/beyond-the-single-writer-limitation-...
(sqlite is quite a bit smaller than DuckDB tho)