Comment by dm03514

Comment by dm03514 2 days ago

0 replies

View on Hacker News

Still hacking on some data tools:

DuckDB for stream processing:

https://github.com/turbolytics/sql-flow

Lightweight kafka stream processing using DuckDB as the execution engine. 300MiB runtime can easily handle thousands of messages / second.

Working on a Kafka Connect alternative:

https://github.com/turbolytics/librarian

Right now mongo replication (through changestreams) is supported to kafka. Working on Postgres support right now.