Comment by simonw

Comment by simonw 12 hours ago

2 replies

"HNSW index on a few million vectors can consume 10+ GB of RAM or more (depending on your vector dimensions and dataset size). On your production database. While it’s running. For potentially hours."

How hard is it to move that process to another machine? Could you grab a dump of the relevant data, spin up a cloud instance with 16GB of RAM to build the index and then cheaply copy the results back to production when it finishes?

tacoooooooo 12 hours ago

i discuss that specifically!

> The problem is that index builds are memory-intensive operations, and Postgres doesn’t have a great way to throttle them. You’re essentially asking your production database to allocate multiple (possibly dozens) gigabytes of RAM for an operation that might take hours, while continuing to serve queries.

> You end up with strategies like:

    Write to a staging table, build the index offline, then swap it in (but now you have a window where searches miss new data)
    Maintain two indexes and write to both (double the memory, double the update cost)
    Build indexes on replicas and promote them
    Accept eventual consistency (users upload documents that aren’t searchable for N minutes)
    Provision significantly more RAM than your “working set” would suggest
> None of these are “wrong” exactly. But they’re all workarounds for the fact that pgvector wasn’t really designed for high-velocity real-time ingestion.

short answer--maybe not that _hard_, but it adds a lot of complexity to manage when you're trying to offer real-time search. most vector DB solutions offer this ootb. This post is meant to just point out the tradeoffs with pgvector (that most posts seem to skip over)

  • the_mitsuhiko 12 hours ago

    > short answer--maybe not that _hard_, but it adds a lot of complexity to manage when you're trying to offer real-time search. most vector DB solutions offer this ootb. This post is meant to just point out the tradeoffs with pgvector (that most posts seem to skip over)

    Question is if that tradeoff is more or less complexity than maintaining a whole separate vector store.