Comment by indigo945

Comment by indigo945 13 hours ago

2 replies

    > None of the blogs mention that building an HNSW index on a few million vectors 
    > can consume 10+ GB of RAM or more (depending on your vector dimensions and 
    > dataset size). On your production database. While it’s running. For potentially 
    > hours.
10 GB? Oh jolly gosh! That will almost show up as a pixel or two on my metrics dashboard.

Who are these people that run production Postgres clusters on tiny hardware and then complain? Has AWS marketing really confused people into believing that some EC2 "instance size" is an actual server?

tacoooooooo 12 hours ago

guess it depends on your scale? for some, 10+ GB of RAM being consumed on an index build is > 25% of the DB's RAM. apply that same proportion to your setup and maybe it'll make more sense