Comment by summarity

Comment by summarity 12 hours ago

3 replies

That's where it's at. I'm using the 1600D vectors from OpenAI models for findsight.ai, stored SuperBit-quantized. Even without fancy indexing, a full scan (1 search vector -> 5M stored vectors), takes less than 40ms. And with basic binning, it's nearly instant.

tacoooooooo 12 hours ago

this is at the expense of precision/recall though isn't it?

  • pclmulqdq 10 hours ago

    Approximate nearest neighbor searches don't cost precision. Just recall.

  • summarity 11 hours ago

    With the quant size I'm using, recall is >95%.