Comment by Eisenstein

In this program CLIP is being used to create embeddings. A multimodal LLM does something very similar. In this case the language model is not needed because the embeddings are being used to search directly.