Comment by bambax

Comment by bambax 4 days ago

> In the US, word2vec might tell you espresso and cappuccino are practically identical. It is not a claim you would make in Italy.

True, and quite funny. This is an excellent, well-written and very informative article, but this part is wrongly worded:

> Let's have a task that looks simple, a simple quest from our everyday life: "What did I do with my keys?" [and compare it to other notes using cosine similarity]: "Where did I put my wallet" [=> 0.6], "I left them in my pocket" [=> 0.5]

> The best approach is to directly use LLM query to compare two entries, [along the lines of]: "Is {sentence_a} similar to {sentence_b}?"

(bits in brackets paraphrased for quoting convenience)

This will result in the same, or "worse" result, as any LLM will respond that "Where did I put my wallet" is very similar to "What did I do with my keys?", while "I left them in my pocket" is completely dissimilar.

I'm actually not sure what the author was trying to get at here? You could ask an LLM 'is that sentence a plausible answer to the question' and then it would work; but if you ask for pure 'likeness', it seems that in many cases, LLMs' responses will be close to cosine similarity.

stared 4 days ago

Well, "Is {sentence_a} similar to {sentence_b}?" is the correct query when we care about some vague similarity of statements. In this case, we should go with something in the line "Is {answer} a plausible answer to the question {question}".

In any way, I see how the example "Is {sentence_a} similar to {sentence_b}?" breaks the flow. The original example was:

    {question}
    
    # A
    
    {sentence_A}
    
    # B

    {sentence_B}

As I now see, I overzealously simplified that. Thank you for your remark! I edited the article. Let me know if it is clearer for you now.

Reply View 0 replies

echoangle 4 days ago

I also don’t see the problem, if I were asked to rank the sentences by similarity to the question, I wouldn’t rank a possible answer first. In what way is an answer to a question similar to the question?

Reply View 1 reply

Dewey5001 3 days ago

I believe the intention here was to highlight a use case where cosine similarity falls short, leading into the next section that introduces alternatives. That said, I would appreciate more detail in the 'Extracting the right features' section, if someone has an example I would love to see it.

Reply View | 0 replies