Comment by teeray

I have fed LLMs PDF files, asked about the content and gotten nonsense. I would be very hesitant to trust them to give me an accurate summary of my emails.

LLMs are terrible at summarizing technical emails where the details matter. But you might get away with it, at least for a while, in low performing organizations that tolerate preventable errors.

I got an email from the restaurant saying "We will confirm your dinner reservation as soon as we can", and Apple Intelligence summarizing it as "Dinner reservation confirmed." Maybe it can not only summarize, but also see the future??

> LLMs are so good at summarizing that I should basically only ever read one email—from the AI

This could get really fun with some hidden text prompt injection. Just match the font and background color.

Maybe these tools should be doing the classic air gap approach of taking a picture of the rendered content and analyzing that.

What system are you using to do this? I do think that this would provide value for me. Currently, I barely read my emails, which I'm not exactly proud of, but it's just the reality. So something that summarized the important things every day would be nice.

I fed an LLM the record of a chat between me and a friend, and asked it to summarize the times that we met in the past 3 months.

Every time it gave me different results, and not once did it actually get it all right.

LLMs are horrible for summarizing things. Summarizing is the art of turning low information density text into high information density text. LLMs can’t deal in details, so they can never accurately summarize anything.

Do you ever check its work?

What is the reason to unsub ever in that world? Are you saying the LLM can't skip emails? Seems like an arbitrary rule