Comment by kingstnap

And for better or worse it feels like the errors are being "pushed down" into smaller, more subtle spaces.

I asked ChatGPT a question about a made up character in a made up work and it came back with "I don’t actually have a reliable answer for that". Perfect.

On the other hand, I can ask it about varnishing a piece of wood and it will give a lovely table with options, tradeoffs, and Good/Ok/Bad ratings for each option, except the ratings can be a little off the mark. Same thing when asking what thickness cable is required to carry 15A in AU electrical work. Depending on the journey and line of questioning, you would either get 2.5mm^2 or 4mm^2.

Not wrong enough to kill someone, but wrong enough that you're forced to use it as a research tool rather than a trusted expert/guru.

so, poor healthcare workforce quality is not just an "issue of an economically poor country", as I thought!?

like, I tried to treat the bloating in one municipal clinic in Ternopil, Ukraine (got "just use Espumisan or anything else that has symeticone" and when it did not work out permanently, "we don't know what to do, just keep eating symeticone") and then with Gemini 3 (Pro or Flash depending on Google AI Studio rate limits and mood), which immediately suspected a poor diet and suggested logging it, alongside activity level, every day.

Gemini's suggestions were nothing extreme - just cut sugar and ban bread and pastry. I was guilty of loving bread, croissants, and cinnabons (is this how they are translated?) too much.

the result is no more bloating on the third week, -10cm in waistline in 33 days, gradually improving sleep quality, and even ability to sleep on a belly, which was extremely uncomfortable to me due to that goddamned bloating!

Just my observation as a sixty-something year old, and my attempt to understand the changes I’ve seen… It seems like these days doctors advice and hospital emergency procedures are mostly (and increasingly so) based on statistical likelihoods of outcomes (from where exactly I’m not sure; and who vetted them??) and a set of options that insurance companies will pay for, probably determined similarly. Probably will get worse until AI gets inserted into the loop. I also wonder if perhaps this is occurring to accommodate a reduction of the typical skill set of the “wellness” oriented practitioners most of us see for check-ups and emergencies. I have received good care from specialists by and large.

Yeah it's bad. That doesn't mean it's necessarily uniformly bad. But if it's bad where you are, yeah it's bad.

You can see multiple doctors (among the ones you're allowed to see by your insurance). The doctors are all in an echo chamber which reinforces their thinking. Their cognitive load and goal seeking is burdened by what they can determine they can bill insurance for (there is still no price transparency). You don't have a "regular" / primary care physician because they rotate through the provider network constantly.

Symptoms which don't fit the diagnosis are ignored / dealt with by deflecting that you should "see your regular physician". "Stare decisis" rules the second opinion. In their minds they believe they have no place to write down e.g. drug interactions with things which they didn't prescribe and don't believe in (the one time I got a call from quality control working for the umbrella organization I utilized this as an example of why I was looking for a different doctor and the QA person, who was, they said, a licensed nurse, said "they can add that to the record, I'll do it right now").

You might get fired as a patient for passing out or having a seizure during a blood draw, hard to say whether that's because they failed to follow SOP and call the meatwagon or because you upset staff by acting unusually. You might get into a conversation with a physician which goes strange and they end up telling you that their clinic gets health inspections like a restaurant... they don't. There's a "wet work" inspection (just like a butcher shop) before occupancy is allowed, but there's no posted inspection report because... there is no inspection! But there's more. There are relatively "safe" and common procedures which still have ooopsies and people end up in the hospital or die. The hospitalization rate might be 1:5000 and the death rate 1:100000 but if you do a million of these there are going to be a few. If the procedure took place in a clinic it's supposed to be reported, and the reports are public record; but surprise surprise, the reported rates for serious complications are far far below what the actuarial tables show.

If you're seeing constellations of incidents similar to these, you need to get a second opinion from somewhere / somebody who is not caught up in that particular bubble. It can be very hard to see what's happening, and also to find a measurable proxy for "in / not in the bubble".

A stark difference with that analogy is that with a bicycle, the human is still doing quite a bit of work themselves. The bicycle amplifies the human effort, whereas with a motor vehicle, the vehicle replaces the human effort entirely.

No strong opinion on if that's good or bad long term, as humans have been outsourcing portions of their thinking for a really long time, but it's interesting to think about.

[dead]

Chatting with an LLM resembles chatting with a person.

A human might be "empathetic", "infinitely patient, infinitely available". And (say) a book or a calculator is infinitely available. -- When chatting with an LLM, you get an interface that's more personable than a calculator without being less available.

I know the LLM is predicting text, & outputting whatever is most convincing. But it's still tempting to say "thank you" after the LLM generates a response which I found helpful.

I feel like I’ve seen more and more people recently fall for this trick. No, LLMs are not “empathetic” or “patient”, and no, they do not have emotions. They’re incredibly huge piles of numbers following their incentives. Their behavior convincingly reproduces human behavior, and they express what looks like human emotions… because their training data is full of humans expressing emotions? Sure, sometimes it’s helpful for their outputs to exhibit a certain affect or “personality”. But falling for the act, and really attributing human emotions to them seems, is alarming to me.

Empathy is a highly variable trait in humans, both from one person to the next as well as within the same person depending on their mental state and the people and situation they are dealing with, so I'd bet that most of the time you're not going to get genuine empathy from people either. They may say empathetic sounding things but I doubt there will be any actual feeling behind it. I'm not even sure doctors could function in their jobs if they weren't able to distance themselves from deeply empathizing with their patients, it would just be one heart wracking tragedy after another if you fully immersed yourself in how each patient was feeling when they're at their worst.

Except that they can talk with you, at length, and seem empathetic, even if they're totally unconscious.

Which, you know, humans can also do, including when they're not actually empathizing with you. It's often called lying. In some fields it's called a bedside manner.

An imaginary friend is just your own brain. LLMs are something much more.