Comment by tantalor a day ago
I'm surprised by the overwhelming positive vibes in the comments here.
Maybe I'm alone? To me, this comes across as extremely creepy, the exact opposite of what we should desire from AI in products aimed at children.
I feel like it would be creepy if the kid was using it without anybody ever checking up on it ... but I think all of my friends with kids would say that the answer to that is "parenting."
I mean, giving a kid an unlocked iPad and not bothering to do basic supervision can also have really creepy results, so I'm unconvinced that something like your work actually makes anything worse in the negligent parenting situation, and seems like it could be a lot of fun in the competent parenting one.
If you haven't already done this, I'd note that I can think of a number of parents who would probably rather enjoy a version of story mode that let them collaborate with their child and your code to put together a bedtime story before they turn it off for the night and tuck the kid into bed.
The Elato toy is currently not aimed at children. The current version has adult characters that are entertaining and fun to engage with like the Chad Brew Barkley character in the videos. I put up more such funny videos on my tiktok tiktok.com/@elatoai
However, while testing it with a friend who has a 5-year old daughter, I added a `Story mode` feature to create dynamic stories for her which she enjoys.
I think what would be even cooler is if each character in a story has unique voices (like voice of an ogre, voice of an elf etc.) which is currently unsupported in the single websocket connnection.
I commented that I like the project, in that it is a project that helps you to create a realtime assistant - i would love to replace alexa/siri/whatever with something actually useful.
That said, I totally agree that I wouldn't want this in a kids toy. The whole idea is super creepy in that respect, with so much scope for abuse.
I hope these toys could be a joy/comfort for kids that don’t have a parent that cares.
I poured hours into games/programming because it was a happy place away from school etc… These toys could be the same.
This technology is neutral, but I see so much potential for projects that do good.
Why is the idea of a child talking to a LLM creepy? Do you think a child is gonna figure out how to jailbreak the “keep it keep kid, friendly” prompt, and start talking about I don’t even know what … kids don’t know about adult things. That’s just not how kids be.
I genuinely can't fathom how it wouldn't be creepy.
Bots are for doing tasks. I don't want to socialize with them and find the idea of kids being socialized by bots supremely weird. At least the AI girlfriend people are (probably unwell) adults.
> Why is the idea of a child talking to a LLM creepy?
The target audience is young kids who are still developing socialization skills. This toy off-boards that development from a human to an AI. We don't really know how that affects a kid.
This also plausibly trains the kid to think of other people as AIs: subservient tools that exist primarily to respond to them. Not exactly a healthy attitude to take towards one's peers.
It's presumably also going to get a lot of unsupervised usage, and the occasional AI model updates. What happens when a bad model update has it advising kids that soap is a forbidden candy that tastes delicious?
(I'm not saying any of these is particularly likely, just trying to share the sort of concerns that would lead someone to feeling creeped out)
Totally get the creepy part, but my criticism of devices like this is that they seem to be made by people with limited exposure to the creative power of children.
Children don’t need this; they are so much more creative than an AI (and the adults that trained the AI), and their creativity is fueled by boredom.