Key Points
- Reddit’s new AI chatbot, “Reddit Answers,” has been caught giving out dangerous medical advice.
- It suggested using high-dose kratom and even heroin for pain relief.
- Moderators are upset because they have no way to turn off or control the AI’s responses.
- In response, Reddit said it will stop the AI from answering questions on “sensitive topics.”
Reddit’s new AI chatbot, “Reddit Answers,” is under fire after a moderator discovered it was giving out “dangerous” medical advice. The moderator found the AI tool suggesting that people with chronic pain should stop taking their prescriptions and use high-dose kratom, an unregulated and sometimes illegal substance.
When the user probed further, the AI gave more alarming responses. It suggested that heroin could be used for pain relief and gave a mix of accurate and potentially harmful advice for treating a fever in a newborn.
The bigger problem, according to several moderators, is that they have no way to turn off the AI or flag its dangerous advice when it appears in their communities. This makes their already difficult job of moderating their subreddits nearly impossible.
In response to the outcry, a Reddit spokesperson said the company has updated the tool to stop it from showing “Related Answers” on “sensitive topics.” However, it’s not clear what Reddit considers “sensitive,” and the company hasn’t given moderators any direct control over the AI’s presence in their subreddits.
This incident highlights a major problem with the current generation of AI chatbots: they are not equipped to handle sensitive topics like medical advice. They can easily pick up and repeat bad or even dangerous information from the internet. In a community like Reddit, which is full of sarcasm and unofficial advice, the risk is even higher.
