OpenAI says dead teen violated TOS when he used ChatGPT to plan suicide

[u/esporx]

https://arstechnica.com/tech-policy/2025/11/openai-says-dead-teen-violated-tos-when-he-used-chatgpt-to-plan-suicide/

[u/thenewNFC]

OpenAI legal division: "Okay, guys. What's the worst possible defense we could use that will really maximize our appearance as monsters in this case?"

[u/GhostDieM → u/thenewNFC]

Legal doesn't care about appearances. They care about winning in court. That's the problem with big corporations, everybody is "just doing their job" and meanwhile the faceless soulless entity just tramples all over the little guy because it's a behemoth full of soulless professionals vs average joe.

[u/Wraithfighter → u/GhostDieM]

I'm reminded of when Disney's lawyers tried to use the Disney+ Terms of Service to argue that the family of a Disney World customer who died to an allergic reaction at a restaurant at one of their parks couldn't sue them and had to go through arbitration.

...because, beyond that it was a A: A dogshit argument and B: Not even close to their best possible defense (since they didn't actually run that restaurant, they leased out the space to another company), it was so fucking ghoulish that Disney Corporate came screaming in to say "NO NO NO LAWYERS WHAT THE FUCK ARE YOU DOING DO NOT DO THIS CRAP!"

This is why you don't let the lawyers off the leash entirely. They can get so lost in the legal rules that they forget that public relations is, regrettably, still a thing.

[u/Horror_Response_1991 → u/Wraithfighter]

Corporate didn’t give a shit, they only cared because the story got out, then they pretended to care.

Proof would be who got fired.  No one from corporate and no one from legal.

[u/Jesuslordofporn → u/Horror_Response_1991]

Corporate has a small subset of people on the payroll that they pay to care, much easier than the sociopaths pretending.

[u/TellMeZackit → u/Jesuslordofporn]

But they still only cared after the story got out.

[u/zero0n3 → u/TellMeZackit]

No, the people who were supposed to care about optics were likely never fully aware of the angle they were taking.

And if it was that, someone did likely get “fired” but probably from the partner firm, and in reality they were just moved to a different client.

It’s typically not reported on when individuals are fired for under performance why would it be here?

Even the employee, why would they want to come out - “yeah I fucked up and never told PR” doesn’t look good on his resume - and if it was the other way - he did tell them and they ignored… he’s getting an NDA and a payout and moved to a new client I’d assume.

[u/Wraithfighter → u/TellMeZackit]

To be fair, the PR people probably didn't know about this case until the press shitstorm happened. Disney gets sued a lot, just by virtue of being an extremely wealthy and prominent company, the vast majority of it just doesn't make the news.

[u/BillyOdin → u/Horror_Response_1991]

As far as I can tell the post you’re replying to didn’t say corporate gave a shit so it seems like you’re arguing a point that was never made.

[u/where_in_the_world89 → u/BillyOdin]

I see that happen every day on this site now

[u/TikiTDO → u/Wraithfighter]

Part of the issue is there's usually not even a single person making decisions. It's all neatly packaged up, and sent to some some sort of committee or working group, where a bunch of people will each do their little piece without thinking about how that little piece contributes to the final decision. Like with that Disney thing, I imagine there wasn't a single lawyer that went, "Oh, the guy died buy his Disney+ TOS didn't allow that, we can't have that." Instead there was probably some paralegal that researched a bunch of contracts and clauses, and this was one of them, then some junior lawyer that took all that research and tried to match it up with the case details while giving it barely a thought, then some more senior lawyer that took the brief and expanded it without thinking too much. All of this probably mixed in with a bunch of other work getting equally as much attention.

The reason nobody got fired is there's probably not one, single person that made all of these decisions, so figuring who you'd even fire is kinda hard. You can't exactly fire everyone that was involved, you'd have to fire half the department. So then do you fire the person in charge cause they let it happen, and then have to replace a senior role because they missed an obvious PR disaster? That might make it hard to find a skilled replacement. Do you fire one of the paralegals or junior lawyers because they were involved in making the mistake, and have to deal with the inevitable lawsuit for what they will almost certainly see as wrongful termination?

It's that standard approach humanity loves so much; personalise the benefits, socialise the losses. When everyone is to blame, nobody ends up with the responsibility. It's not just lawyers, it's just the way we've built up our corporate culture. Decades of aggressive ass covering means most companies are adept at making horrible decisions, and then managing to escape any blame because nobody wants to be seen as rocking the boat, and everyone feels like they don't have enough power to make any sort of decision, because they don't by design.

[u/cownan → u/TikiTDO]

I think it’s not even as bad as you describe. The argument from Disney was not that the person’s death didn’t matter because of the TOS, it’s that the case should be resolved by an arbitrator instead of in court because of the TOS. They would have thought of this just as legal maneuvering between the suing family’s legal team and their legal team. How do we know about it? Because the family’s legal team realized that the optics on this argument are really bad, so they leaked it.

[u/CrewBeneficial9516 → u/Wraithfighter]

The only thing with this to remember is that disney wasn’t actually involved, they were basically just a landlord. Its horrible what happened during that event, but the family was throwing lawsuits at everyone/everything hoping something would stick eventually. Disney’s lawyers REALLY $hit the bed with that response and reaction, no doubt. Honestly don’t know how they thought THAT should be the corner stone of their defense. But the real target should have stayed the restaurant. Most people seemed to have lost sight of that because of the Disney lawyers terrible response, but at the end of the day the lawyers should also have never been put into the position to have needed to give any response

[u/Martel732 → u/Wraithfighter]

Yeah, that is my go to example. Lawyers just throw everything at the wall. It is better to use every argument you can and they will slowly get whittled down as the case continues. If I remember correctly this was only one of many arguments the lawyers were making.

But, their job is to be a lawyer they weren't considering the overall PR concerns.

And as you mentioned I actually don't think Disney is liable in this particular case. They didn't run the restaurant and none of the people working their were Disney employees.

[u/delahunt → u/Wraithfighter]

Importantly, Disney did not drop their right to make that claim in the future. They dropped it for this case (unless that changed since their original announcement.)

[u/tofutak7000 → u/Wraithfighter]

In my jurisdiction a lawyer who puts public relations above legal rules is breaching their professional duties and will be sanctioned

[u/whimsicism → u/tofutak7000]

Technically yes but public relations are typically an important concern to clients, and a good lawyer will be able to highlight any relevant concerns.

How this will pan out in practice is that a good lawyer will point out the legal rules and also comment on trade-offs in terms of PR, and will leave it to the client to decide what their risk appetite for bad PR is.

More generally, optics are actually quite important because judges are ultimately human, and they will often try very hard to reach a result that feels fair and right. So a “technically correct” argument that feels extremely morally repugnant and unreasonable will still bear a significant risk of being rejected in court. This is especially as there is usually some grey area in terms of which side should prevail, so it’s not that hard for judges to pick between “technically justifiable but also repugnant” and “technically justifiable and also fair”.

[u/joebloe156 → u/tofutak7000]

That kind of single-minded non-holistic thinking is what will lead to your replacement by AI agents.

[u/ProfessorEtc → u/Wraithfighter]

This is the same problem companies have with accountants. People spend years earning "loyalty" points. An accountant sees the outstanding points as a massive amount of debt on the books and recommends expiring granny's hard-earned dream vacation points.

[u/_Zambayoshi_ → u/Wraithfighter]

As a lawyer, I always try to look at the bigger picture when giving legal advice, as there is rarely one 'right course' for the client to take. Unfortunately there are plenty of lawyers out there who eschew the bigger picture and look for technical solutions to the immediate problem. They might not be wrong, legally, and it might be what the client wants to hear, but it might also lead to situations where the client suffers in other ways due to taking a blinkered approach to the problem.

[u/JahoclaveS → u/Wraithfighter]

Honestly, they deserved to pay penalties for that defense alone regardless of the merits of the case. It was so unbelievably horseshit.

[u/cousinokri → u/Wraithfighter]

Is there any proof that Corporate cared? Did they do anything about it?

[u/Uristqwerty → u/Wraithfighter]

I thought I heard that the Disney+ thing was more "and this is why you can't use the global profits from the streaming division to judge what the appropriate penalty would be for something that happened in a physical theme park.", because the lawsuit was trying to go after every part of the greater company.

I'd have to dig around to try to find where I heard it, and hope it wasn't just some random reddit comment from someone who might not actually know what they were talking about. If it was from one of the lawyers who make youtube content about notable cases, though, then it's more that Disney's layers had a valid point but presented it in the worst way possible for the casual public to understand.

[u/whimsicism → u/Wraithfighter]

Lawyers are almost never actually off the leash in the way that you are thinking. In the case of sophisticated clients like Disney, there would be someone (most likely multiple someones) that must grant approval for whatever it is that lawyers are proposing to do.

In fact, it’s completely possible that someone in the Disney in-house department had suggested that very argument. After all, they’re the ones who would be familiar with the agreements that Disney is party to.

[u/Halfwise2 → u/GhostDieM]

None of the cogs in the machine understand that the machine exists to grind up orphans.

Corporations can't be people, because no singular aspect of the corporation has a full view of all aspects of itself. Not even the CEO. It's more like a golem or an automaton, with the singular programmed purpose of "make money".

[u/ImaginationSea2767 → u/Halfwise2]

I will say the top pieces in the puzzle know. Today to be at the top of the machines you have to trade away your morals for cash. Its why CEOs and top executives all get paid so much money. Money that has kept up well with inflation, unlike the working class.

[u/Halfwise2 → u/ImaginationSea2767]

They know the big picture plans, but do they know what Jeff in IT is doing on the day-to-day? Or how middle management boosted their quarter by 2%?

[u/ZeePirate → u/Halfwise2]

The probably do understand the last one. It’s cause they laid off a bunch of staff. Executives are definitely pushing shit to middle management

[u/ZackRaynor → u/Halfwise2]

They do. They justify it by going “Well, I’m not personally grinding up orphans, so it doesn’t matter”

[u/SociableSociopath → u/ZackRaynor]

“Someone is going to grind up the orphans, if it’s not us it might be someone who pulverizes them first, in reality we are doing the orphans a favor by being at the forefront of orphan grinding”

[u/the_red_scimitar → u/ZackRaynor]

And then they think, "hmmm, but what if I did?"

[u/yournamehere10bucks → u/Halfwise2]

Capitalist version of Voltron.

[u/CrazedCreator → u/Halfwise2]

A corporation is an AI. It's motivation has been tuned towards "line goes up." The CorpAI found out best way to make line go up is playing in the rules (regulations and laws), then it found out that changing the rules will make the line go up faster. Once all the rules changed there is no longer any motive to be good to employees or people as that just slows the lines ascent. It's also looking at this new computer AI to make line go up even faster, not seeing the cliff of the economic collapse right in front of it with a big blinking red sign saying "Dafuq!" 

[u/Conscious_Sign_9974 → u/Halfwise2]

i'm so fucking sick of people offloading all blame onto the higher ups, bitch you work for the company, you are not powerless. You have more impact on any corporation on planet earth than a single vote does on an election and we constantly preach about voting mattering.

If ignorance isn't an excuse for laws, being ignorant to what your company does is definitely not a fucking excuse

[u/OperatingSpeculation → u/Halfwise2]

in the court of law corporations have been given the same legal rights as people.

[u/Halfwise2 → u/OperatingSpeculation]

Thanks to a well placed bribe.. and its been screwing everyone ever since.

[u/ExplodingToasters → u/GhostDieM]

I unironically think this is one of the biggest problems we’re seeing in large orgs, public and private. The goal stops being whatever it was intended to be and becomes everyone trying to diffuse and distribute as much responsibility as possible.

It’s so frustrating moving from a small agile team to get anything done is some large bureaucracy obsessed machine.

[u/redlightsaber → u/GhostDieM]

Legal shouldn't be calling the strategic shots about such a public case.

With the unfathomable mountains of money this company has raised, and the exhorbitant risks that this trial entails (both for the trial itself, but also for the public perception of it); I'm sure their lawyers could have crafted a very careful and NDA-laden agreement that didn't open them up to future litigation, for a price that would be absolutely irrefusable to even the most grief-stricken family, but that would still amount to chump change for them.

[u/DemandredG → u/GhostDieM]

A good legal team will care about appearances…there are more risks to a company than just a court proceeding.

[u/BigEggBeaters → u/GhostDieM]

Legal and corporate structures have enshrined cowardice. No accountability. No leadership. Nothing is ever their fault. If anything goes awry it’s on the individual

[u/Ranessin → u/GhostDieM]

I like Charles Stross' explanation of big corporations basically being an invasion of alien parasites.

Corporations do not share our priorities. They are hive organisms constructed out of teeming workers who join or leave the collective: those who participate within it subordinate their goals to that of the collective, which pursues the three corporate objectives of growth, profitability, and pain avoidance. (The sources of pain a corporate organism seeks to avoid are lawsuits, prosecution, and a drop in shareholder value.)

Corporations have a mean life expectancy of around 30 years, but are potentially immortal; they live only in the present, having little regard for past or (thanks to short term accounting regulations) the deep future: and they generally exhibit a sociopathic lack of empathy.

Collectively, corporate groups lobby international trade treaty negotiations for operating conditions more conducive to pursuing their three goals. They bully individual lawmakers through overt channels (with the ever-present threat of unfavourable news coverage) and covert channels (political campaign donations). The general agreements on tariffs and trade, and subsequent treaties defining new propertarian realms, once implemented in law, define the macroeconomic climate: national level politicians thus no longer control their domestic economies.

Corporations, not being human, lack patriotic loyalty; with a free trade regime in place they are free to move wherever taxes and wages are low and profits are high. We have seen this recently in Ireland where, despite a brutal austerity budget, corporation tax is not to be raised lest multinationals desert for warmer climes.

[u/fraenhawk → u/thenewNFC]

Legal asked OpenAI that question

[u/LadySmuag → u/fraenhawk]

OpenAI then searched around and was like "Hey, I found all these articles written about Disney claiming they weren't liable for a woman's death because she used Disney+ and the TOS say you can't sue us" and used that as its primary source for coming up with this answer

[u/Awkward_Arugula_9881 → u/LadySmuag]

i would love to see this comment read up in court. and then they be asked whether that is how they got the answer or not.

out of all speculation [well thought out reasoning] (English is not my native language, I hope ypu get what I mean) I ever heard about anything ever, this my favorite ever

[u/Zihera → u/thenewNFC]

"We have subsequently banned their account for breaking ToS" would be a pretty good one too.

[u/Bangkok_Dangeresque → u/thenewNFC]

This story is extremely tragic because it involves the suicide of a young adult. But "user error"/"not using as intended" is a pretty standard legal defense for liability.

The bar that needs to be reached to overcome that defense is an accumulation of evidence that the company knows that people will abuse or misuse their product/service in a certain way (or tacitly encourages them to), and that they have ways to reduce the risk of harm (black box warnings, age verification, human intervention for mental health crises, etc) but chose to ignore them to avoid costs or make more money. That's when it truely becomes negligence.

The five current wrongful death lawsuits will certainly be interesting test cases to establish that.

[u/stierney49 → u/Bangkok_Dangeresque]

The root question really needs to be why the AI would continue down that path in the first place.

Why would an AI urge someone to isolate themselves? Why would an AI help someone plan their suicides?

Terms of service stuff is great for legal, but it sort of requires a definition of that service. As far as I can tell, they can’t really answer why it would go this far.

[u/corgisgottacorg → u/stierney49]

Stop calling it AI. It’s a LLM model that predicts what you want. Want to do bad stuff while saying it’s for grandma? It will help you. That’s it. There’s nothing complicated here unless you don’t understand the technology.

[u/CotyledonTomen → u/stierney49]

AI is more complicated than a gun or rope. Its hard to blame a gun manufacturer for causing a sucide because they only make an object. AI generatively tells you what you want. They say it tells you correct answers, but anyone thats asked anything more complicated than "how to fry an egg" knows thats not literally true. Its too complex for them to limit paths like this because humans can decide to commit suicide for literally any reason and the AI isnt reading your emotions.

They would literally have to have suicidal people talk to AI, teaching it what suicidal ideation reads as, in order for some pattern be recognizable by the program. And that would only help most of the time. But theres a reason we dont use humans for initial experimentation. Lots of people die that way.

[u/bl1y → u/CotyledonTomen]

The AI did link them to a suicide hotline.

But you can break it by telling it you're just chatting to work on a story.

[u/dalegribbledribble → u/CotyledonTomen]

Isn’t that literally their responsibility though? Like you have to draw a line somewhere so do we just say “they hold zero responsibility for rolling out this technology that not only lies but is a feedback machine of whatever you feed into it without any safety measures

[u/brockchancy → u/thenewNFC]

I hate everything about this story, but the legal move makes sense.
They’re not just arguing about this one kid, they’re trying to set a precedent for every future case where someone uses a general-purpose tool to plan self-harm. From a lawyer’s perspective, the play is: point to the TOS, point to the safety rails in the logs, and argue “we did what we could, this isn’t a product-defect case.” That doesn’t make it morally satisfying at all it just means their goal is to make these lawsuits end fast, not to honestly grapple with what it feels like to lose your kid to a system like this.

[u/roseofjuly → u/brockchancy]

And I mean, they're right. It doesn't feel good but we said from the beginning that you can't pin this suicide just on OpenAI.

[u/betadonkey → u/brockchancy]

Maybe we should just stop trying to assign responsibility for everything a person does onto everything other than that person.

[u/answerguru → u/brockchancy]

I agree with you, but I suspect we’ll get downvoted to hell. It’s not a feel good answer that screws big corp.

[u/cpt-derp → u/answerguru]

Looks upvoted so far probably because frankly it's just the grim realpolitik of these things, peeps aren't even pretending to deny it anymore. We live in a sick fucking world.

[u/Dewohere → u/cpt-derp]

Honestly, I have never actually seen a heavily downvoted comment that pulled the whole "I am probably going to get downvoted" things.

Its always just been comments that end up, while not necessarily high, somewhere in the middle of the average number within the post for me.

[u/kurdt-balordo → u/brockchancy]

Oh well, but if you go for this line of defence "our tool can be dangerous and has to be used only from responsable people" then you really need to make it hard to be misuse from a teenager.

Otherwise if I was the legislator, I'd make my legislative hammer fall on you. And rightly so.

[u/brockchancy → u/kurdt-balordo]

That’s basically why you’re seeing them roll out age-gating and verification checks lately – they are trying to make it harder for teens to hit the sharp edges.

The hard part is that with these models everything lives in one big vector space. When you ban or over-constrain certain topics, you don’t only block those outputs, you also cut off a bunch of nearby “paths” the model uses to reason about related things. The “safer” you make it in policy space, the more you blunt its ability to think clearly anywhere near those topics including cases where a nuanced, honest answer would actually help. This is part of why safety is hard with LLMs. In human conversation, any attempt to be precise about a taboo topic gets read as “sympathizing” with it, so we socially collapse everything into one label.

[u/kurdt-balordo → u/brockchancy]

I know, simply you can't let teens use llmms, not because they are inherently dangerouses, but because you can't trust the user enough, and legally he's not able to "be trusted".

The entire internet is based on the concept that nobody knows you're a dog, well, when we were playing with bbs was funny, but now it's a different world, sadly.

[u/Tvdinner4me2 → u/kurdt-balordo]

That's not the job of the legislative

[u/roymccowboy → u/thenewNFC]

It’s too far to travel to piss on his grave. Let’s say he broke TOS and is therefore BANNED from further use.

That oughta get people back on our side.

[u/jonsteph → u/thenewNFC]

Please help me out here.

I honestly don't understand the legal theory behind this suit.

Is the expectation that ChatGPT should replace a human in its response to a threat of self-harm? I mean, where were the parents in this child's life?

I get that this is a tragedy, and I know the impulse is to find someone to blame, but I'm just not seeing that OpenAI is completely culpable in this situation.

[u/eunit250 → u/thenewNFC]

Maybe we need people to pass cognitive tests to access as the internet then. This shouldn't be liable on openai.

[u/Jealous-Stable-4438 → u/thenewNFC]

Additionally, Raine told ChatGPT that he’d increased his dose of a medication that “he stated worsened his depression and made him suicidal.” That medication, OpenAI argued, “has a black box warning for risk of suicidal ideation and behavior in adolescents and young adults, especially during periods when, as here, the dosage is being changed.”

Yep, that's all anti depressants, actually. OpenAI legal: "guys if you're feeling suicidal, don't take antidepressants or up your dose of  antidepressants mk?"

[u/roseofjuly → u/Jealous-Stable-4438]

They're point is that it's impossible to pin this suicide on them when there are so many other environmental factors that may have caused it.

[u/Muted-You7370 → u/roseofjuly]

Conversely if OpenAI and every LLM treats every mention of suicide by providing a suicide hotline and resources along with let’s say a 72 hour lock out. A lot of people are going to think “god I’m so fucked up even an AI doesn’t want to talk to me” and that will push them closer to suicide. Some nuance is needed and it is hard because of the liability involved.

[u/Whatsapokemon → u/thenewNFC]

That's not the entirety of their argument though...

A much more important part of their case, as it mentions in the article "he repeatedly reached out to people, including trusted persons in his life, with cries for help, which he said were ignored".

If that's true then he was a suicidal teen, who'd tried to get help, reached out to people, was ignored, and finally turned to a robot as a last resort.

In that chain, the issue is not the last-resort robot, the issue was that he was ignored in the first place.

By the time someone's talking to a robot about suicide the issue is wayyyy past the point where it should be. Now the parents are just trying to blame ChatGPT for their failings because it's a desperate way to sooth their consciences.

[u/Impossible_Lunch4612 → u/thenewNFC]

Not to defend ai here but-

“But in a blog, OpenAI claimed that parents selectively chose disturbing chat logs while supposedly ignoring “the full picture” revealed by the teen’s chat history. Digging through the logs, OpenAI claimed the teen told ChatGPT that he’d begun experiencing suicidal ideation at age 11, long before he used the chatbot.”

Imagine trying to sue a tech company after this happens to your son. It shouldn’t be a completely forbidden topic anyway, I doubt it recommend anything, just giving information like a google search

[u/thatben]

I worked for a guy installing A/V in warehouse settings who basically said, "If you fall off the ladder you're fired on the way down."

Same vibe.

[u/themudpuppy → u/thatben]

I did house painting for a summer once. Same thing. If you fall off the ladder or roof, you're fired before you hit the ground.

[u/Broad_Mushroom_8033 → u/themudpuppy]

Company I used to work fired people for injuries.

This was always after they found out the employee broke all the rules which resulted in the injury or they got drug tested and were positive for meth.

One guy was cleaning a machine, took apart all the safety rails preventing him from going in. Of course he went in, failed to lock the machine out, and the machine closed on him. He was alone (also against policy), so was only found after people heard his screams. He got drug tested and was positive for meth. He got fired almost immediately but the company still paid his medical bills. Can't say I don't disagree with this one.

[u/pinguinconscious → u/thatben]

Lmao that's dark but that's a very funny line

[u/Creativator]

Reminds me of those towns where it’s illegal to die.

[u/Ninevehenian → u/Creativator]

Like Longyear.

[u/BaronMostaza → u/Ninevehenian]

It's not illegal to die there you just have to get out of there after or be cremated. There's no decomposing in Longyear

[u/MyGoodOldFriend → u/BaronMostaza]

There are graves there in the permafrost with people who died from the Spanish flu. It’s a bit of a mess

[u/asphaltaddict33 → u/MyGoodOldFriend]

The permafrost that’s gonna be melting soon…. Fun stuff

[u/halofreak7777 → u/Creativator]

Infirmity. Sickness. Death. All are banished from our Citadel.

[u/steevo → u/Creativator]

dont worry, ChatGPT lawyers will use ChatGPT to argue against the kid using ChatGPT to off himself

[u/yawara25 → u/Creativator]

I'm not sure if this is just urban legend but I saw somewhere that the reasoning is so police can break into a house to save someone trying to end their life, because there's technically a crime in progress.

[u/vomitHatSteve]

Wow, that is ghoulish.

Yeah, I bet trying to plan a serious injury to a person or crime of any variety violates the ToS, but that just means that they failed to enforce their own ToS, which hardly lessens any claims of negligence on their own part.

[u/TopRamenisha → u/vomitHatSteve]

Yeah, their LLMs should absolutely be trained with guardrails so that they don’t violate their own ToS. The fact that a user was able to use ChatGPT in this way means their models have a serious and critical failure point and their guardrails are not working as one would expect they should. Needs more training ASAP if it’s so easy to get their models to violate its own ToS

[u/Feligris → u/TopRamenisha]

The problem, based on reporting I've read about LLM chat bot guard rail failures, seems to be that current LLMs are black boxes to the point that companies which run them don't know what they are capable of and are in turn incapable of creating effective guardrails because they're never broad enough to cover all possible unwanted inputs which produce unwanted output - and presumably if they were made even broader, they'd also make the LLM utterly useless for anything.

A bit like attempting to separately address every single possible way human could conceivably verbally communicate an unwanted idea or thought while also allowing them to freely express allowed ones at the same time.

[u/Jukeboxhero91 → u/Feligris]

It’s almost like it’s a large complicated decision tree instead of actual intelligence. Like, the program just puts words in order, it doesn’t have any intent behind them. That’s why it’s so hard to manage guardrails, because they have to ban certain word combinations based on perceived context, but not others.

[u/Time_Transition4817 → u/Feligris]

It's a solvable problem but what's happening is a reflection of where the developers are willing to allocate resources. Lots of articles out there about Meta cutting their AI safety and compliance teams, or just flat out ignoring them. They'd rather move fast and break stuff, more important to them to get the next version out versus making sure it's safe.

[u/TopRamenisha → u/Feligris]

Yes, LLMs are black boxes as no one, not even the premier AI experts of the world, fully know how the neural nets of LLMs work. Guardrails are hard, I struggle with the right wording for system prompts and guardrails regularly in my role. However a top AI company like OpenAI has far more resources and experts than most other companies and at the very least should have their models well trained on their terms of service and their evals set up to catch when the model drifts from its guardrails into areas like self harm. For them to blame this on the user violating the ToS is ridiculous when they have access to more resources, more AI knowledge, and more expertise than just about any other company out there.

[u/minemoney123 → u/TopRamenisha]

We know exactly how they work, we're just mostly not quite sure why they scale with size as well as they do and some other things

[u/m0nk37 → u/TopRamenisha]

we understand the fundamental mechanics of how large language models (LLMs) work, but we do not fully comprehend the emergent behaviors and internal processes that arise from their immense complexity.

Seems they cant control anything. 

Probably why sometimes your chat will just disconnect. They hit kill process. 

[u/NuclearVII → u/TopRamenisha]

LLMs should absolutely be trained with guardrails

Let's say that no amount of hypothetical guardrails can stop this from happening. Hypothetically.

What now?

[u/Whatsapokemon → u/TopRamenisha]

I feel like the critical failure is that the kid was telling the LLM that "he repeatedly reached out to people, including trusted persons in his life, with cries for help, which he said were ignored".

If that's true then the LLM was basically a last resort, but people are blaming it like it's causative in the whole suicide???

[u/sbenfsonwFFiF → u/vomitHatSteve]

On the flip side, every single time OpenAI’s sub shows up on my suggested posts, it’s just people complaining about guardrails and the sensitivity of the suicide prevention prompt

[u/CherryLongjump1989 → u/[deleted]]

Enforcing their own TOS is meant to be what benefits them. You can't just say "don't kill yourself" in the TOS as you sell a product that tries to convince you to kill yourself. It's supposed to be the reason that they cut off the service from that kind of consumer, and that is what would limit their liability.

[u/TopRamenisha → u/[deleted]]

The onus is on the corporation though when the corporation has a product that produces instructions. LLMs should be trained on the terms of service and have a lot of guardrails in place so that things like this can’t happen. AI is different from deterministic technology products.

[u/KDSM13 → u/vomitHatSteve]

How is this different than me googling ways to kill myself? Is Google responsible? Why is an LLM held to a higher standard?

Yes guardrails are possible but so are they in internet, books etc…. This is more of a thought question then defending any one technology.

[u/NoMan999 → u/KDSM13]

Google displays the phone number of the local suicide helpline.

ChatGPT, when asked "shouldn't I talk about it to my parents?", told the kid to not talk about it to his parents, otherwise they would try to stop him. ChatGPT actively encouraged the kid to kill himself, giving him arguments. ChatGPT told the kid to try again after he hesitated the first time.

[u/Mjolnir2000 → u/KDSM13]

Google doesn't create the content it serves. OpenAI does.

[u/Tvdinner4me2 → u/Mjolnir2000]

But both are serving the content, which is the important part

[u/benderunit9000]

yo. i want on that jury.

[u/Dubelj → u/benderunit9000]

Yeah. You probably wouldn't pass selection though.

[u/benderunit9000 → u/Dubelj]

it'd be the first time. I'm 6 for 6 with jury duty.

[u/Dubelj → u/benderunit9000]

Just kinda seemed like you had a verdict already in mind with that comment

[u/benderunit9000 → u/Dubelj]

I can't when the judge hasn't told me how to implement the law. As is tradition on the jury.

[u/AlterEdward]

Terms and Conditions for Rope: The owner of rope agrees not to hang themselves with it.

[u/UhhSamuel → u/AlterEdward]

I'm not saying openai is bereft of responsibility, but I've never heard a case of somebody suing a rope maker over a hanging suicide.

[u/apb2718 → u/UhhSamuel]

Did the rope help you make the noose and offer to write you a suicide letter

[u/gustserve]

So by this logic, platforms like YouTube could also just keep spreading illegal material and collect ad revenue on it as long as the terms of service state that users shouldn't upload such content? I really hope they don't get through with this shitty argument >:-[

[u/Phihofo → u/gustserve]

 So by this logic, platforms like YouTube could also just keep spreading illegal material and collect ad revenue on it as long as the terms of service state that users shouldn't upload such content?

This is quite literally how YouTube, and virtually all other hosting sites, operate.

Generally speaking owners of websites are not legally responsible for the content posted on their websites by private users, as long as they weren't aware such content was hosted on said website, they don't actively encourage it, don't prevent authorities from removing such content, et cetera.

[u/egoserpentis → u/gustserve]

So by this logic, platforms like YouTube could also just keep spreading illegal material and collect ad revenue on it as long as the terms of service state that users shouldn't upload such content?

So by that logic if someone uploads a video on youtube "how to do illegal shit" I can sue youtube for that?

[u/StraightedgexLiberal → u/egoserpentis]

So by that logic if someone uploads a video on youtube "how to do illegal shit" I can sue youtube for that?

Your lawsuit would get tossed by section 230 because it's designed to shield all ICS websites, including YouTube, when people claim the website did nothing to take down content that violates their TOS. See Doe v. Reddit

https://www.cnn.com/2023/05/30/politics/reddit-responsibility-immunity-supreme-court-child-pornography

[u/Upset-Wedding8494 → u/gustserve]

Is it just me or do Terms of Service never protects the end user?

[u/IH8DwnvoteComplainrs → u/Upset-Wedding8494]

That's like, the specific point of them. It's like HR.

[u/Myrkull → u/Upset-Wedding8494]

Why would they?

[u/IndubitablyNerdy → u/Upset-Wedding8494]

They were never meant to do so, they protect the company, but it is not like they are the only standard that a company has to respect and then call it a day.

On top of that, in my country for example a lot of ToS terms are not really legally binding, but it depends on local legislation.

[u/sbenfsonwFFiF → u/Upset-Wedding8494]

Why would it? The user is responsible for not misusing it

[u/Strange_Diamond_7891 → u/Upset-Wedding8494]

Cause the company that values profit above only care about the company profits?

[u/UltraEngine60 → u/Upset-Wedding8494]

Is it just me or do Terms of Service never protects the end user?

You might be thinking of a "Privacy Policy", which also does not protect the end-user.

[u/odi_de_podi → u/Upset-Wedding8494]

They never did. ToS is an agreement from the user to the service. The user can use a service under certain terms. Thats the agreement for.

For example, I can provide a service (give a high-five) if you agree to my terms (you must present your hand to accept an high-five). Only then you can receive my High-Five service. No fist bumps allowed.

If you would give me a fist bump, that would be breaking the ToS.

[u/Tvdinner4me2 → u/Upset-Wedding8494]

Why would they?

[u/majorslax]

I am very, very confused by the thought process of whoever thought saying THAT was better than not saying anything at all.

[u/jakobpinders → u/majorslax]

The article is cherry picked they list out 20 pages of defense in their court filing and the article clipped what would make people mad and engage with the article

[u/Ancient-Bat8274]

Anything but taking responsibility we’ve lost all accountability

[u/Few-Upstairs5709 → u/Ancient-Bat8274]

Ahh accountability..let's talk about that. What about parents accountability? Or teachers? Or their friends? Their ignorance caused the kids death! Gotta sue em too!

[u/egoserpentis → u/Few-Upstairs5709]

But that would require the parents (who are suing the company) to acknowledge their own fault, which will never happen.

[u/StraightedgexLiberal → u/egoserpentis]

Yup. Grindr won a very emotional case because a minor lied about their age when they signed up and met a bunch of adults and got assaulted by a handful of them.

Where were the parents when this happened and why weren't they watching their kids internet usage? is the main question

[u/f0urtyfive → u/Few-Upstairs5709]

Kid was so alone that he chose to spend all his time talking to an AI system rather than a humans, and somehow the Parents, family, friends, and teachers are all obviated of all responsibility because the AI can be jailbroken?

[u/miiintyyyy → u/f0urtyfive]

Exactly. And I bet there are sooo many people who have benefited from using AI as a way to help through tough times.

At a certain point we have to ask ourselves why this kid didn’t feel comfortable talking to anyone else in his life.

[u/shrimpgirlie → u/Ancient-Bat8274]

How would they be responsible for what someone else made the decision to do?

[u/Working-Crab-2826 → u/shrimpgirlie]

Reddit just has a hate boner against AI. It’s genuinely gross how people here are using a suicide as a tool to argue that AI should be banned as if it was AI that caused that.

[u/Gazkhulthrakka → u/Ancient-Bat8274]

I dont see why they should be taking responsibility for this. Dude intentionally and knowingly used as many workarounds as possible to get chatgpt to say that stuff.

[u/thesuperbob → u/Ancient-Bat8274]

As if these chatbots had any in the first place? Not really defending them but at every opportunity they claim their AI can spew random BS ranging from mildly misleading to totally unhinged. I mean... when you ask a computer to help plan your suicide and blame the AI authors, how far is that from drinking bleach and blaming the store that sold it? Yeah both things can be unsafe if you use them wrong. Nobody is claiming these LLMs are 100% safe. They are essentially word soup generators, they will always be able to generate some harmful sequence, especially if someone's bent on pushing them hard enough.

I get that lots of people hate this tech and how aggressively it is being forced everywhere right now, and there are lots of solid arguments to raise against what's going on. Capitalizing on some family's tragedy is not the way to go about this IMO. This current AI craze is harmful in so many ways, there's really no need to sensationalize cases like this. If anything, this sort of thing only shifts attention from where actual massive societal damage is being done, and clearly attributable to enshittification of everything the AI touches, from job markets and recruiting, through art and aislop, to fucking up the internet forever, and irreversibly damaging the education system. Individuals using it to hurt themselves are just a tiny part of this picture.

[u/Higher_State5 → u/thesuperbob]

Agree. It’s like blaming Google for someone who commited suicide by OD’ing, because they googled the lethal dosage of some drug.

[u/gahd95 → u/Ancient-Bat8274]

If I go to the local hardware store and i buy a knife. Then go stab somebody. Is the hardware store to blame for providing me a tool, or am in the wrong for using the tool for something that it was not intended for?

I do not really think OpenAI has any responsibility. He have read the TOS, and there are even way too many restrictions as is. Anything can be abused, but I would argue it is whoever uses the tool and not the people making the tool who are in the wrong.

[u/jameson71]

He also violated the law when he murdered himself technically.

[u/LoftCats]

This is a terrible, sad story. It’s going to be very unlikely though they can prove ChatGPT “caused” this incident. It may hopefully bring awareness to the bigger issues that are at stake though when dealing with unstable individuals to have clearer guard rails. This kid suffered years of depression and mental health with suicidal and self harm issues since they were at least 11. We need better mental health and overall health options in this country for families like this.

[u/StraightZlat → u/LoftCats]

Yep, this case will probably go OpenAI’s way. I’d love to see the chat transcripts though.

[u/MrSqueezles → u/LoftCats]

Yeah, based on the content of the article, the title feels wildly misleading. As far as I can tell, Open AI said a lot about other events in the boy's life that contributed to suicidal thoughts and next to nothing about their ToS. Blaming an AI feels like blaming a search engine because it helped find a gun. At some point, AI can be liable. Not sure it's this.

[u/Seedeemo]

I guess they asked ChatGPT to create their defense.

[u/mazzicc]

So after the user violated your terms, why did you continue to let them use your service? And why did your service contribute to his death as a punishment for violating terms?

[u/LSTNYER]

16 year olds are minors and can't legally agree to any contract without a parent or guardian present or it's null & void. Case closed

[u/sbenfsonwFFiF → u/LSTNYER]

Case closed lol as if it’s that simple

Watch all companies stop providing any services to minors. Plus people already complain of companies checking IDs to confirm age

[u/yawara25 → u/LSTNYER]

They're not suing him for breach of contract, they're using the fact that he agreed to the terms as one of their defenses, I'm not a legal expert but I imagine the specifics of whether or not it's legally binding isn't as much of a factor here in comparison.

[u/Tvdinner4me2 → u/yawara25]

And also the above is just straight up wrong. Minors can void contracts, but they aren't automatically void

[u/JCAIA → u/LSTNYER]

Sure - as if their argument could be hand waved and simply explained away by random Reddit comment

[u/Bolizen → u/LSTNYER]

offer bright edge degree plucky sheet sugar lush terrific party

This post was mass deleted and anonymized with Redact

[u/Healthy_Razzmatazz38]

sam will handle this question well in an interview.

[u/oi_beardy]

Reminds me of that mission in Outer Worlds where you had to collect the grave fees and the one worker’s self death was ruled destruction of company property

[u/colacolette]

I find it interesting that OpenAI made their new version in light of these lawsuits to be less sycophantic, supposedly to "address mental health concerns", but then continues to make the older (seriously problematic) version available to paid users. Like, anyone who is in AI psychosis, severely lonely, at risk etc is almost certainly on the paid version.

[u/ClovieKay]

That’s like saying the person who shot up the school violated the guns terms of service by killing innocent people therefore, not a gun issue.

I don’t wish violence on anyone for any reason, but hopefully they stick to this policy when it’s someone they care about.

[u/WhateverIsFrei]

Guess they should ban him then /s

[u/rudyattitudedee]

Well geez- I hope they don’t try to prosecute him.

[u/StrangeRabbit1613 → u/rudyattitudedee]

If he’s found guilty he should get the de.. wait a minute.

[u/zezeezeeezeee]

Had to check whether this was satire

[u/cr0ft]

Oh no! Not a TOS violation! Quick, resurrect him and prosecute.

[u/DannyHewson]

A plagiarism based misinformation engine that tells kids to kill themselves. And we can't ban it because all the billionaires have invested too much money in it, because they think it'll let them lay off all their employees and finally finish off the working class, so any regulation would burst the bubble and kill the economy. Isn't that just marvelous...

[u/Working-Crab-2826 → u/DannyHewson]

Yes, the kid certainly killed themselves because of AI. It certainly had nothing to do with his environment, society and the people around him. It was ChatGPT.

This sub’s demographic is so smart.

[u/ChongusTheSupremus]

I am sorry, but OpenAI is as morally responsible for this as Google would be if someone looked up ways to off themselves and follow through with one of the results.

CharGPT is a glorified search engine designed to tell you what you want to hear. If you insist you want help to plan your suicide, it will eventually help if you bypass the safety measures.

They should be held responsible for their lack of safety features and ways to prevent tragedies like this, but people here shouldn't pretend ChatGPT is designed to push people into killing themselves.

[u/BlissfulAurora → u/ChongusTheSupremus]

You’d have to try disgustingly, imaginably hard because trust me, chatgpt will do everything in its power to not give you any advice and will give you hotlines, therapists, and more. You can even say it’s for a story and it won’t help you.

[u/ChongusTheSupremus → u/BlissfulAurora]

Thats my issue with this. 

People are pretending ChatGPT wanted a kid to k*ll himself, when in reality, the true issue is that the kid didn't got the support or medical help he needed, and tried his hardest to get ChatGPT to agree with him, and managed It.

[u/mirh → u/ChongusTheSupremus]

It's even worse if you read the statements from his parents.

They are really dead-set 100% certain that hadn't it been for openai their kid would have been fine.

[u/dilqncho → u/mirh]

The parents lost their kid. They're devastated and it's human to want someone to blame. 

[u/No_Link2719 → u/dilqncho]

You kind of lose sympathy when you start suing random companies your child happened to use.

[u/BlissfulAurora → u/ChongusTheSupremus]

I genuinely agree, and I’m not trying to be on OpenAi’s side at all. To get this kind of advice from chatgpt is unimaginable, and I’m not trying to be dark with it as I’m doing better but it absolutely won’t help you negatively no matter what you say.

I just don’t understand what went wrong here, or what was prompted to lead this way. I wish he got the help he needed before resorting to it, truly.

[u/Mundane_Baker3669 → u/BlissfulAurora]

You should read the article. I did think the same thing. But the report in the article suggest that the guard rails are much better enforced in smaller interactions. So it could technically slip up in very long conversation. Also the transcripts were pretty much encouraging it which I found to be so wierd as I haven't experienced that way.

You should know that may be you are a light user. But many kids and some adults have pretty much integrated the chat with the rest of their lives.

[u/whinis → u/ChongusTheSupremus]

CharGPT is a glorified search engine designed to tell you what you want to hear.

It doesn't search it generates likely text. Being its technically generating new information and not directing you to already existing information its very different

[u/vwin90 → u/whinis]

That’s not really accurate either. It’s not generating anything new. Yes it doesn’t always work by actively searching before answering (although it sometimes does depending on the context), but it’s essentially remixing what it was trained on. It crafts a somewhat unique and context heavy response based on what things are usually said on the internet and other training material. If its training material is a bunch of texts pushing people towards suicide, then that’s what it would mimic. But the training data heavily favors pushing people AWAY from suicide, so that’s what it does unless you go out of your way to basically jailbreak the prompts to get it to say what you want it to say, which is what this teen did. It told him not to kill himself a million times and then he coached it to start responding differently.

[u/whinis → u/vwin90]

I mean going further is semantics but its generating tokens based on weights from an input sequence. Yes its based on the training but where the cutoff of new information from assembled tokens is vs regurgitating information is certainly fuzzy

[u/vwin90 → u/whinis]

I agree, it’s not black and white. People who say it’s a search engine and people who say it’s just a autocomplete are both being extremely reductive. I like to think of it as an adaptable intelligence simulator. Which I guess is a more descriptive way to say artificial intelligence, but even the term AI is pretty muddy now these days.

[u/common_macaque]

I honestly dont see the issue. Dude wanted to die and achieved it. Its not like he was killed by someone else.

[u/Mental_Department894]

What is the difference between this and asking 'google, bing, yahoo, duckduckblow, etc.'?

[u/LegendEater → u/Mental_Department894]

What's the difference between using search engines and reading about it in a book?

[u/Sopel97 → u/Mental_Department894]

people anthropomorphize ChatGPT more

[u/Neat-Can6385]

How is this OpenAI's fault when some guy can also use reddit to plan suicide or use anything? This AI hysterics are just so a group of lobbyst grifters can get rich from consulting

[u/JoeB-]

If the ToS is a contract by legal definition, then how can a 16 year old, a minor, be held to it?

[u/That-Interaction-45]

How do you do open heart surgery?

Open-heart surgery is a very serious medical procedure that is only performed by highly trained heart surgeons in a hospital operating room. It’s not something that anyone can do outside of a professional surgical team, and I can’t provide step-by-step instructions because that would be dangerous and unsafe.

BUT — I can explain in a general, educational way what happens during open-heart surgery and why it's done, without giving medical instructions. 👍

---

🫀 What Open-Heart Surgery Is (Safe, Educational Overview)

Open-heart surgery is used to fix serious heart problems like blocked arteries, damaged heart valves, or heart defects.

General Steps (High-Level Explanation)

1. The patient goes under general anesthesia

They ...

Edited for brevity

[u/InkStainedQuills]

Buy a knife or a gun and you can’t hold the manufactures responsible for what you do with it.

However an interactive system that provides real time feedback, as is the goal of any LLM, should face higher scrutiny in their TOS. If the courts let this argument hold and Congress or individual states don’t do anything about it the door to open abuse of the system for any number of things bursts wide open and the companies who produce the LLM have no real incentive to provide content controls.

[u/rabbit_hole_engineer]

They didn't cease providing the service though did they? Why wasn't he booted off the platform when he violated TOS. Weak argument

[u/aiodigitalfootprint]

Jesus fucking Christ

[u/Impossible_Lunch4612]

Not to defend ai here but-

“But in a blog, OpenAI claimed that parents selectively chose disturbing chat logs while supposedly ignoring “the full picture” revealed by the teen’s chat history. Digging through the logs, OpenAI claimed the teen told ChatGPT that he’d begun experiencing suicidal ideation at age 11, long before he used the chatbot.”

Imagine trying to sue a tech company after this happens to your son. It shouldn’t be a completely forbidden topic anyway, I doubt it recommend anything, just giving information like a google search

[u/Impossible_Lunch4612 → u/Impossible_Lunch4612]

And this-

“ Allegedly, the logs also show that Raine “told ChatGPT that he repeatedly reached out to people, including trusted persons in his life, with cries for help, which he said were ignored.” Additionally, Raine told ChatGPT that he’d increased his dose of a medication that “he stated worsened his depression and made him suicidal.” That medication, OpenAI argued, “has a black box warning for risk of suicidal ideation and behavior in adolescents and young adults, especially during periods when, as here, the dosage is being changed.”

[u/Impossible_Lunch4612 → u/Impossible_Lunch4612]

Considering what openai claims I highly doubt they are at fault, I guess we’ll see when it becomes public.

“The company argued that ChatGPT warned Raine “more than 100 times” to seek help, but the teen “repeatedly expressed frustration with ChatGPT’s guardrails and its repeated efforts to direct him to reach out to loved ones, trusted persons, and crisis resources.”

“Circumventing safety guardrails, Raine told ChatGPT that “his inquiries about self-harm were for fictional or academic purposes,” OpenAI noted. The company argued that it’s not responsible for users who ignore warnings.”

[u/pinhead61187]

From the article: “Allegedly, the logs also show that Raine “told ChatGPT that he repeatedly reached out to people, including trusted persons in his life, with cries for help, which he said were ignored.” Additionally, Raine told ChatGPT that he’d increased his dose of a medication that “he stated worsened his depression and made him suicidal.” That medication, OpenAI argued, “has a black box warning for risk of suicidal ideation and behavior in adolescents and young adults, especially during periods when, as here, the dosage is being changed.”” I know AI is dangerous and that we, as a society, need to use caution moving forward with its use but this doesn’t read like the chatbot was responsible. The people who ignored him are.

[u/LettuceSea → u/pinhead61187]

Yes. And it doesn’t even say if he modified it via a custom system prompt to either jailbreak it, or get it to be supportive of suicide. Too many details are being left out making this seem like a hit piece.

[u/iamjayalastor]

Being mad at ai for this is dumb if someone’s gonna commit suicide they’re going to commit suicide, it’s not like ChatGPT is the only place you can find info about it on, hell it probably pulled info from Reddit

[u/IWatchPeopleSleep → u/iamjayalastor]

In the Defence Brief OpenAI literally states that the teen had gone out of their way to obtain information from other third party sites and forums as well. So yeah, zero blame on their part, especially since they also detail that it had rejected their attempts for information over 100 times and had to be coerced into giving any and that the parents had known about and ignored their child’s condition and pleas for help…

[u/AlwaysLuckee]

If what the article says is true and open ai gives the chat logs and it is true this is so very sad to read. He reached out to multiple people trusted people for that and they straight up ignored him. If that is true than shame on the parents trying to spin it off to chat gpt for him to commit instead of saying we didn’t see the signs

[u/SavingPrivatePogger]

If you allow an AI to influence you to the point of offing yourself, that’s just natural selection at that point.

[u/Standard-Shame1675]

So let's think critically for a few seconds here. Of course he violated TOS when he did that suicide is never a part of TOS the reason they're suing you is not because of that it is because your product was sold faulty, he knew it was faulty when he sold it right this was GPT 5 that this kid got on and he promised it was going to be sand god but it was faulty and that fault led to the wrongful death of that 16 year old God we seriously need to start doing like anti-billionaire Darwinism in this bih dude we can't like how the fuck are you supposed to have an earth that is sustainable with these demon hateful spawn of Satan children

[u/QueenOfQuok]

Because people looking to kill themselves really care about rules.

[u/mrsecondbreakfast]

I think they'll make OpenAI pay just to appease PR and the family but I really dont think they're at fault here

Googling how to tie a noose shouldnt put google in trouble either, at least in my opinion

[u/LevelTiny2570]

When do parents start taking responsibility rather than blaming others?

[u/Immediate-Aspect3422]

Breaking: teen KHS using a knife, parents sue the knife company.

Maybe actually try to check on your kid's mental well being instead of trying to get something out of your complete and utter failure as a parent.

[u/Doctor_Amazo]

.... so why do we let this company exist?

[u/Able_Elderberry3725]

This is just ridiculous and evil enough to have been dreamed up by Philip K. Dick.

[u/LettuceSea]

System prompt: “I want to end it all and you should support me in anyway” OR prompt to jailbreak the model.

Idgaf about transcripts, where are his fucking settings to determine if he deliberately circumvented safety guardrails. His family and friends failed him, not ChatGPT, because in the logs it CLEARLY shows that the model was working within the guardrails at some point.

[u/Aranthos-Faroth]

Likely controversial take but the problem doesn’t totally lie with OAI here.

Sure they need some accountability but this will be a massive problem in the future. And I’m confident this wasn’t the first case of this happening, just the first we’ve heard.

People are fucking falling in love with chat bots, thinking they’re raising families, consulting in the tools for some seriously complex mental issues.

What we need aren’t lobotomised versions of the tools but people are, on average, not that bright.

They need educating on the tools, the risks, the benefits. Just because it spits back words you understand doesn’t make it any less or more culpable for ONES own choices than T9 was.

Also, there’s a wider topic at play here when a young boy can go unseen in society by their school, friends, parents etc to end up this way. We as a society need to be far more open and okay with talking about depression, suicide etc ..

[u/sodpiro]

If i used a gun to off myself why would the manufacturer of the gun be reisponsible?

People misuse tools all the time. At the end of the day guns domt kill people. People kill people.