Photo-Illustration: Intelligencer; John Herrman

Following a string of reports about users developing intensely unhealthy relationships with ChatGPT, OpenAI has formulated a response. “There have been instances where our 4o model fell short in recognizing signs of delusion or emotional dependency,” the company says, so it’s “developing tools to better detect signs of mental or emotional distress so ChatGPT can respond appropriately.” When it comes to “personal challenges,” such as relationship advice, the company says “ChatGPT shouldn’t give you an answer” but rather “help you think it through.” Never fear: “New behavior for high-stakes personal decisions is rolling out soon.”

A third item stood out as somewhat more familiar: “Starting today, you’ll see gentle reminders during long sessions to encourage breaks. We’ll keep tuning when and how they show up so they feel natural and helpful.” ChatGPT, like so many tech companies before it, is learning to love the nudge.

If you spend a lot of time on your smartphone — particularly if you spend a lot of time on social media — you’ve been nudged before. Apple’s software is full of nudges, from reminders to turn down the volume or move the screen farther from your face to the aspirational behavior modification implied by its Screen Time monitor features; Google’s smartphone software has its own “Digital Wellbeing” suite with similar features. TikTok allows its users to set daily limits and trigger reminders to take a break and to get some sleep; the app recently started testing “Well-being Missions,” or tasks intended to “help people develop long-term balanced digital habits” and to “encourage and reinforce mindful behaviors.”

Photo: TikTok

Instagram has “Take a Break” reminders, daily time alerts, and a usage dashboard, while YouTube nudges young users by pausing videos and suggesting they take a breather.

As optional features, these nudges range from harmless to good to have; as standard behaviors, they may sometimes seem patronizing, but they’re both minimally obtrusive and not terribly effective. Broadly speaking, life on a commercial tech platform is almost entirely nudges — to use TikTok, for example, is to be nudged around constantly, toward new content and ads and back to your feed again — but OpenAI’s new feature joins the narrower tradition of healthful nudges intended to either fix users or, more charitably, help them help themselves. They tend to appear, as in the case of Apple’s Screen Time features, in response to swings in public sentiment and critical press reports (in Apple’s case, around smartphone addiction and worries about young people spending too much time on their devices).

New nudges also offer clues about how people are actually using these products. While OpenAI begins its announcement with an assurance that its “goals are aligned with yours” and that “instead of measuring success by time spent or clicks, we care more about whether you leave the product having done what you came for,” lengthy engagements with chatbots — encompassing everything from therapeutic chat to fantasy role-play — are clearly a large and growing use case. Everything from chatbots’ personification to their recurring issues with engagement-boosting sycophancy suggests that entertainment, not just utility, is what keeps a lot of users around. As a communications strategy, nudges project ambivalence about excessive platform use and raise the possibility that users may be misusing products. As a bit of marketing, they’re likewise helpful: Our product is so compelling we have to tell our users to take a break.

Nudges are perhaps best understood, though, as attempts to demarcate liability. They imply concern for users’ well-being and offer tools to help them manage it. But they also make it clear that, should these new products drive you sort of insane, that’s ultimately your problem, not theirs. You were warned! Or at least you were nudged.

More screen time

SEO Is Dead. Say Hello to GEO.Age Verification Is Coming for the Whole InternetMark Zuckerberg Finally Submits His AI Manifesto

From Intelligencer - Daily News, Politics, Business, and Tech via this RSS feed