UPDATE: Feb. 22,???? ??? 2024, 7:21 a.m. PST This article now includes a statement about this incident from OpenAI.
ChatGPT hallucinates. We all know this already. But on Tuesday it seemed like someone slipped on a banana peel at OpenAI headquarters and switched on a fun new experimental chatbot called the Synonym Scrambler.
This Tweet is currently unavailable. It might be loading or has been removed.
Actually, ChatGPT was freaking out in many ways yesterday, but one recurring theme was that it would be prompted with a normal question — typically something involving the tech business or the user's job — and respond with something flowery to the point of unintelligibility. For instance, according to an X post by architect Sean McGuire, the chatbot advised him at one point to ensure that "sesquipedalian safes are cross-keyed and the consul's cry from the crow's nest is met by beatine and wary hares a'twist and at winch in the willow."
This Tweet is currently unavailable. It might be loading or has been removed.
These arewords, but ChatGPT seems to have been writing in an extreme version of that style where a ninth grader abuses their thesaurus privileges. "Beatine" is a particularly telling example. I checked the full Oxford English Dictionaryand it's not in there, but Wiktionary says it relates to the theologian Beatus of Liébana, a scholar of the end times who died in the year 800, so maybe"beatine" meant "apocalyptic" at some point in the first millennium CE. Or, judging from how it's used in dusty old books, maybe it's just another way of saying "beatific" which one would think is already an obscure enough word. In other words, ChatGPT was giving new meaning to the term "esoteric."
You May Also Like
SEE ALSO: 8 wild Sora AI videos generated by the new OpenAI tool you need to see
The chatbot was briefly doing things like this to tons of its users. One Redditor, homtanksreddit, noted that ChatGPT 3.5 — the one available to free users — was apparently unaffected, so the bug may have only affected paying users.
In a number of cases, ChatGPT started by answering the user's question in clear, contemporary language before devolving into an over-caffeinated version of Martin Prince. One user on Reddit helpfully included a screengrab from ChatGPT that included one prompt used to generate ChatGPT's madness. In it, the user described the bug ChatGPT was experiencing to ChatGPT, and then asked what such a bug is called. Indeed, ChatGPT's answer started off well:
There isn't a commonly used, specific term for when a language model like GPT (Generative Pre- trained Transformer) starts providing output that descends from a clear, relevant response into gibberish or total irrelevance. However, this issue could be described as a type of "output degradation" or a "failure mode" of the LLM (Large Language Model).
Not bad. But then it got weird, noting that "byte-level [...] miscreance can lose institutional shading."
...then it got weirder still:
In real-world application, if you notice an NLP system returning fine commencements that then unravel into lawlessness or written collapse, it may involve jumbled inceptions, affected parts blindness, higher perplexity stoked in modules, or a notably malfunctioned determiner thrust — a multicause sachem, really.
Such behavior is worth documenting for developer autopsy, per se, or cycling in fail-safes, learning the gait, and feed clutch, spanning in relativity so developers might trait and palm to cohere or distend correction."
So there's your answer. ChatGPT was clearly suffering from a bad case of "jumbled inceptions" with a bit of "affected parts blindness" and acute "higher perplexity stoked in modules." Simple as!
Many users wondered on X if ChatGPT was having a stroke:
This Tweet is currently unavailable. It might be loading or has been removed.
...and at least one wondered if he was the one having a stroke:
Related Stories
- ChatGPT will now remember things about you
- OpenAI is adding watermarks to ChatGPT images created with DALL-E 3
- ChatGPT now lets you pull other GPTs into the chat
This Tweet is currently unavailable. It might be loading or has been removed.
But as of Wednesday morning, I was unable to provoke ChatGPT into generating one of these wild outputs, even when I specifically asked it to drone on as much as possible about a boring topic. So it's safe to say the situation was temporary.
Early Wednesday the bug page for this issue said the problem had been identified, but was still being monitored. By late morning, however, the page listed the issue as "resolved." When asked for comment, an OpenAI PR rep referred Mashable to the general status page for ChatGPT, which just says "All Systems Operational," as of this writing.
Mashable asked OpenAI to elaborate on what had happened, perhaps in an obscurantist and grandiloquent style, but the request was not immediately granted in the fullness of our unstinting if somewhat caviling journalistic desiderations.
On Wednesday evening, OpenAI published the following "postmortem" on the incident page for this bug:
On February 20, 2024, an optimization to the user experience introduced a bug with how the model processes language.
LLMs generate responses by randomly sampling words based in part on probabilities. Their “language” consists of numbers that map to tokens.
In this case, the bug was in the step where the model chooses these numbers. Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense. More technically, inference kernels produced incorrect results when used in certain GPU configurations.
Upon identifying the cause of this incident, we rolled out a fix and confirmed that the incident was resolved.
Topics Artificial Intelligence