Eine mathematische Obergrenze begrenzt die generative KI auf Kreativität auf Amateurniveau. Während generative KI/LLMs wie ChatGPT die Arbeit einer durchschnittlichen Person überzeugend nachbilden können, ist sie nicht in der Lage, das Niveau erfahrener Autoren, Künstler oder Innovatoren zu erreichen.
A mathematical ceiling limits generative AI to amateur-level creativity
24 Kommentare
I’ve linked to the news release in the post above. In this comment, for those interested, here’s the link to the peer reviewed journal article:
https://onlinelibrary.wiley.com/doi/10.1002/jocb.70077
From the linked article:
**A mathematical ceiling limits generative AI to amateur-level creativity**
A new theoretical analysis published in the Journal of Creative Behaviour challenges the prevailing narrative that artificial intelligence is on the verge of surpassing human artistic and intellectual capabilities. The study provides evidence that large language models, such as ChatGPT, are mathematically constrained to a level of creativity comparable to an amateur human.
To contextualize this finding, the researcher compared the 0.25 limit against established data regarding human creative performance. He aligned this score with the “Four C” model of creativity, which categorizes creative expression into levels ranging from “mini-c” (interpretive) to “Big-C” (legendary).
The study found that the AI limit of 0.25 corresponds to the boundary between “little-c” creativity, which represents everyday amateur efforts, and “Pro-c” creativity, which represents professional-level expertise.
This comparison suggests that **while generative AI can convincingly replicate the work of an average person, it is unable to reach the levels of expert writers, artists, or innovators**. The study cites empirical evidence from other researchers showing that AI-generated stories and solutions consistently rank in the 40th to 50th percentile compared to human outputs. These real-world tests support the theoretical conclusion that AI cannot currently bridge the gap to elite performance.
“While AI can mimic creative behaviour – quite convincingly at times – its actual creative capacity is capped at the level of an average human and can never reach professional or expert standards under current design principles,” Cropley explained in a press release. “Many people think that because ChatGPT can generate stories, poems or images, that it must be creative. But generating something is not the same as being creative. LLMs are trained on a vast amount of existing content. They respond to prompts based on what they have learned, producing outputs that are expected and unsurprising.”
Our ai is the “we have ai at home” version of artificial intelligence
This puts more wood behind the observation that LLMs are a useful helper for senior level software engineers, augmenting the drudge work, but will never replace them for the higher level thinking.
My guess is this ignores the existence of RL-based post training
So what you’re telling me is AI is just an acronym for Average Intelligence… I thought these things were supposed to be learning on their own and reaching some sort of singularity….
Tell that to my ex-employer.
I wouldn’t bet on this.
Quite the insult to anyone replaced by AI (assuming they actually, successfully, were).
Conversely, if the model were to select a word with a very low probability to increase novelty, the effectiveness would drop. Completing the sentence with “red wrench” or “growling cloud” would be highly unexpected and therefore novel, but it would likely be nonsensical and ineffective. Cropley determined that within the closed system of a large language model, novelty and effectiveness function as inversely related variables. As the system strives to be more effective by choosing probable words, it automatically becomes less novel.
Anyone that knows anything about AI already knew this. It doesn’t make it any less powerful of a tool.
Ai music has been proving this everyday for years.
This seems like word salad trying to roughly rephrase the standard (trivially incorrect) claim that LLM’s just average their training data.
By their definition a sentence created by rolling duce to select totally random words from the dictionary would be maximally „creative“
That’s a pretty low bar: a potato can convincingly replicate the work of the average idiot human
>To evaluate the creative potential of artificial intelligence, the researcher first established a clear definition of what constitutes a creative product. He utilized the standard definition of creativity, which posits that for an output to be considered creative, it must satisfy two specific criteria: effectiveness and originality.
Per my handle, I think I’m well suited to opine on this. I dispute his definition of creativity as it excepts all fiction or fantasy, for one. I’m also surprised that he doesn’t reference Stephen Thaler or DABUS, an AI specifically built to be creative (although whether it is is a different argument).
Personally, I agree that AI is not currently creative – at least, as we currently architect it. Though I think there are strong arguments to the contrary, Thaler being the most likely person to provide them.
Edit: removing double negative
For music, they don’t care. They’ll let independent artists solving away on minimum wage jobs innovate, steal any money they actually do make because they don’t make enough. (See Spotify’s payout methods if you don’t believe me). And then their AI will be able to make pop hits in this new innovative style.
This really doesn’t help much.
If this is truly a ceiling, then it will never replace experts, but it only needs to replace the bulk of average employees to completely destroy our entire economy.
How do you quantify creativity? I didn’t know you could measure how creative a given work is, how does that work?
I’ve heard that the big bottleneck of LLMs is that they learn differently than we do. They require thousands or millions of examples to learn and be able to reproduce something. So you tend to get a fairly accurate, but standard, result.
Whereas the cutting edge of human knowledge, intelligence, and creativity comes from specialized cases. We can take small bits of information, sometimes just 1 or 2 examples, and can learn from it and expand on it. LLMs are not structured to learn that way and so will always give averaged answers.
As an example, take troubleshooting code. ChatGPT has read millions upon millions of Stack Exchange posts about common errors and can very accurately produce code that avoids the issue. But if you’ve ever used a specific package/library that isn’t commonly used and search up an error from it, GPT is beyond useless. It offers workarounds that make no sense in context, or code that doesn’t work; it hasn’t seen enough examples to know how to solve it. Meanwhile a human can read a single forum post about the issue and learn how to solve it.
I can’t see AI passing human intelligence (and creativity) until its method of learning is improved.
I could see this from experimentation, let me know if this is a good understanding:
AI can easily do 1 degree removed from its data set (a dog with a snail shell = dog x snail)
and pretty well do 2 degrees removed from its data set (snail dog in a tree = (dog x snail)(tree)
And so on, but every degree removed from its original data set requires exponentially more “creativity” so we come to a mathematical limit.
if it reaches the point where i can be smarter than the material it was trained on and make novel discoveries on its own, then that’s kind of general intelligence rather than artificial.
This is why companies like Disney are not investing in AI. They know and understand that they cannot get these models to do better than actual masters of the craft. Anyone who works in the creative industry can tell you that, and as a hobby artist I’ve been saying it since it’s inception. You can feed all the phrasing you want into the machine but it’ll never be able to envision what a true artist can. For many businesses AI might be „good enough“ but it won’t be for the entertainment industry if they want to be competitive.
See: https://www.reddit.com/r/ArtistHate/comments/1i3a8qx/yall_mind_if_i_just_go_off_on_an_image/
Also, let’s not forget that you can easily add “soulful” to the prompt.
I’ve seen the kind of japanese entertainment ai can produce and i beg to differ.
I’m as skeptical as the next person about AI’s future, but these points feel weak to me. (A) Humans build on what we’ve seen, so Im not sure originality point is true. (B) the forward projection assumes future AI will just be larger/faster versions of today’s LLMs. IMO there is significant odds of innovations that they fail to consider