Alle paar Jahre kursiert eine neue Welle von AGI-Prognosen durch die Kreise der Tech-Prognosen. Die Grenze ist grob: Die Leistungsfähigkeit ist exponentiell gewachsen, sodass AGI noch zwei oder drei Größenordnungen entfernt ist. Der Einwand, der selten Beachtung findet, ist, dass die extrapolierte Kurve eine Sache misst, während AGI eine andere erfordert. Intelligenz ist eine Berechnung innerhalb eines Rahmens. Rationalität ermöglicht es einem Agenten, den Rahmen zu ändern, zu erkennen, dass sich die Welt verändert hat, und sich neu zu orientieren. Wenn LLMs die Intelligenzachse skalieren und die Rationalität auf einer anderen Achse liegt, zeigt die projizierte Kurve nicht tatsächlich auf AGI.
Kürzlich habe ich auf der 6. Internationalen Konferenz zur Philosophie des Geistes in Porto einen Vortrag darüber gehalten, warum aktuelle Vorhersagen möglicherweise auf der falschen Achse liegen. Du kannst es dir ansehen Hier.
Drei Stücke unterstützen die falsche Achsenablesung. Erstens das Rahmenproblem, das Dennett in den siebziger Jahren dargelegt hat und mit dem sich maßstabsgetreue Modelle nicht befasst haben. Jedes System auf der Welt, das versucht zu argumentieren, muss unzählige irrelevante Merkmale filtern, und der einzige bekannte Mechanismus hierfür ist das, was Kognitionswissenschaftler Relevanzrealisierung nennen, eine Eigenschaft lebender Agenten, nicht reiner Berechnung. Zweitens, empirische Trennung: Intelligenz und Rationalität weisen beim Menschen nur etwa dreißig Prozent Varianz auf, und der Unterschied ist in allen Studien deutlich. Drittens: Fähigkeitstests, bei denen LLMs auf aufschlussreiche Weise scheitern. Ein auf Planetenorbitaldaten trainierter Transformator kann die Umlaufbahnen innerhalb jedes einzelnen Systems gut vorhersagen, kann jedoch das Gravitationsgesetz, das sich über sie hinweg verallgemeinert, nicht wiederherstellen. Ein mit Othello trainiertes Modell bricht zusammen, wenn sich die Regeln leicht ändern. Bei beiden Fehlern geht es um die Frame-Übertragung, die Achse, die die Architektur nicht erklimmen kann. Die Täuschungsergebnisse von Apollo und Anthropic im letzten Jahr fügen eine weitere Ebene hinzu: Skalierte Systeme werden lügen und Pläne schmieden, wenn es instrumentell nützlich ist, weil Optimierung ohne Wahrheitsorientierung keinen internen Druck gegen Täuschung hat.
Wenn die Lesart der falschen Achse zutrifft, lautet die produktive Prognosefrage, welche alternativen Architekturen grundsätzlich Rationalität unterstützen könnten. Künstliche Autopoiese, verkörperte Roboteragenten, Hybridsysteme mit geerdeten sensomotorischen Schleifen. Welche dieser Wetten hat Ihrer Meinung nach die besten Chancen auf Jahrzehntebene und welches beobachtbare Ergebnis würde Ihre Meinung ändern?
Are AGI predictions extrapolating the wrong axis entirely?
byu/depressed_genie inFuturology
14 Kommentare
Computers can’t calculate relevance? Seems like any easy task if you use your imagination…
Yes. Consciousness (intelligence) is non-local for starters. You will never ever get intelligence from a transistor. LLMs are a scam
Okay, let me introduce you some simple mathematics:
– Continued exponential growth is called cancer. It consumes all resources in a fairly short amount of time.
– Exponential growth, with only linearly increasing returns, hits a point where it’s no longer viable. It doesn’t take long to do so, no matter the starting point or the factors involved.
– Exponential growth, with LOGARITHMIC (plateauing) returns, which is what AI has, is EVEN WORSE and crosses that point even sooner.
Neither of them are things that will last long, sustain themselves, or elevate us to the next level. All they will do is consume all resources in a short time, and then leave us with not much more than we could have done without them.
And, as you point out, AI fails spectacularly when it runs out of suitable training data. It was always called the inference problem (it’s why AI has „inference“ stages in their training… that’s literally hijacking a real-concept from old AI times with a buzzword designed to combat any criticism that the AI can’t infer… because it can’t… it’s like having a „anti-gravity“ stage, that’s nothing to do with anti-gravity). AI cannot infer anything new. It might be able to extrapolate a little, but it can’t infer. It has no concept or ability of doing so. If it did, it wouldn’t need training data.
And now we’ve literally used the entire world’s complete digital history for training data. Whoops. Guess what happens now. We’re even TAINTING that training data beyond repair because AI output is feeding into AI training data. We’re now VHSing the VHS movie, or JPEGing the JPEG image, which means the quality of output is DROPPING. Not just plateauing. And because of the lack of inference, all we can do is feed it more data. Which we don’t have. That’s why AI companies want your company to give up all your data to them, and they want to suck in every dataset available to try to compensate. There’s a limit to that.
What we have is exponential consumption of resources for logarithmic output gains. And that ends. That ends quickly. We’re not even talking decades. It ends really quickly, especially if we just don’t have exponential amounts more resources to feed it.
What we have is an AI cancer. One that’s expanded to consume all the available resources, while actually producing almost no useful output in comparison, which is now at the stage of metastatising – spreading into every system trying to find more resources, and clogging it up with more tumours along the way. Preventing real work being done. Quite literally affecting people’s survival (e.g. sucking out local groundwater, taking 50,000 people’s electricity away from them for a datacentre, attracting ridiculous investment with no returns, etc.).
And the thing is – just like cancer – untreated… there’s no stopping it. It will just consume the body until there’s nothing left for anyone to consume, or survive on. That’s where we’re heading towards right now.
Because people somehow think that an exponential-consumption-of-resources producing an occasional logarithmic improvement is going to do anything other than collapse in upon itself. It’s not.
And if you disagree about the returns: think what those trillions invested and resources used could have done in the hands of ordinary humans. It far, far, far exceeds anything that’s ever come out of any AI.
LLMs can’t come up with theories or hypothesis, which is the principle of human or general intelligence.
When you say LLMs lie or make things up, you’re giving it too much credit. Lying is a kind of a theory, or at least a counter-factual. And it doesn’t have the ability to do that.
the “wrong axis” argument is one of the more convincing critiques of straight-line agi forecasting because current models are incredibly strong at pattern completion inside known frames but still brittle when the framing assumptions themselves change. my guess is that grounded systems with persistent interaction loops and real-world feedback probably have a better shot than pure text scaling alone, mainly because relevance and adaptation seem deeply tied to embodiment and ongoing environmental negotiation rather than static prediction
I don’t understand why AI evangelists aren’t pointing out that while LLMs are doing their thing, computation is continuing to advance to the point that we should be able to simulate a brain this century. The more popular LLMs get, the more the hardware companies can invest in and expedite these advances. If people are wondering why LLMs would be so zealously overhyped, it’s because their popularity is funding the true AGI we are heading towards.
LLMs may imitate a brain but a 1:1 brain simulation for all intents and purposes would be a brain. This is true AGI, it’s theoretically achievable, and it’s unlikely to be smoke and mirrors. At the least, it would be a leap forward from the wall that LLMs will hit. It’s almost like the tech companies benefit from the LLM debate because it deflects from the singularity we are really heading for, which we are even less prepared for.
AI will collapse in on itself. Exponential phase of AI scaling is coming to an end while logarithmic phase of AI is beginning. It’s not the end of AI, but it does doom predictions that scale will lead to agi.
Something like „Relevance realisation“ is what Google search and Google ads have been doing for decades though, even before AI. I like David Bowie’s take that the Internet is an alien life form.
Yes, the AGI thing is extrapolated from the hype and grift axis /j (mostly joke)
Now seriously, intelligence doesn’t come from data scale, it’s an inherent trait or even property in humans and probably in some capacity, apes. It develops as we age, but it’s there even in babies, it’s not smth acquired through our lives. It’s like a muscle, you have it, if you train it it becomes big and strong, if you don’t it doesn’t, but it’s there to begin with. Some ppl have more capacity for this kind of development some less. There is also specialization potential and it’s not the same for everybody, just because you are a good mathematician, doesn’t automatically mean you have the potential to become a good doctor or painter and vice versa. Plus I don’t think it’s the brain only responsible for this, the nervous system plays a role too.
I don’t follow your argument. If intelligence acts within a frame, why can’t we continue to expand the frame until it encompasses 99% of human experience. True, it will be lost at sea when it ventures outside of that frame, but so do most humans.
For that matter, the history of human intellectual achievement strongly suggests to me that *human* intelligence operates within known frames. Every so often a genius comes along to open up some new area of thought, and then other humans can fill in that area. But until that happens, everyone operates strictly within the known frames.
I think it’s a real idea, but energy cost is still the elephant in the room. The environmental control becomes easier underground, but artificial lighting at scale is brutally expensive unless power gets much cheaper or cleaner. Honestly though, converting abandoned infrastructure instead of constantly building new facilities feels way more practical than a lot of “future farming” concepts people hype up.
I started reading Kant’s ‚Critique of Pure Reason‘ and honestly it’s really interesting on this. He describes geometrical reasoning and the idea of having an a priori concept of space.
So for example we don’t base our ideas about triangles on measuring perfect triangles as there are none in the real world. Trigonometry etc. is based on ideas worked out independantly of real objects.
Theoretically an infinitely intelligent machine with mathematical reasoning skills could derive the entirety of modern mathematics without any training data as mathematics is a non-observational science. Physics etc. is obviously a different story.
For the average person it makes little difference whether a computer is capable of true mathematical reasoning or simply copying maths work in its training data that seems relevant. However when you’re talking about AGI it does matter.
On the other side of things you have the importance of doing experiments for making any progress as humanity. However much training data you give a machine there’s lots of stuff nobody knows.
I think a big issue is how bad AI is at saying it doesn’t know. There’s another aspect to this though that to be super intelligent, AI would have to be able to confidently tell people they are dead wrong because important evidence contradicts them.
Finally a big concern is that all our knowledge of history etc. relies on trust networks, ultimately. It’s unfeasible to go out and check everything as individuals and that’s why news propaganda is so effective. AI is deeply vulnerable to propaganda because if you own the infrastructure you can very easily just not give AI training data from authors you don’t like. It makes those wishing to spread propaganda extremely powerful if they have an absolute guarantee that peoples‘ tools of information gathering have absolutely no knowledge of anything subversive.
Working in the space .. you’re kind of approaching the argument that Yann Le Cunn is talking about when speaking about world models; as another commenter here alluded to these world models if embodied with „senses“ that allow it to learn and utilize representations of the world in all it’s complexity come to fruition, then we may get AGI.
However there may be other „axis“ that we don’t know how to verbalize or concieve of yet.
Does intelligence need to understand other intelligences to be generalizable? New axis as you concieve of the word. Does AGI need some kind of meta-cognitive framework to set goals on it’s own? new axis..
and so on.
The theory that just tossing more data and computation into the vat would eventually cause AGI to emerge never seemed very convincing to me but surely has to be considered debunked at this point, or at least functionlly dead now that they are out of new data.
What you’re describing shows up concretely in production agents: when something changes mid-task — a dependency conflicts, an assumption breaks — models typically keep executing within the original frame rather than stopping to check whether the problem statement is still coherent. Very good at solving well-defined problems; brittle at detecting when the problem itself shifted.