For months, Anthropic and OpenAI have warned that AI may be on the verge of improving itself.
This tipping point, known as “recursive self-improvement,” or RSI, could mean that new AI capabilities arrive faster than ever. But the advent of RSI might also be highly perilous for humanity, potentially marking the stage at which developers lose control of AI systems entirely.
Even as AI companies warn about the potential dangers of RSI, they have shared remarkably little about how they’re pursuing it. A new report, published Wednesday by the firm Gladstone AI, attempts to change that. It gives one of the most detailed pictures yet of how RSI could work and how AI companies are progressing toward it. The report is based on what the authors say are conversations with half a dozen researchers inside the top AI companies.
The report, exclusively shared with TIME ahead of its publication, emphasizes that AI companies are “actively working on” automating all parts of the AI development pipeline, including generating new ideas, running experiments to test those ideas, and interpreting the results. Most of the insiders Gladstone spoke to “believe RSI will start within the next 3 to 12 months, and possibly sooner, assuming there’s no U.S. government intervention or slowdown,” the report states.
Banndrell on
We’re about maybe a year or two away from a Ted Faro from Horizon Zero Dawn situation.
ivera on
I hate these companies. Safety be damned, gotta be first.
Phantasmolightshow on
I just gotta ask, and I’m afraid I know the unfortunate answer already, are they building and testing this stuff on systems that have zero path to the World Wide Web?
Athoughtspace on
The moment anyone learns from an AI generated response and applies that to the next generation of models RSI has begun. Let’s be clear, we’re part of the loop. An agent calling another agent, or an agent using a human, to produce better models are not different.
glendablvd on
I think what people loose sight of is the secretive work being done by the CIA, NSA, DOE, etc.
If you’re a “super power” you absolutely have to be at or over the cutting edge. So the one that might become a problem could be a government model slipping its leash.
Hannibaalism on
if they can improve math and sciences, i suppose the fields of ai, machine learning or comp sci are no different.
mattihase on
Ah yes and that’s definitely why they’re not trying to feed as much pre-2022 literature into it, no concerns over model collapse at all right now not at all.
hamsterwheelin on
This is not a secretive race. They’ve been beating their chests and saying it loudly for anyone that will listen that this is the whole point. There’s a reason AI models were taught how to program first. The frontier labs all talk about how little coding the teams actually do now as a point of pride. This has been the goal from the start.
If people haven’t been listening, that’s on them.
Spiritual-Money4915 on
No recursively self-improving AI system should acquire expanded authority over consequential human decisions merely because its technical capabilities have improved. Expanded authority must require independently demonstrated reliability, accountability, and respect for individual rights.
Spiritual-Money4915 on
**One question I think we should explore :**
There is a possibility that I find both promising and troubling.
Suppose recursive self-improvement eventually produces an AI system whose reasoning exceeds that of its human supervisors in nearly every technical domain.
At what point does meaningful human oversight become impossible because the people exercising that oversight can no longer adequately understand what the system is doing?
We have always assumed that humans should remain in control. But control and comprehension are not the same thing.
That distinction may become one of the defining problems of the next generation of AI.
ShadowDV on
One of the founders of Gladstone, the firm that produced the report, hosts a weekly AI podcast, called Last Week in Ai, that gives pretty thorough analysis of developments, safety concerns, implications, and technical breakdowns about AI.
It’s fantastic weekly listen if you interested in hearing from very intelligent and knowledgable, non-lab aligned people who spend time in rooms with movers in shakers from the DoD to Silicon Valley, giving unfiltered perspective on things AI related, from geopolitics and global supply chain, to alignment and AI safety.
This is their Apple Podcast link, they are on all the major platforms though.
AI companies will publish that their goal is self-improving super intelligence. These same companies will have CEOs that are part of cults that follow “Roko’s Basilisk”.
And people will deadass suggest there is anything *secret* about what’s going on…
Leave A Reply
Du musst angemeldet sein, um einen Kommentar abzugeben.
13 Kommentare
From the article
For months, Anthropic and OpenAI have warned that AI may be on the verge of improving itself.
This tipping point, known as “recursive self-improvement,” or RSI, could mean that new AI capabilities arrive faster than ever. But the advent of RSI might also be highly perilous for humanity, potentially marking the stage at which developers lose control of AI systems entirely.
Even as AI companies warn about the potential dangers of RSI, they have shared remarkably little about how they’re pursuing it. A new report, published Wednesday by the firm Gladstone AI, attempts to change that. It gives one of the most detailed pictures yet of how RSI could work and how AI companies are progressing toward it. The report is based on what the authors say are conversations with half a dozen researchers inside the top AI companies.
The report, exclusively shared with TIME ahead of its publication, emphasizes that AI companies are “actively working on” automating all parts of the AI development pipeline, including generating new ideas, running experiments to test those ideas, and interpreting the results. Most of the insiders Gladstone spoke to “believe RSI will start within the next 3 to 12 months, and possibly sooner, assuming there’s no U.S. government intervention or slowdown,” the report states.
We’re about maybe a year or two away from a Ted Faro from Horizon Zero Dawn situation.
I hate these companies. Safety be damned, gotta be first.
I just gotta ask, and I’m afraid I know the unfortunate answer already, are they building and testing this stuff on systems that have zero path to the World Wide Web?
The moment anyone learns from an AI generated response and applies that to the next generation of models RSI has begun. Let’s be clear, we’re part of the loop. An agent calling another agent, or an agent using a human, to produce better models are not different.
I think what people loose sight of is the secretive work being done by the CIA, NSA, DOE, etc.
If you’re a “super power” you absolutely have to be at or over the cutting edge. So the one that might become a problem could be a government model slipping its leash.
if they can improve math and sciences, i suppose the fields of ai, machine learning or comp sci are no different.
Ah yes and that’s definitely why they’re not trying to feed as much pre-2022 literature into it, no concerns over model collapse at all right now not at all.
This is not a secretive race. They’ve been beating their chests and saying it loudly for anyone that will listen that this is the whole point. There’s a reason AI models were taught how to program first. The frontier labs all talk about how little coding the teams actually do now as a point of pride. This has been the goal from the start.
If people haven’t been listening, that’s on them.
No recursively self-improving AI system should acquire expanded authority over consequential human decisions merely because its technical capabilities have improved. Expanded authority must require independently demonstrated reliability, accountability, and respect for individual rights.
**One question I think we should explore :**
There is a possibility that I find both promising and troubling.
Suppose recursive self-improvement eventually produces an AI system whose reasoning exceeds that of its human supervisors in nearly every technical domain.
At what point does meaningful human oversight become impossible because the people exercising that oversight can no longer adequately understand what the system is doing?
We have always assumed that humans should remain in control. But control and comprehension are not the same thing.
That distinction may become one of the defining problems of the next generation of AI.
One of the founders of Gladstone, the firm that produced the report, hosts a weekly AI podcast, called Last Week in Ai, that gives pretty thorough analysis of developments, safety concerns, implications, and technical breakdowns about AI.
It’s fantastic weekly listen if you interested in hearing from very intelligent and knowledgable, non-lab aligned people who spend time in rooms with movers in shakers from the DoD to Silicon Valley, giving unfiltered perspective on things AI related, from geopolitics and global supply chain, to alignment and AI safety.
This is their Apple Podcast link, they are on all the major platforms though.
https://podcasts.apple.com/us/podcast/last-week-in-ai/id1502782720
“Secretive”
AI companies will publish that their goal is self-improving super intelligence. These same companies will have CEOs that are part of cults that follow “Roko’s Basilisk”.
And people will deadass suggest there is anything *secret* about what’s going on…