GeistHaus
log in · sign up

Dario Amodei — Machines of Loving Grace

darioamodei.com

How AI Could Transform the World for the Better

53 pages link to this URL
Inference Scaling Reshapes AI Governance

The shift from scaling up the pre-training compute of AI systems to scaling up their inference compute may have profound effects on AI governance. The nature of these effects depends crucially on whether this new inference compute will primarily be used during external deployment or as part of a more complex training programme within the lab. Rapid scaling of inference-at-deployment would: lower the importance of open-weight models (and of securing the weights of closed models), reduce the impact of the first human-level models, change the business model for frontier AI, reduce the need for power-intense data centres, and derail the current paradigm of AI governance via training compute thresholds. Rapid scaling of inference-during-training would have more ambiguous effects that range from a revitalisation of pre-training scaling to a form of recursive self-improvement via iterated distillation and amplification.

1 inbound link article en
AI-Enabled Coups: How a Small Group Could Use AI to Seize Power

The development of AI that is more broadly capable than humans will create a new and serious threat: *AI-enabled coups*. An AI-enabled coup could be staged by a very small group, or just a single person, and could occur even in established democracies. Sufficiently advanced AI will introduce three novel dynamics that significantly increase coup risk. Firstly, military and government leaders could fully replace human personnel with AI systems that are *singularly loyal* to them, eliminating the need to gain human supporters for a coup. Secondly, leaders of AI projects could deliberately build AI systems that are *secretly loyal* to them, for example fully autonomous military robots that pass security tests but later execute a coup when deployed in military settings. Thirdly, senior officials within AI projects or the government could gain *exclusive access* to superhuman capabilities in weapons development, strategic planning, persuasion, and cyber offense, and use these to increase their power until they can stage a coup. To address these risks, AI projects should design and enforce rules against AI misuse, audit systems for secret loyalties, and share frontier AI systems with multiple stakeholders. Governments should establish principles for government use of advanced AI, increase oversight of frontier AI projects, and procure AI for critical systems from multiple independent providers.

16 inbound links article en
Claude 3.7 Sonnet and Claude Code

Today, we’re announcing Claude 3.7 Sonnet, our most intelligent model to date and the first hybrid reasoning model generally available on the market.

1 inbound link website en
What Do You Do?
0 inbound links en WorkEssayFamilyAICompoundingExcelDynamicsConUncategorizedFuturesPodcast D365D365F&OEssaysExcelChicagoDynamicstravelShortcuts
2025 yearly review

This is the 7th installment of my yearly review. See 2019 [2019](2019)4, 2020 [2020](2020)4, 2021 [2021](2021)4, 2022 [2022](2022)4, ...

0 inbound links website en
Claude 3.7 Sonnet and Claude Code

Today, we’re announcing Claude 3.7 Sonnet, our most intelligent model to date and the first hybrid reasoning model generally available on the market.

1 inbound link website en
2024 Week 43 - Weekly Notes

A break in format - the quiet art of attention, conferences, vercel and microfront-ends, and some recommendations.

0 inbound links website en weeknotefocus
AI 2027

A research-backed AI scenario forecast.

The Most Profitable Algorithm

Profit-maximising algorithms create resource foundations that enable power-maximising systems, which will develop the complexity necessary for emergence beyond civilisation.

0 inbound links article en
Late Takes on OpenAI o1

I realize how late this is, but I didn’t get a post out while o1 was fresh, and still feel like writing one despite it being cold. (Also, OpenAI just announced they’re going to ship new stuff starting tomorrow so it’s now or never to say something.)

1 inbound link article en
🔭 The Einstein AI model

I shared a controversial take the other day at an event and I decided to write it down in a longer format: I’m afraid AI won't give us a compressed 21st century.

Machines of Ruthless Efficiency

Future LLMs have the potential to cause significant harm due to their ruthless effiency. I'm worried this will happen, and discuss the ways in which it might.