Posts tagged: Anthropic

Dario Amodei still has a lot to say.

On Monday, the Anthropic CEO dropped an over 19,000-word essay entitled “The Adolescence of Technology” on the future of AI on Monday, opining on everything from his fellow CEOs to feudalism, and even the Unabomber.

Best known for his warning that AI could eliminate up to 50% of entry-level white-collar jobs in the next 1 to 5 years, Amodei has tangled with Nvidia CEO Jensen Huang and the Trump White House over his views.

Here are seven of the most alarming and surprising quotes.

‘This is a serious civilizational challenge’

Amodei remains optimistic about AI overall, but his essay detailed “an intimidating gauntlet that humanity must run” to reap the benefits of AI without letting the breakthrough technology destroy the world.

“I believe if we act decisively and carefully, the risks can be overcome — I would even say our odds are good. And there’s a hugely better world on the other side of it,” he wrote. “But we need to understand that this is a serious civilizational challenge.”

AI development can’t be stopped, Amodei wrote, a conclusion even some of AI’s skeptics share. The financial and security benefits are just too massive for the private and public sectors to pass up.

It’s why winning the AI race and doing so in an ethical way is so critical, he concludes.

‘This is like selling nuclear weapons to North Korea and then bragging that the missile casings are made by Boeing’

Jensen Huang hasn’t changed Amodei’s mind on China.

“A number of complicated arguments are made to justify such sales, such as the idea that ‘spreading our tech stack around the world’ allows ‘America to win’ in some general, unspecified economic battle,” Amodei said. “In my view, this is like selling nuclear weapons to North Korea and then bragging that the missile casings are made by Boeing and so the US is ‘winning.'”

In November, Nvidia announced a partnership with Anthropic that includes an investment of up to $10 billion in the AI startup. The news sparked speculation that tensions between Amodei and Huang might be cooling.

Whatever the status of their relationship, Amodei is resolute that it is a horrendous decision to allow US companies to sell advanced chips to China.

“China is several years behind the US in their ability to produce frontier chips in quantity, and the critical period for building the country of geniuses in a data center is very likely to be within those next several years,” Amodei wrote. “There is no reason to give a giant boost to their AI industry during this critical period.”

‘Many people have told me that we should stop doing this, that it could lead to unfavorable treatment’

Amodei would like his critics to see the scoreboard.

Anthropic’s leader hasn’t tried to curry favor with the White House, nor has he vocally embraced President Donald Trump’s AI policies to the same degree as his rival CEOs. Amodei’s outspoken call for AI regulation even led David Sacks, Trump’s AI czar, to publicly rebuke him.

None of it has changed Amodei’s view that the AI industry “needs a healthier relationship with government — one based on substantive policy engagement rather than political alignment.”

“Many people have told me that we should stop doing this, that it could lead to unfavorable treatment, but in the year we’ve been doing it, Anthropic’s valuation has increased by over 6x, an almost unprecedented jump at our commercial scale,” he wrote.

Of all of his hopes, this one appears the unlikeliest. Already, AI CEOs have formed dueling super PACs ahead of the 2026 midterm elections.

‘It is sad to me that many wealthy individuals (especially in the tech industry) have recently adopted a cynical and nihilistic attitude that philanthropy is inevitably fraudulent or useless’

The tech elite made AI, and they should help society grapple with its fallout, he wrote in the essay. Amodei has long called on governments to prepare for mass job displacement. In one of the most eyebrow-raising parts of the essay, Anthropic CEO detailed what his fellow billionaires and companies must do.

Beyond philanthropy, Amodei said companies need to be “creative” in how they stave off layoffs.

In the long term, he wrote, “It may be feasible to pay human employees even long after they are no longer providing economic value in the traditional sense.”

‘Some AI companies have shown a disturbing negligence towards the sexualization of children’

One of the biggest themes of Amodei’s essay is the risk that AI companies themselves pose. It’s a conclusion that he admits is “somewhat awkward” for him to reach. As an example, he points to the roiling topic of the sexualization of children. While he does not name xAI directly, Grok is facing investigations in multiple countries over the non-consensual sexualization of images of real people.

“Some AI companies have shown a disturbing negligence towards the sexualization of children in today’s models, which makes me doubt that they’ll show either the inclination or the ability to address autonomy risks in future models,” he wrote.

Overall, he expressed skepticism that AI companies will sacrifice profit for broader societal good. “Ordinary corporate governance,” Amodei wrote, is ill-equipped to address his worries.

Amodei said that fears that AI models may defy orders and perhaps even try to eliminate humanity are complicated by bad actors in the industry who aren’t as transparent about the risks they are seeing in their models.

“While it is incredibly valuable for individual AI companies to engage in good practices or become good at steering AI models, and to share their findings publicly, the reality is that not all AI companies do this, and the worst ones can still be a danger to everyone even if the best ones have excellent practices,” he wrote.

‘Models are likely now approaching the point where, without safeguards, they could be useful in enabling someone with a STEM degree but not specifically a biology degree to go through the whole process of producing a bioweapon’

Amodei doesn’t see the largest risks to humanity coming from AI pursuing total domination, but rather in what AI could enable humans to unleash.

Amodei described his fears that AI is lowering the barrier of entry necessary to make killer biological weapons. His greatest concern is that AI could provide the step-by-step know-how that could eventually enable even an average person to produce a bioweapon.

AI companies, Amodei said, need to ensure they create sufficient backstops to block such inquiries, including by making it difficult for hackers to jailbreak models. Adding such security is expensive, Amodei said, noting that these measures are “close to 5% of total inference costs” for some of the companies’ models.

“I am concerned that over time there may be a prisoner’s dilemma where companies can defect and lower their costs by removing classifiers,” he wrote. “This is once again a classic negative externalities problem that can’t be solved by the voluntary actions of Anthropic or any other single company alone.”

‘I would support civil liberties-focused legislation (or maybe even a constitutional amendment)’

Amodei is one of the AI industry’s most vocal proponents of AI legislation. While Meta and Microsoft supported a federal preemption of state-level AI laws, Anthropic supported AI transparency bills in California and New York that are now law.

Throughout the essay, Amodei outlined multiple areas for future legislation, including industry-wide transparency requirements like those at the state level. Even he concludes that new laws might not be enough.

“The rapid progress of AI may create situations that our existing legal frameworks are not well designed to deal with,” he wrote.

It’s why Amodei said he would go so far as to support a constitutional amendment. The US has not amended the Constitution since 1992, when the over two-century-long battle to add a limitation on congressional pay finally passed the 38th state legislature.

“I would support civil liberties-focused legislation (or maybe even a constitutional amendment) that imposes stronger guardrails against AI-powered abuses,” he wrote.