A Breather in the Intelligence Explosion

What should Anthropic employees do while their work is on pause?

Pieter Bruegel the Elder, The Land of Cockaigne (1567)

The intelligence explosion is taking a breather. The US government has banned non-US-citizens from using Claude Fable (including internally!) over a reported jailbreak. To comply, Anthropic has pulled public access to those models.

At least until they can collect up the passports of their employees and herd the Americans and the foreigners onto separate floors, Anthropic will probably have to stop internal Mythos use too.

An amusing side effect: Anthropic employees might finally have some spare time. They have some of the highest density of human capital in the world. What should they do with it?

1. Write up your (safety) research! (h/t Riya)

Perhaps you have a backlog of safety techniques that you have been meaning to write for a while. You’ve got a new idea, perhaps. Use that compute that isn’t outputting Fable tokens to run it on the latest open-source models. My overall sense is Fable still was not yet at the blog-post writing bar and many still prefer to write everything by hand. Open-source your classifier infrastructure. Of course, don’t leak IP.

Being transparent about your safety research has many benefits. Most obviously, it allows for other labs to be inspired and copy your techniques (race to the top, ya?) But also perhaps it could prevent a situation like this from happening again, from a legal standpoint, where the government can really understand what lengths your poor safeguards team has gone to.

2. Figure out your donation strategy

Anthropic’s valuation probably has a few doublings left in the tank. Your net worth, pretty soon, will be liquid. Much has been said about what to donate to (politics? mass media? prizes?) but donating takes time. Even figuring out who to defer to is pretty hard.

Given the amount of capital that may come into AI safety, in particular, I think having decorrelated bets is pretty important (otherwise what are METR and Transluce going to do with their $100Ms). You should figure out what you’re uniquely excited about, which networks you might trust, and take a moment to evaluate the broader strategic picture and see which things your money can enable to happen.

And, given that e.g. your discount rate for donations should be very high + there are sharp tradeoffs between c3, c4s and hard money donations, the impact of your donations could vary massively.

Use this chance to call up your friends in civil society (hey there), or have a conversation with your trusted fellow Ant who’s thought more about this than you have. Start / contribute to the slack channel.

3. Figure out your taxes.

Building the AGI doesn’t leave much time for life’s mundanities. For many of you, money was never the central goal, anyway. But I would bet there’s a good amount of optimization to be done. My guess is that, in aggregate, on the order of 100s of millions of dollars are being lost, due to optimizations not done. For one, you should probably figure out what your net worth is (ahem). Invest the rest in something safe and decorrelated? I don’t have any particular tax advice, but I can put you in touch with people who do.

4. Write up blogposts, forecasts, and personal reflections

The world is missing the inside story on the development of superintelligence. It’s highly under-appreciated the extent to which employees at AI companies are most often genuinely good people, trying their best, clear-eyed about the trade-offs they are navigating. Write up a diary entry, a personal reflection. Fill out your forecasts for when you expect AGI, when you expect various different milestones, how worried you are about safety.

I think this is great for transparency’s sake, so that the rest of the world can hear from inside, and the historical record. If we make it through, these first-person accounts will be awesome. Spare a thought for the AI historians of the future. Also, in the meantime, perhaps you can get some meaningful time to reflect on what you’re doing, why, and whether things are on track to be okay.

5. Look into a policy transition

If truly what you care about is for AI to go well, rather than the “US winning” or “Anthropic winning”, seriously consider becoming the state capacity that the government so needs. Don’t get me wrong: it won’t be easy. But I still truly believe that, for the fraction of you who are American, likeable and socially-skilled, serving your country is a viable option (even as it tries to destroy your company).

6. Second yourself to an organization in need (like an AI safety nonprofit)

The flip side of so much talent flowing into Anthropic is that the organizations outside it are starved. I like Claude Corps, and sending the young and freshly-trained-in-Claude out into civil society will do real good. But consider the handful of the AI safety nonprofits (with which Anthropic competes, effectively, for talent) could use with some help. Dabble in AI welfare research for a day, perhaps, or lend some engineering muscle.

7. Take a break.

Has there been a lower opportunity cost time to take PTO? It’s now (and while the YOLO run is happening, I guess).

You’re not one of the poor policy people who has to be alert 24/7 lest the government issue another 90-minute directive while you’re on meditation retreat. Open up that poetry notebook. You’ll need your full energies in the run-up to superintelligence.

Thanks to Riya & James for conversation.