Summary: We start with a primer on compute verification: understanding what’s going on inside a datacenter and how we can prove it to someone else. This involves proving certain properties about compute, e.g. where it is, what it’s doing, and how much was used, under particular trust assumptions. We then argue:
Inference-only verification is the highest-value near-term property and the highest-priority target for large-scale demo buildout,
International verification will never be trustless, only low-trust, so we should focus on stacking decorrelated defense-in-depth measures,
Verification today looks like compute governance in 2021: obviously important in retrospect; pre-paradigmatic; lacking in talent, R&D and policy effort.
We finish with a list of shovel-ready projects for funders, researchers, and fellow weekend warriors: we’re most excited about building a commercial-scale verified datacenter, and funding physically accessible datacenters for researchers.
The field of “compute verification” -- proving what’s happening inside a datacenter -- is having a moment. A few weeks ago, Anthropic asked for mechanisms that could support an international “credible slowdown or pause” of AI progress. AI 2040: Plan A (and Romeo Dean’s excellent verification supplement), published last week, is some state-of-the-art futurism about the best case for verification. And the OpenAI Foundation, now the largest nonprofit foundation in the world, may choose to spend some of its billions here.
We’ve spent the last few months looking into the state of the field. To make a low-trust international deal between the U.S. and China possible, this field needs to get its act together. Designing and prototyping an effective verification regime won’t create the political will for a deal, but without doing so, such a deal will not be possible. Here’s our little contribution.
What follows is:
an accessible primer on compute verification,
some observations on the technical approaches, and state of the field,
and a list of projects we’d love someone to start tomorrow.
What’s going on inside that datacenter?
In AI governance, compute verification asks the question: what’s going on inside a datacenter, and how can we prove it to someone else?
Quickly: “compute” is the field’s shorthand for the hardware used to build and query AI models (in the lingo, “train” and “run inference on”).1 “Verification” here refers to the task of demonstrating certain properties of that compute -- and of how it’s being used -- to a party who isn’t happy taking your word for it.
What you can prove
Mauricio Baker et. al’s excellent RAND report splits compute verification into (a) checking that compute is generating “artifacts” (AI models, AI outputs, etc.) that comply with agreed rules, and (b) checking that there is no undeclared large-scale compute. However, we find it easier to grasp via example properties. The first below is RAND’s category (b); the rest are flavors of (a). To start:
Existence: Whether there exists a certain amount of compute in a given location (“location verification”), established by satellite imagery, power draw, supply-chain tracking or any number of creative methods. You cannot prove anything about workloads being run on compute that you don’t know exists.2
Training vs. inference. Whether a chunk of compute is being used for building models or just running them.3 This property underlies Plan A’s deal: of all the things you might show about a datacenter, the most slowdown-relevant is that it’s only doing inference: monitoring or slowing recursive self-improvement toward superintelligence means monitoring training, which (at least at the beginning) is enormously compute-intensive.
Compute thresholds. Proving that even a permitted training run stays under some FLOP ceiling. Back in 2023, when the regulator-folk had greater ambitions, some floated the idea of a hard threshold of 10^25 or 10^26 FLOP (around the number of “floating point operations” you’d need to train GPT-4 and GPT-5, respectively) to define and regulate what counted as a “frontier AI model.”
Model identity / provenance: Whether the model deployed is the same model that was evaluated. We haven’t seen egregious violations of this yet (some amount of “oh METR did the safety testing on an earlier checkpoint”). But when deployments, internal and external, become higher-stakes (or regulators want to see compliance to future laws), this question of proving that the model being used is the same one that was tested will be important. This is useful for proving to employees that you’re abiding by your RSP, or to regulators that the model you released is the same one that passed evaluation (“evaluation gating”).
Many other things: Workload provenance (the deployed model really came from the training run the developer describes -- X code, Y data, Z hyperparameters, not swapped or misrepresented), data provenance (not trained on, say, bioweapons data), inference-use monitoring (a deployed model isn’t being queried to make one), and so on. These are mostly secondary or nice-to-have for current governance proposals, and occasionally require different access or architectures.
Who do you trust?
The other core concept, alongside which property you prove, is your trust assumptions: how much must the verifier assume about the prover’s trustworthiness?
Every verification setting has a prover, someone making a claim, and a verifier, someone checking. The relationships we might care about include between AI model developers and their customers, between developers and regulators, and between states. Verification tends to be required in a low-trust environment between adversarial parties. For example, a developer might have strong incentives to misreport, and also wants privacy, so it may not let a verifier see raw data, weights, workloads, or records of computations (“logs”).
Consider a decreasing scale trust assumptions about a prover that a verifier could take on:
You have full trust in the prover’s honesty, and verify nothing (the current state of the world)
You trust the prover’s high-level self-report.
You trust the prover to hand over complete, faithful logs, but you run the analysis yourself.
You trust logs only once they’re cryptographically signed, or you audit a sample (checking they describe the real computation, not some other run entirely).
You stop trusting the prover’s word or logs and lean on a hardware root of trust, eg. attestation from a chip vendor’s “trusted execution environment” (an isolated part of the chip that will record if it has been tampered with). Note this doesn’t remove trust so much as move it: you now have to trust NVIDIA or Huawei instead.
You trust only your hardware, e.g. instruments that you install yourself like network taps. You don’t trust their hardware, you don’t trust them not to tamper with yours, and you don’t trust them not to smuggle data on and off chips (USBs, side channels)
And above all, and at the end, is that you don’t fully trust anything -- you never reach certainty because of unknown unknowns. You can only accumulate degrees of confidence, by stacking independent, decorrelated measures, such as espionage of internal company comms, power-draw readings, satellite estimation. Adversaries can be very tricky indeed: to badly paraphrase Luke Marks, one can imagine the hardest version of the security game features an adversary firing neutrinos at your chips to load data that shouldn’t be there. And even that achieves a reasonable bit rate!
The project of verification R&D is to relax these trust assumptions: building measures to develop confidence under ever-fewer assumptions about the prover’s goodwill. Our guess, given political realities, is that the U.S.-China agreement case lives the highest level: international dealmaking will only be on the table when both countries want it, assuming some level of baseline openness, but you never fully trust anything. The goal of verification, therefore, is not a solution robust to nation-state adversaries out of the gate; it is steadily increasing defense-in-depth.
Oh, also, in the real world, the prover might have to trust the verifier (not to steal their IP, sabotage their training runs etc.) This is an easier problem because the verification systems are simpler, and can be more easily open-sourced. One relevant distinction is structural vs semantic approaches to verification (relying on information about the shape of the data, such as the amount of bandwidth consumed, compared to looking at the data itself), of which we’d much prefer the verifier seeing as little as possible.
Now, a little more on how exactly you do this verification.
Notes on technical approaches
At a high level, all verification proposals have to answer three questions:
Where does the data about the prover come from? This might be from on-the-ground inspectors (old-school nuclear-style), developers handing over logs, devices that capture data being transferred between chips or look into the chip itself, or remote data collection (e.g. looking at satellite imagery).
What can you conclude from the data? For analysis, a verifier could use any number of techniques (some of which we’ll explore below), including “recomputation”, cryptographic checks and asking an AI agent.
How do find out the above while protecting the prover’s privacy, and the validity of the verifier claim? We may be worried about sensitive IP leaking, or the prover tampering with the verification system.
For a more comprehensive treatment, we recommend this reading list, but, in the meantime, we’ll speak about the areas we’re most excited about:
Hardware-enabled mechanisms
The first way you might think of verifying a claim is to put a physical inspector in the datacenter (or have cryptographically-signed camera feeds). If you can’t take the operator’s word to collect data, you’ll have to install devices that monitor the compute itself.
Saving that, though, verification mechanisms can be inserted at different depths in the hardware stack. We can sort these on a spectrum from “on-chip” to “off-chip.” Quick intuition pump: the closer you are to the silicon, the more secure you are against the operator, but this comes at greater implementational cost and complexity. Starting from the lowest-down:
Silicon mechanisms live inside the GPU package (“die”): either built into the chip during manufacturing, or as a small chip beside it.
These include “secure enclaves”, which are sealed-off portions of the chip which have additional security, and purpose-built governance circuits.
Others, like this proposal for a “guaranteeable memory chiplet” watches the traffic between the GPU and its memory, and can be added without changing the GPU itself.
All these mechanisms aim to put a “trust root” inside the chip, so that the chip’s own account of itself can be trusted by a verifier. If they are not “tamper-proof” (i.e. preventing tampering outright), they aim to be at least tamper-evident, so that the verifier can tell if they’ve been altered.
Unfortunately, silicon mechanisms are the highest-cost option: you need buy-in from the designers of the chips, and the factories that manufacture them (the “fabricators” or “fabs”). And security has proven hard to actually deliver: trusted execution environments (“TEEs”, a form of secure enclave) have a long history of side-channel attacks, which can read secrets from that portion of the chip without touching it — so tamper-evidence mechanisms have nothing to catch.
Board mechanisms add extra chips to the circuit board that the GPU sits on, rather than the GPU itself. This means coordinating with board designers, rather than fabs, a lower lift. An example of this are “FlexHEGs”, a proposal for wrapping an existing chip inside a sealed enclosure with a small auditor chip inside, and an alert if that box is opened.
Firmware mechanisms don’t involve changing any hardware at all, and instead have verification as updates to the chip’s built-in operating instructions. An update might make the chip what it’s running (attestation), keep usage records or turn certain features off. Because the industry-leading GPUs already refuse to run software that isn’t cryptographically signed, some mechanisms, like offline licensing, could potentially reach chips already inside datacenters. These mechanisms are fastest to deploy, but also the easiest to circumvent: a determined operator can undo (“rollback”) a firmware update, for example.
Facility mechanisms piggyback on the operator’s existing datacenter infrastructure. Probably the most popular proposal is to attach “network taps” -- cheap bits of hardware clamped onto the optical cables connecting chips -- to record all the traffic in a cluster, which can be checked mathematically or rerun (both explained below). Others include power meters, cameras, and seals. These are cheaply retrofittable, but the signal (how much information the verifier is getting) is much coarser.
Recomputation
A network tap, for example, on its own gives you a bunch of data (thousands of terabytes) on the workloads being run on the associated chips.4 One proposal for analyzing that data is recomputation: take the traffic records from a datacenter (collected by taps, or by alternative routes like taking custody of logs), then, in a separate “recomputation cluster,” reproduce the prover’s claimed computation and compare the outputs. Here, a mismatch could be evidence of an undeclared workload.
There are a few difficulties for recomputation regimes:
Nondeterminism: surprisingly, running the same AI computation twice often doesn’t give you exactly the same answer. This quirk, which is an artifact of how today’s chips juggle many calculations at once, is actually a relatively solvable problem with clever engineering: Deepseek recently made training reproducible for its V4 model, and Thinking Machines independently fixed it for inference.
Different hardware gives different answers (h/t Luke Marks): the recomputation cluster will not be built from the datacenter’s hardware. So even if you make your own machines perfectly repeatable, they still won’t agree with someone else’s! Different chips round their arithmetic’s tiny errors differently, and the disagreement grows when models are run with numbers stored at very low precision (in the lingo, heavy “quantization”).
At the lowest common precision, NVIDIA GPUs just one generation apart agree on a model’s next word only ~85% of the time, and we’d guess a datacenter of Huawei Ascend chips checked by an NVIDIA cluster falls to around 70%.
We can fix this by studying the target chip closely enough to imitate its arithmetic exactly, down to every rounding error, as Hawkeye recently did for one GPU generation -- but the imitation runs many times slower than the real thing.
Physical security: the recomputation cluster sees the records of someone’s frontier computations, so it has to be defended like the secrets themselves -- at SL5, the top tier of RAND’s security levels, meant to resist nation-state attackers. That is likely extremely hard, and this alone could be enough reason not to use recomputation.
Cryptographic mechanisms
Cryptography is another way to do the “structural” check (looking at the shape of a computation without looking at its content) earlier, establishing a computation that was run was the one declared.
The basic tool here is the “cryptographic commitment” (the mathematical equivalent of sealing your answer into an envelope). The prover computes a hash: a short, 256-bit fingerprint that uniquely identifies a pile of its training data, code, and model checkpoints (versions of the model mid-run), and hands the fingerprint over. Later it can be forced to open the envelope, reveal the originals, and show that they match both the fingerprint and the workload it declared (and it cannot swap in different files, which would change the fingerprint).
More ambitiously, we could try for “zero-knowledge proofs”: proving a claim is true without revealing anything else. The familiar version is proving you’re over 18 to a website without handing over your birthday. In principle, you could prove that a training run used at most a certain amount of compute, or that a model was really trained on the data the developer claims.5
These tools map onto several of the properties from earlier. Verifiable inference -- a proof that a given model really produced a given output (see zkLLM) -- could enable model identity. Proof-of-training -- a proof that a model really came from the training run described -- could enable compute thresholds and workload provenance. And proofs can serve data provenance too, including the harder direction: proving what wasn't in the training set.
The bottleneck to deploying any of this is efficiency: all of these techniques are orders of magnitude too slow to run at frontier scale. We are, however, convinced that large efficiency improvements are reachable.6 We need far more talent and effort directed at speeding them up.
AI agents
It turns out, of course, that even we weren’t AI-pilled enough. Humans are annoying because they remember things. An agent can read everything, report only a violation, and then be deleted. Beyond being memory-wipable, AI agents can process far more information than any human (the records of a large datacenter often exceed thousands of terabytes), can be trained in an open-source and neutral way, and get better as AI capabilities do.
Consider, for instance, the following potentially powerful technique proposed to us by Luke Marks:
Verifier trains some agent that it trusts to audit the prover’s data
Prover commits to the data the verifier will audit
Verifier sends the model weights and inference code for its agent
Prover sends the agent’s output back to the verifier + a ZKP showing that the output was correctly computed
This is strong because (2) prevents the prover from adversarially choosing the inputs to the agent, and (4) means that the verifier never has to see the prover’s data.
Difficulties: agents are vulnerable to sophisticated adversaries -- no one has yet solved jailbreaking, prompt injections, secret loyalties, or backdoors.
Privacy-preserving agent auditing, we’ll note, is a ripe space for startups, because trusted agents could be used for basically any type of data: applications abound.
Intelligence
Not a technical approach per se, but if none of the more sophisticated approaches work or are politically viable, the “dumbest” solution we can pull off the shelf is espionage. Fortunately, datacenters leak so much evidence: they are enormous, hot, power-hungry, and built from chips that are in theory traceable (see AI 2040’s Covert AI supplement, and the difficulty of hiding a datacenter).
Epoch AI and SemiAnalysis are doing early versions of this with public information, but the field as a whole needs to get much better at intelligence analysis, such as of public data (“OSINT”), signals intelligence (“SIGINT”) and satellite imagery (“GEOINT”), as the backstop, both inside and outside the state intelligence agencies.
Our project list
Funders, engineers, researchers, policy folks, fellow weekend warriors, here’s our best guess at what to start on now, in roughly the order of our excitement. We’ve sorted it into [eng] [funder] [policy] [ops].
[Everyone] Inference-vs.-Training Real Verified Datacenter: The most promising intermediate step towards international trustless verification is a feasibility demo of inference verification that can be deployed at scale, in commercial datacenters. This is because we expect inference verification to be the most important and slowdown-relevant property to verify. Concretely, then:
[eng] Technical work. If you’re an engineering / hacker / researcher / security expert, hack on a bunch of approaches for inference verification and try to deploy the most promising ones at scale. The field has converged on recomputation and network taps, but it’s not clear to us that they’re indeed the right bets to make – we need new ideas and the grit to scale.
[policy] [ops] Lab-facing policy. Get a lab to do it! This will look like identifying a reasonable proposal and doing a great deal of lobbying of senior lab folks to sign-off on it for the goodness of their hearts, or making it part of some SL-5 standard that is incorporated into an RSP. There might be a few sympathetic neolabs? [Eng] One particularly exciting development is the emergence of specialized verification teams within labs, which can often come about with a single empowered internal champion. Consider! (Also, we’re looking at you, lab employee organizers -- this could be a very reasonable demand)
Nicely, this kind of verification aligns with lab incentives: though the first goal of verification is to provide attestation to third-parties that a certain rule is being followed, it will likely have a secondary benefit of security. A friend recently claimed to us that 5% of a major lab’s datacenter compute could be diverted by an outside actor (a rogue AI, an adversary) and the monitoring systems do not currently exist to catch that. As we get better AI schemers, and more sophisticated attackers, verification as a backstop for determining compute usage and enforcing security is useful for the developer themselves, to be robust to compute subversion attempts, workload tampering, rogue deployments, and backdoors.
[policy] Middle-power governments. If you’re from one, get your government involved in verification buy-in and build-out. We’re excited about the international verification efforts that the Singaporeans are making here. “Middle powers” – particularly neutral ones like Singapore and Switzerland – have, we think, important roles to play in international governance and cooperation.
[diplomacy] U.S./China government tracks. We need to build policy and diplomatic interest in verification. If you’re in the U.S. and involved in government, think tanks, or a credible technical expert yourself -- socialize slowdown in government, and convince DARPA and natsec to invest. If you’re interested in China-facing verification efforts, talk to Jasmine.
[Ops] [Funders] Run a big verification prize (more prizes, please). We scoped out one possible design, focused on telemetry analysis, at https://verifierchallenge.org/, where we also spoke a little bit about some of the experiments we ran with Claude on designing a good AI agent inspector. We have many notes on how to do this well, which we’re happy to share more in detail, but here’s our best guess:
A few open problems we think are good fits for prizes: cryptographic acceleration (like ZPRIZE), telemetry analysis.
The latter could be red-team blue-team style, as the NSA sometimes does at DEFCON for cybersecurity, to create both stronger defenses and check them against a smart adversary.
Prizes are good for pre-paradigmatic problems where you can define the problem well, but there exists no strong commercial incentive to solve them. There are a few notable successes (self-driving, scroll-reading, arguably computer vision, and a variety of wins from XPRIZE) and some current prizes we’re excited by (the “Boom Prize,” Cadenza Labs’ lie detection competition, Gray Swan’s brilliant redteaming arenas).
The ideal version of this starts with a cascading prize structure, with entry-level open problems (“upload a real datacenter log + design an adversarially robust classifier to detect property X,” “prove Llama-32B-scale inference at 3x speedup from current SOTA”) and then increasing in scope / difficulty.
It can be challenging to design contests in a way that actually elicits good solutions and innovation and makes the effort of running a competition worthwhile, instead of just ‘solving’ the problem yourself (e.g. with a focused research organization).
The most successful competitions we’ve seen invest effort into developing one side (either the redteam/blueteam), and invite the public to tackle the other (e.g.: ImageNet provided a dataset, Gray Swan provides challenges).
[Everyone] The regular fieldbuilding playbook:
[ops] [eng] Introductory course for hardware security. “Build a Bluedot course” is an oft-recommended solution to nearly everything, but that’s because it works. We’d love for more accessible intros to the hardware behind AGI, and if it had a section on verification, even better. Lump it into an “AI security” course.
[ops] [policy] Introductory diplomacy course. Make it a Plan A reading group that goes lightly into technical verification proposals, dark-compute tracking, and precedents from international agreements. You may have to run country-specific versions.
[eng] “Top-of-the funnel” technical upskilling. Teach verification-relevant hardware and cryptographic knowledge. We think this should include both “top-of-the-funnel” opportunities, such as open-sourced curricula, and more involved ones, like in-person R&D programs, directed towards e.g. mid-career security experts and datacenter engineers (this field seriously needs more technical talent!). For inspiration, see Cambridge AI Safety Hub’s Hardware Assurance Program and AI Security Bootcamp.
[eng] Hackathons. Hackathons, hackathons, hackathons: bring security-minded engineers together to hack at proposals, and reach them before the next CUDA shop swoops them up. Alongside ML conferences, the most recruiter-y of events. Get the applied cryptography people involved too.
[ops] Mixers. Governance mixers in DC. Hardware mixers in SF. Verifiable inference dinners alongside cryptology conferences. Rich people mixers in NYC. You know the deal.
[Policy] Finding dark compute. Build think-tank capacity to find compute, and -- if you have ins to state intelligence agencies -- nudge them to lock in.
There’s only so much outsiders can do: agencies are structurally resistant to third parties telling them what to do, and they have the most resources to do this well.
If you’re one of the NSA’s forward-deployed engineers detailed from Anthropic, a cleared RANDite, or personal friends with the Director of National Intelligence: tell them to track global total compute. It’s quite the interagency endeavor.
NGOs will never do intelligence as well as state agencies, but a future IAEA-for-AI will have to rely on NGO capacity, because it won’t have that sweet, sweet national intelligence.
[Eng] [Funder] Cryptography R&D: For the more mathematically inclined among you (so: not us), consider working on making the cryptographic for confidential AI verification radically more efficient. Efficient verifiable inference (for model provenance, evaluation gating, verifiable auditing) may not be far off (see zkML, SNARKs as mentioned above); it looks bottlenecked mostly by talent and money. Proof-of-training is further out.
For the funders: figure out how to provide funding to incentivize this work.[Funders] Fund open-access hardware buildout. Very few people in the field -- even the experts! -- have been able to do hands-on experimentation. Building on Amodo and Lucid Computing, which have set up open research clusters for bare-metal accelerator access, consider funding additional open-access cluster capacity, particularly in neutral jurisdictions (e.g. Singapore).
To be clear, we don’t think hardware access is the biggest current bottleneck in verification (rather, talent and ideas). However, operational capacity seems upstream and important for a flourishing R&D ecosystem: we want to help people develop much better hardware intuitions and incentivize credible mechanism ideas. Also, cluster setup itself takes O(months) -- so we should start sooner.
[Policy] [Eng] Proposed standardized protocols. Start easy with a standardized logging protocol. You’d want it to be flexible enough that it can account for the various different kinds of inference, experiment and training computation, but also not have the design leak any valuable IP. Maybe also standards for when to log for transparency’s sake (beyond just verification, also for concentration-of-power-or-avoiding-lab-coup reasons.) In general, doing things that replicate the SL-5’s taskforce focus on standards (in their case, ISO-compatible) but for verification: not the most sexy work, but incredibly important.
[Policy] Macrostrategize. We need to answer core questions about how verification regimes would work, what they look like, how long they last, what we use bought time for, etc. Like everything else, verification needs to know its theory of victory.
[Eng] [Policy] Open questions paper. Yes, one of our research questions is to list the research questions. We have a sneaking suspicion that a field becomes real via the open-questions paper that hyperstitions it into existence (case in point: technical AI governance – its verification section gives a good overview but needs far more specificity and technical grounding).
Now, a field
Remember the days before “compute governance”? It took Lennart Heim’s patient evangelism for many years before; now, of course, it’s so obvious that compute (the one input to AI that you can count) would be the substrate governing AI. But the rest of us spent those years fumbling -- including one of your authors (!) -- talking about “third-party” verification in the abstract.
The field is small. Pulling these numbers a little bit from thin air: maybe 60 people in the world think about compute verification full-time (with many more technical folks than policy and strategy ones). They could all fit in a single floor of a Berkeley office. They aren’t that diverse, either: mostly American nonprofits. The names worth knowing (apologies for the incoming alphabet soup): Oxford’s HAIG Lab, RAND, FLI, MIRI, and IAPS on research; for funding and high-level strategy, the OpenAI Foundation, IFP, Coefficient Giving, Longview, and Astralis.
Verification today is where compute governance was in 2021: obviously important in retrospect, but sorely orphaned in overall field strategy, R&D implementation, diplomatic groundwork, and actual funding deployment. The verification field that has to exist must develop expertise in many disparate fields, from cryptography, hardware, and agent security to standards, policy, and diplomacy. And because the groundwork and technical/diplomatic infrastructure needed for verification may take years in the least to develop, we must start now.
Ronald Reagan is quoted ad-nauseum, speaking amid disarmament talks with the U.S.S.R, saying “trust, but verify.” In this race to superintelligence, as the world gets crazier, we cannot rely on blind trust between the U.S. and China. Verification, first, then.
Huge thanks to Luke Marks, Atharva Nihalani, Daniel Reuter, Will Hodgkins, Brendan Halstead, Tom Milton, Sudarsh Kunnavakkam, Catherine Brewer, and Emlyn Graham for their feedback on this post.
Thanks to many experts and friends for conversations -- too many to name all <3 -- but in particular Sabrina Shih, Daniel Reuter, Mauricio Baker, Wayne Tee, Aaron Scher, Holden Mui, Elsie Jang, Abbey Chaver, Dewi Erwan. (Being listed here, of course, does not mean they endorse the claims in the post.)
To our great chagrin, Google docs’ spellcheck still does not recognize “compute” as a valid term-of-art, and angrily insists with its little blue squiggly line that we must mean “computer.” (but Substack’s does!)
There is, of course, the opposite property -- showing that a chip is not in a restricted jurisdiction. More reading about location verification.
Conceptually, this is possible because, though inference and training often use the same chips, they have different workload characteristics in the current paradigm: training requires many GPUs making and syncing updates to the same set of weights (a highly-communicative, parallel operation) while inference tends to happen over a small number of chips, on a single query or batch of queries at a time.
There are a few asterisks to this explanation, though. Training, here, technically refers to pre-training: this is most frustrated by the fact that RL relies on long “rollouts,” which look similar to where gains in inference come from. As the science of AI training continues to develop, and, e.g. people develop approaches like some forms of continual learning (eg. live weight updates during a user session, perhaps that persist across sessions / sets of users), this line will blur. On the other hand, increasing progress in “inference-only” chips (because there are, indeed, optimizations you can make if you know the particular kind of matrix multiplication that is going to happen many times) means that inference-training distinctions may become more obvious just from the hardware itself.
In more detail – recomputation “makes the verifier pay” (it needs to create a second recomputation cluster and secure it) and shows the verifier much more. Cryptographic proofs make the prover pay some overhead, and shows the verifier nothing but the claim.
Tools under active research include SNARKS, proof-of-learning, and proof-of-training-data. “Fully homomorphic encryption” (FHE) is a related family that lets a verifier run compliance checks directly on encrypted data without ever seeing it. ZKPs buy integrity and privacy together, while FHE buys privacy alone: it doesn’t guarantee that the prover’s inputs were honest in the first place.
zkML, for instance, is small and under-optimized, with many low-hanging efficiency gains. In addition, ZKP systems are far cheaper over integer matmuls than over floating-point ones, so we think there should be more research on engineering better proof systems and integer-friendly kernels / hardware. This is so far underexplored.
incredible work- thank you both for writing this up
yall COOKED