What Does NVIDIA's Vera CPU Actually Do?

88 custom cores with one job, keeping a rack of GPUs busy.

🧭 Part 9 of the ⚡ Hardware & Inference course

Vera is NVIDIA’s first data-center CPU built on cores it designed itself, 88 custom Olympus Arm cores. Its job is to keep the Rubin GPUs beside it busy. It runs the tool calls, code, and data pipelines a GPU handles badly, so the GPUs stay on the math.

TL;DR

• What it is: the host CPU of NVIDIA’s Vera Rubin systems and the successor to Grace, the first NVIDIA designed on its own cores.

• The problem it solves: an AI agent burns most of its time on CPU work, and a slow CPU leaves the costly GPUs idle between steps. Vera keeps them busy.

• Who runs it: Meta, Oracle, ByteDance, and CoreWeave, for heavy agentic and inference work.

• The secret sauce: a 1.8 TB/s link between Vera and its GPUs, double Grace’s, so the CPU hands work over faster than any standard server can.

• Market position: NVIDIA’s first real entry into the server-CPU market, sold only inside its own racks, never as a chip you buy on its own.

Before Vera, What the Host CPU Actually Does

Your agent just paused to call a tool, run a database query, and execute a snippet of its own code in a sandbox. None of that runs on the GPU. It runs on the CPU, and until the CPU finishes, your most expensive chips sit idle and wait. Multiply that across thousands of agent steps a minute, and you are paying the highest compute rates in the building for chips doing nothing.

A GPU never runs on its own. A host CPU loads the model, feeds it each batch of data, decides what to run next, and hands the work across. Picture a busy kitchen: the GPU is the line of cooks repeating one step thousands of times over, and the CPU is the head chef who plans the orders and keeps every station supplied, the split we covered in Why Does AI Need a GPU?.

For years that made the CPU an afterthought. The GPU did the expensive math, so the CPU only had to stay out of the way. A trained model runs the same calculation over and over on the GPU, and between requests there is little for the CPU to do.

Agentic AI broke that balance. An agent does not run one big matrix multiply and stop. It loops: think, call a tool, read the result, run some code, think again. We covered that loop in What is an AI Agent?. The model's turn on the GPU is brief; everything else runs on the CPU. As one NVIDIA executive put it, “Agents don’t operate on GPUs alone. They need CPUs... for tool calling, SQL queries and the compilation of code.¹”

Here is where each step of that loop runs:

NVIDIA is not the only one saying this. AMD sells the very CPUs Vera competes with, and it made the same case in March 2026. The CPU, it wrote, does “the critical system work that keeps accelerators productive.” Two firms with competing chips to sell rarely agree on where the bottleneck sits².

So the category is clear. Every second the GPU waits on the CPU, you burn the most expensive compute in the building. The fix is a CPU fast enough, and wired tightly enough, that the GPU rarely has to wait. That is the job Vera does.

The Story in Two Chips: Grace, Then Vera

NVIDIA’s first data-center CPU was Grace, shipped in 2023 on 72 of Arm’s Neoverse V2 cores, a strong but off-the-shelf design it licensed rather than built. Grace paired with a Hopper or Blackwell GPU over an NVLink-C2C link at 900 GB/s. In the GB200 superchip, one Grace hosted two Blackwell GPUs³.

Grace did the host job through the pretraining era. NVIDIA had pointed a generic Arm design at AI work, but never shaped one around it. Licensing Arm’s cores meant NVIDIA could not tune what matters most for orchestration: how the cores work, how much fast memory sits next to them, how they juggle many small tasks at once.

Vera is the answer, and it is a big bet. Its Olympus core is the first CPU core NVIDIA has designed from scratch since Denver, the core in its Tegra mobile chips more than a decade ago. Designing your own core costs years and hundreds of engineers, a commitment most companies never make. NVIDIA made it because the host CPU had become the bottleneck on its own flagship product.

What You’re Actually Buying

When you get right to it, Vera is 88 custom Arm cores on one big chip. A wall of fast memory surrounds them, and the fastest CPU-to-GPU link NVIDIA has shipped ties them to the GPUs. Every piece exists to keep the GPU fed.

The cores. NVIDIA tuned the 88 Olympus cores for the stop-start work of running an agent. It built them to run that work fast, about 1.5x the per-core speed of Grace⁴.

The memory. Vera carries up to 1.5 TB of memory, triple what Grace held, and feeds the cores far faster. The extra room lets an agent hold a lot in memory at once, a long document or a large data frame, without spilling to slower storage.

The link to the GPU. This is the part that matters most. Vera talks to its Rubin GPUs over a second-generation NVLink-C2C link at 1.8 TB/s, double Grace’s 900 GB/s and many times faster than the PCIe bus a normal server uses⁵. The link is also coherent: the CPU and GPU share the same memory, and the GPU reads it directly. On a normal server, the CPU copies data across PCIe and waits for it to land, and that round trip is dead time. Vera’s link skips the copy, so the GPU reads the agent’s fresh data in place and the per-step handoff stays quick.

NVIDIA also put all 88 cores on one piece of silicon. AMD and Intel build their big CPUs from several smaller chips wired together. That leaves some cores closer to memory than others, so a few requests take longer than the rest. On Vera, every core sits the same distance from memory, so each one responds at the same speed. For an agent, where the whole loop waits on the slowest step, that even speed is what counts. The tradeoff is money: one large chip costs more to make and limits how many cores fit.

Set the whole chip against Grace and the leap is plain: more cores, far more memory and bandwidth, and double the link to the GPU.

The Two Ways NVIDIA Sells Vera

NVIDIA sells Vera in two forms. The first is the common one: a single Vera feeding two Rubin GPUs, the setup you rent by the hour like any GPU box today. The second is stranger. It is a rack with no GPUs at all, 256 Vera CPUs in one liquid-cooled box, more than 22,500 cores of nothing but CPU.

The CPU-only rack exists because much of an agent’s work never needs a GPU at all. Put that work on a rack of CPUs and the GPU racks do nothing but run the model. It also opens a market NVIDIA could not sell into before. A box of pure CPUs competes head-on with Intel and AMD for ordinary server work, where NVIDIA is the newcomer and the other two already own the customers and the software they run⁶.

The Honest Take

Vera is the right pick when your workload lives next to NVIDIA GPUs and the CPU side is what stalls them: heavy agent orchestration, tool-calling, and data prep feeding a Rubin rack. No other CPU has that coherent 1.8 TB/s link, and it is the whole reason to choose Vera over a generic Arm server chip.

Vera is the wrong pick when your workload never touches an NVIDIA GPU. A fast Arm server CPU is not new. Amazon’s Graviton and Ampere’s chips have offered Arm cores in the cloud for years, and a Graviton instance does the same general computing for less. Strip away the GPU on the other end of that link and Vera’s one real advantage goes with it.

NVIDIA released its own benchmarks on pre-production Vera chips. The tests were everyday server jobs: compiling code, streaming data through memory, running Python and Java, encoding video, database work. Across that whole set, Vera came out about 10% ahead of AMD’s 64-core EPYC 9575F and roughly 55% ahead of Intel’s 128-core Xeon. Read those with care. The 10% edge comes against a chip with 24 fewer cores, so the per-core story is thinner than the headline. NVIDIA also picked the tests itself and withheld the one number that decides the data-center bill: performance per watt.

The elephant in the room is lock-in. You cannot buy a Vera and drop it into your own server the way you can a Xeon or an EPYC. It ships inside NVIDIA’s racks, on NVIDIA’s wiring, in NVIDIA’s prebuilt systems, because the 1.8 TB/s link only exists when both ends are NVIDIA chips. For almost every reader, the practical truth is that you will rent Vera by the hour from a cloud provider, the way you rent H100s today. We walked through that economics in Why is Inference Slow and Expensive?. What matters to you is whether your agent workload keeps those rented GPUs busy, and that is the bill Vera exists to lower.

The Reframe

For a decade, the story of AI hardware was the GPU, and the CPU was plumbing. We told that story ourselves in What Does NVIDIA Actually Do? and H100 vs H200 vs B200. The CPU never did the hard math, and it still does not. Agentic work changed something else: how fast the CPU feeds the GPUs is now the number that decides the bill. Vera is NVIDIA admitting the bottleneck moved, and then turning the fix into a second business: a chip that only pays off next to its own GPUs, plus a standalone rack to sell beside it. That is how a GPU company finally walks into the server-CPU market it never had a seat in.

---

💬 Would you put an agent's tool-calling and sandbox work on a dedicated CPU rack, or keep it next to the GPUs? Tell me how you would split it in the comments.

---

FAQ

What does NVIDIA’s Vera CPU do?
Vera is the host CPU of NVIDIA’s Vera Rubin platform. It handles what a GPU does poorly: loading models, preparing data, running tool calls and sandboxed code, orchestrating each agent step. It then streams that work to the Rubin GPUs over a 1.8 TB/s link, so the GPUs stay busy instead of idle between steps.

How is Vera different from Grace?
Grace used 72 of Arm’s licensed Neoverse V2 cores. Vera uses 88 cores of NVIDIA’s own Olympus design. It roughly triples Grace’s memory ceiling (1.5 TB versus 480 GB), more than doubles memory bandwidth (1.2 TB/s versus 512 GB/s), and doubles the CPU-to-GPU link to 1.8 TB/s.

Why did NVIDIA build its own CPU core?
Because the host CPU had become the bottleneck on its own GPUs in agentic workloads. Licensing Arm’s cores meant NVIDIA could not tune the cache, the cores, and how they handle parallel work for orchestration. Designing the Olympus core in-house let it shape the chip around the exact work that was keeping GPUs idle.

Can I buy a Vera CPU for my own server?
No. Vera ships only inside NVIDIA’s Vera Rubin racks and CPU-only racks, never as a socketed chip for third-party motherboards. Most engineers will use it by renting Vera Rubin instances from cloud providers, the same way they rent GPUs today. The platform entered production after its CES 2026 launch, with availability from partners in the second half of 2026.

🔜 Friday: AI Coding Tools, What Changed in 6 Months, the agents left the editor and the stack you trusted is already behind.

1. Nvidia crams 256 Vera CPUs into a single liquid cooled rack, The Register (March 2026)

2. Agentic AI Brings New Attention to CPUs in the AI Data Center, AMD (March 2026)

3. NVIDIA’s Vera CPU in Detail, ServeTheHome (March 2026)

4. NVIDIA’s Vera CPU in Detail, ServeTheHome (March 2026)

5. Inside the NVIDIA Vera Rubin Platform, NVIDIA (January 2026)

6. Nvidia crams 256 Vera CPUs into a single liquid cooled rack, The Register (March 2026)