I just wanted to build accelerators

It started with a CPU

I knew I liked chips, but I had no idea where to begin. A senior at SRA-VJTI pointed me towards Synapse32. The idea was to build a soft-core RISC-V CPU that could eventually be useful in embedded projects. I wanted to make my own RV32I core and understand what actually went into it.

I was studying electrical engineering at VJTI, where much of the coursework focused on large-scale and power systems. I wanted to understand chips, so I had to find that education elsewhere. SRA became the place where I could ask questions, get stuck, and learn from people building things around me. SRA’s board progression, board builds and PCB photos are worth a look too.

The first time you make a pipeline work, the diagrams stop being abstract. Instructions move through stages; hazards have to be handled; everything has to agree about what happens on a clock edge. Once I understood a little about one RISC-V core, I had more questions:

• The instruction set: What does the processor need to be able to do, and which instructions make that practical?
• The pipeline: Where should the work be split into stages, and how fast can each stage run?
• The cost of splitting: What happens to latency, hazards, and control when I add more stages?

Pipelining gave me a useful way to think about trade-offs. A longer combinational path can limit the clock frequency. Splitting the work into stages can shorten the longest path between registers, letting the clock run faster. But registers have overhead, and more stages can mean more latency and more costly hazards.

In this example, breaking a long path into stages helps at first. Beyond a point, the costs of extra stages and workload effects outweigh the faster clock, so latency rises again. The turning point depends on the design and workload; a real pipeline may behave differently. The question is where to split the work for the system I want to build.

Around the same time, I followed Ben Eater’s 8-bit computer builds, learned how transistors can be used to build digital circuits, and went through Onur Mutlu’s lectures. I wanted to keep going down the stack and understand what the pieces of a computer were actually doing.

Memory, and the application on the other side

Learning about processors made me realise how much of the machine is memory. I played around with memory circuits in LTspice and tried to understand what happens when the thing you need isn’t where the computation is happening. That question keeps coming back, whether the system is a tiny FPGA or a rack full of machines.

I also tried to implement a transformer on an FPGA. At the time, I understood neither the large FPGA I was trying to use nor the transformer well enough. It was difficult, and I didn’t get the complete system I had imagined. But it made the gap obvious: learning to build a processor is only one part of running an application.

The purple banana running through my HDMI driver.

A way of thinking about this that I like is to look at the application and the compute underneath it together. What does the workload actually need? What can the hardware provide? How much of either side can you change? People generally call that hardware–software co-design. For me it became a reason to understand both sides instead of treating the chip as a black box.

Spikes and Prosperity

My internship at IIT Madras introduced me to a different approach: spiking neural networks. Their event-driven structure takes inspiration from biological neurons. That was exciting to me because it brought the question back to where it started, even if an artificial spiking neuron is only a simplified model of the real thing. Is this the right architecture for intelligence? I don’t know. I wanted to understand it by building it.

I worked on a hardware implementation of Prosperity, an SNN accelerator architecture from a research paper. Getting from a paper and simulator to hardware meant working through timing, state, data movement, and the uncomfortable details that an architectural diagram can leave out. I had a lot of fun with it. I also found out later that some of my attempted extensions were based on a wrong understanding of the design. Going back and figuring out why was part of the work too.

I got outputs from the design. Here’s the demo, and I want to revisit what worked and what didn’t.

Making the whole system run

I kept coming back to matrix multiplication. My first tiny TPU was a small systolic array on an FPGA, with a MicroBlaze controller and BRAM tiling to feed it data. Getting it to run MNIST was exciting: I could follow the computation from the program down to the hardware I had built.

Once the multiply worked, I wanted to run more than a single kernel. That meant building the system around it: moving inputs, getting results back, running the rest of the program, and giving the hardware a way to receive work.

Tiny3TPU grew out of that. On a KC705, it brought together a VexRiscv CPU, two 4×4 int8 systolic cores, on-chip RAM, firmware, and a compiler path from JAX through StableHLO to the board.

The interesting part was being able to send a program to the board and get an answer back. I tried a digit classifier, then mesh transforms and cloth physics. For the mesh demo, the board transformed the vertices and my laptop drew the coordinates it received.

The cloth made the limits harder to miss. In one experiment, each frame took about 4.8 seconds, mostly in software floating-point physics. The GIF above is sped up. An earlier capture measured a different, smaller physics step.

I also worked on a small matrix accelerator for Tiny Tapeout, where area and interfaces made the constraints impossible to ignore.

The most useful part was being able to change something and measure it. A faster arithmetic unit doesn’t automatically make the whole program fast. Data still has to move; the CPU and compiler still have work to do. I wrote more about those experiments in The array was the easy part.

From the board to the rack

As I read more, I realised that people were approaching the same broad problem from different directions. Some change the model architecture. Some build specialised hardware. Others change how memory and compute are connected. I started paying attention to work from companies like Cerebras and Etched as well as different approaches to compute-in-memory. They make different bets, and I want to understand where each one helps.

CXL was interesting because it gave me a way to think about memory beyond a single board, at the scale of servers and racks. I worked on CXLMemSim, a way to study CXL-attached memory behaviour. There are more papers in progress, and I’ll share them when I can.

At Zett AI, I have also worked on KimiBox, an on-device language-model system. The questions feel familiar at a different scale: where do the model’s weights live, what needs to move, and how do you make the whole system fast on hardware people can actually afford? We experimented with ideas around memory and prefetching. I remember a run around 20 tokens per second, but I need to check the configuration and current result before treating that number as a benchmark.

What should we measure?

A question I want to come back to is: what are all these architectures trying to maximise?

I keep coming back to a few ways of asking it:

• Tokens per second: How quickly can the system produce output? I also need to distinguish the speed one user sees from total throughput across users.
• Tokens per joule: How much output do I get for the energy spent?
• Tokens per dollar: What does it cost to produce that output, including the costs I choose to count?

None is a complete measure. I also need to know the model, quality, workload, batch size, context length, and what’s included in the cost. A faster stream of worse answers isn’t automatically a better system.

I want to compare model and hardware architectures using those kinds of measures, based on the numbers their authors actually report. I haven’t done that analysis here yet. The interesting part will be figuring out which comparisons are fair and where the missing numbers are.

The bitter lesson is also in the back of my mind. As I understand Sutton’s argument, general methods that can keep using more computation, especially learning and search, have often outperformed approaches built around our hand-coded ideas about a problem. That doesn’t prove spiking networks can’t work, or that specialised chips are the wrong approach. It does make me ask whether a clever idea can keep improving when the workload, the data, and the system scale up.

Cerebras is interesting to me in that context: it makes a very specific hardware bet to support large-scale computation. The question I want to investigate is how such bets affect what can scale and at what cost. I have similar questions about memristors and compute-in-memory: can they deliver their promise at the scale of a useful system, including the memory, interfaces, tooling, and manufacturing around the devices? I don’t know yet. Someone who makes that work at scale could change a lot.

The tools have to get better too

I think language models and agents could eventually do much more of the work of designing accelerators. I can imagine giving a system a direction and letting it propose designs, run experiments, and improve them. That’s a long-term ambition, not something I think we’ve already solved. Verification, physical constraints, and knowing whether the result is actually useful still matter.

One near-term problem is the speed of the loop. Hardware design tools can be slow and awkward to iterate with, and an agent can’t learn much from an experiment it can’t run or verify. That is part of why my work on CIRCT and counterexample generation matters to me: better tools can make the next design easier to understand and debug.

I don’t expect to answer all of this soon. But these are the kinds of questions I can see myself working on for the next 20 or 30 years. I want to keep building things, seeing where they break, and getting a little closer to understanding how computation becomes something that looks like thought.

Looking back, none of this would have happened without the people who supported me, answered my questions, and believed I could figure things out. I’ve been really lucky to learn from mentors and build alongside friends who made all of this feel possible. Thank you.