Arcturan Labs
Engineering deep-dives on distributed systems, architecture decisions, and the craft of building reliable software at scale.
The Class You Didn't Train For
Classifiers are built to pick a winner from a fixed list. Most on-device recognition problems are the other kind — where the honest answer is usually 'nobody I know' — and the gap between those two framings is where beginners lose months.
The Latency You Cannot Optimize Away
Streaming inference has two kinds of delay: the time your model spends computing, and the time it spends waiting for input it has not received yet. Only one of them responds to a faster chip.
Every Rung on the Ladder Costs Something
Compression is not a switch you flip at the end. It is a descent through precision, structure, and capacity — and the discipline is measuring what each step takes from you before you take the next one.
Joules Per Correct Answer
An always-on system is defined by what it does when nothing is happening — which is almost always. The cascade is how you spend nearly nothing on the empty hours and everything on the moments that matter.
The Accelerator That Quietly Says No
A phone is four or five processors wearing one name. The neural engine is the fastest and the fussiest, and when it refuses part of your model it does not raise an error — it hands the work back to the CPU and lets you believe you are accelerated.
When Brute Force Is the Right Index
Approximate nearest-neighbour indexes exist to make search over billions of vectors tractable. On a device holding a few thousand, they cost memory, accuracy, and the ability to delete — to solve a problem you do not have.
A Score Is Not a Probability
Combining several weak signals into one confident answer requires that the signals be measured in the same currency. Model outputs almost never are, and calibration is the step most fusion designs skip on the way to a number that means nothing.
Learning Without Training
A device cannot retrain a neural network, and does not need to. Separating the model that learns representations from the memory that stores identities is what lets a system acquire someone new in thirty seconds — and what makes the quality of those thirty seconds decisive.
The Benchmark That Runs for an Hour
A thirty-second measurement captures a device at its best and never again. Everything that determines whether a feature can ship — throttling, duty cycle, the real cost of staying awake — only appears once the benchmark outlasts the thermal mass of the chip.
On-Device Is a Claim, Not an Architecture
Running inference locally is the beginning of a privacy guarantee, not the whole of one. Crash reports, backups, analytics, and the quiet assumption that an embedding is anonymous are where most of the data actually leaves.
A PID Is Not a Health Check
The most dangerous production failure isn't the crash — it's the process that passes every check you thought to run while doing exactly nothing. Liveness is a property of the whole system, not of any process in it.
A Full Restaurant With a Cold Kitchen
A background system can report itself at capacity while the expensive resource it exists to protect sits idle. The usual cause is the same one twice over: the limit counts the wrong thing, and work that is only waiting is allowed to hold a seat.
Isolation Is a Configuration Property, Not a Hosting Tier
The day-one reflex to buy a second cloud account for staging conflates two different purchases. Environment isolation is a property of the boundary you draw — and you can draw it on hardware you already own.
A Policy That Isn't a Gate Is a Suggestion
My agent fleet had a simple model policy: plan on the expensive model, build on the cheap one. An audit showed it running exactly inverted — and nothing had errored, because nothing was checking. What it took to make the policy real.
Speculative Execution for Generative AI UX
GPU cold starts make the first image the slowest one — right when the user is judging your product. Instead of paying for an always-warm GPU, start the job before they click Generate, and treat the money problem with a payments-industry trick.
Parallel Agents, Serial Reviews
Running more AI coding agents at once looks like turning a throughput dial. It's really the oldest problem in software management wearing new clothes — and past a handful of agents, the dial starts running backward.
Agents in CI Fail Silently
Wiring an autonomous AI reviewer into a pull-request workflow sounds like a config exercise. It's really a lesson in what breaks when you move an interactive tool into a headless one — and why almost none of it fails loudly.
Seeds Fix Where, Not What
A fixed seed makes image generation reproducible — but not editable. Here's why keeping the seed and tweaking the prompt works for some changes and scrambles the whole image for others.
Diffusion Models Aren't LLMs
I tried LLM prompt-engineering tricks on a diffusion model. Most of them backfired. Here are the counter-intuitive lessons that finally got the model to render what I asked for.
Why Boring Technology Usually Wins
Every shiny new tool has a cost that doesn't show up on the benchmark page. Here's how I think about technology choices at scale.