All Writings

everything i could verify about nvidia hopper/blackwell and amd mi300x through mi355x for llm inference. silicon, number formats, interconnect, how vllm/sglang/tensorrt-llm...

A response to Mario Souto's clay jar essay: energy orchestration only works if inference systems expose workload class, latency tolerance, and value per accepted result.

GPU utilization can be 78% while 30% of requests fail SLO constraints. The goodput frontier test replaces single-number benchmarks with decision-grade surfaces that measure...

The gap between what a model is supposed to do and what it actually does in production is enormous. Forward-deployed engineering exists to close it. Here's why i left Amazon...

Not all inference is the same. Per-workload LCPR exposes the cross-subsidy that blended averages hide, with cost models for conversational, agentic, RAG, extraction, voice, and...

Open-source calculator for loaded cost per result. Three worked examples, cache break-even analysis, and KV memory sizing.

A repeatable diagnostic for going from raw trace events to loaded cost per accepted result. Twelve requests, four data sources, five cost mechanisms, and the reconciliation protocol.

The most common mistake in inference economics is dividing by the wrong number. LCPR (loaded cost per accepted result) reveals a 12x gap between naive token cost and actual...

TCO frameworks, vendor evaluation, and architecture patterns for teams adopting open-model inference. Includes the LCPR calculator, migration gates, and a staged playbook from...

KV cache research and interpretability research are measuring the same prefill->decode shift from different angles. This post argues for a shared consistency-model framing and...

Continuous batching is algorithmically activation-safe under per-token normalization and masked attention, but hardware nondeterminism, prefix caching, and data-methodology...

Activation-capture pipelines can look healthy in steady state while silently dropping the most informative samples under transient pressure; this piece maps the main failure...

Prefill-trained interpretability dictionaries are routinely deployed in decode-time regimes; this piece argues for a concrete measurement standard for prefill->decode drift...

Current-state synthesis of the persona-circuits project: robust steering and partial concentration support, but weaker-than-expected distinctness, necessity, and sufficiency...

Branch report on Generative Latent Priors (GLP) for activation repair in persona steering: public-checkpoint transfer failed, matched checkpoints were more stable but still...

On trusting deeply in the waiting period and being intentional about reaching out to people.

First-principles thinking matters, but reality and rapid experimentation are the only reliable validators for complex systems and ambitious work.

Why creating in public is less about attention and more about agency, proof of work, and meaning.

Responsibility is often its own reward, whether in crisis operations or self-funded interpretability research.

Goals are useful for alignment. Attachment to timeline and route is what creates stress, rigidity, and worse decisions.

AI can match your aesthetic judgment, curate better than your friends, and predict trends before they happen. So what's the actual human edge?

GPU selection, inference framework choice, and the upstream decisions that determine what optimizations are even possible in production vLLM deployments.

Naval Ravikant says desire is a contract you make with yourself to be unhappy until you have the thing. Here's what that looks like in practice.

Something shifted recently. Not the usual swing between secular ambition and spiritual grounding. This time it feels like both are pointing at the same thing.

On writing, presence, and reclaiming the moment.

You don't know your research focus in advance. You try things and notice what pulls you back.

The lack of creative output isn't a model problem. It's a skill issue. Yours, not the model's.

The orgs best positioned for AI agents aren't the ones with the best tooling. They're the ones with the strongest human writing cultures.

Everyone's watching humanoid robots. But if you think something is revolutionary, variants probably exist. The robotics community is vast. Here's what i'm learning as a...

Rumi on moving within, fear, and finding beauty in the doing.

Org theory applies to agent swarms. I've lived three different org structures—here's how they map to multi-agent patterns.

CAP theorem gives you a lens for distributed systems. There's a missing dimension for infrastructure decisions: whether your system is latency-bound or throughput-bound. Most...

Three weeks into a new role, agent orchestration finally clicked. Not just using AI tools—actually managing complexity with them.

Real production failures from running Ray at scale: lost training runs, enterprise network disasters, and cascade outages the documentation never warns you about.

I built an automated Twitter feed scraper that triages 100 tweets daily, analyzes community sentiment with Claude, and delivers a 5-minute digest to Slack.

I chose Telegram over Slack for my AI assistant setup, then realized I was making decisions based on old baggage instead of current evidence.

Memory fragmentation, throughput cliffs, and quantization accuracy issues that only show up in production—lessons from running vLLM at scale for conversational AI.

How I eliminated context bleeding in my AI assistant using openclaw's multi-agent features and Telegram forum topics.

Patterns and tradeoffs from building RAG infrastructure in regulated environments.