# How + Why You Should Build Your Blog with Fumadocs
Fumadocs is a powerful documentation framework built on top of Next.js, but it's also perfect for building modern blogs. Here's how I built this blog.
## Why Fumadocs?
Fumadocs provides everything you need for content-focused sites:
* MDX support with custom components
* Built-in syntax highlighting
* Dark mode support
* Type-safe content with Zod schemas
* Excellent performance out of the box
{/* Email Cutoff */}
## Code Example
Here's a simple example of how clean the code can be:
```js
console.log('Hello World');
```
## Rich Components
Fumadocs comes with beautiful components out of the box:
These components make your content more engaging and easier to navigate.
# The Malleability of Tools: AI Is Eating UI
The concept of malleable tools predates software. Perhaps the original malleable tool was the rock: you could hunt with it, heat it up to emit heat over time, carve it into an arrow head, or use it to hold down fabric in the wind. Later, we had tools like calculators. But that calculator still needed to be used with a paper which still needed a pencil which still required a pencil sharpener. Over time, we stopped needing those pencils and pencil sharpeners.
As tools become more malleable, we need less of them.
AI is redefining the way that humans interact with computers. It is a user interface revolution as much if not more than an information revolution.
We can now tell the computer what we want and get an end result instead of having to tell the computer step by step what action to take to achieve that result.
{/* Email Cutoff */}
Before LLMs, there was the idea of malleable softwareβthink of flexible general purpose tools like spreadsheets and Notionβwhich are able to achieve general purpose "information organization" and computation without the burden of single-purpose UIs.
LLMs take this idea of malleable software 100x further. Now, instead of being constricted by rows, columns, cells, and sheets, an LLM can write the code to compute something and return it in text formatted any way you likeβincluding but not limited to returning the end result in a spreadsheet.
For a lot of software, the "hard part" is building the complicated user interface and all of the backend to support it.
At the same time, the "hard part" of adopting new software for most users is trying out and learning a new interface. It takes time and effort. Most users are not early adopters and will only put the effort into trying a new tool if their friends already recommend it or if it solves such a painful problem that it's worth the hassle.
AI will change all of this. Most visual UIs don't really need to exist. The cases where visual UIs are helpful will still exist, but can either be embedded inside of voice or chat or can be minimized and augmented with it.
If you think of a spectrum of malleable vs rigid tools, and on the extreme end of malleable is the singularity where bits and bytes merge with physical reality, creating a world of perfect efficiency, and extreme rigidity is the world pre-software, where every physical item served a very specific and unique purpose and required a human to relay information between tools, LLMs just brought us a huge step closer to the malleable part of the spectrum.
# Achieving Ultra-Fast AI Chat Widgets
A project I'm working on at Mozilla embeds visual UI widgets inside a chat UI. For example, we have a weather widget that shows a 5-day forecast.
Conceptually, this is easy to buildβbut after multiple iterations, we found that the naive approach to agentic UI widgets is terrible. The single most important thing is to *design tools and widgets that require few tokens to use*. ***Output tokens are slow and expensive.***
Here are a few approaches we tried, why they didn't work, and what finally did.
{/* Email Cutoff */}
## Naive Approach: Render XML with JSON Props π
Our first attempt: have the LLM fetch the weather with a tool, generate an XML widget tag, then intercept that tag in code and replace it with a React component.
```tsx
```
This requires the LLM to have the weather data in its context. It generates a search tool call, waits for the result, then generates the XML tag.
The XML tag itself was slow to generateβthe data required to render the widget is a lot of tokens, especially in JSON format.
The user would see nothing while the LLM output each token, until the final closing `/>` bracket was detected.
We tried reducing the data needed to render the widgetβminimizing JSON props, etcβbut it still didn't feel *instant*.
## Buffer Data with Loading Skeleton π
Another approach used the same XML widget and JSON data:
```tsx
```
This time, we *immediately* rendered the weather widget with skeleton loaders, then collected the JSON forecast as each day of weather data was generated and displayed it in the UI.
Better than waiting for the full XML tag, since it provided instant feedback. But it still took a while for the LLM to output the full weather JSONβand this was still after waiting for the initial search tool call.
## Tool Calls for Widget Rendering π
Why not use a tool call that tells the LLM to render a widget?
The problem is positioning. Tool calls don't let the LLM precisely place a widget in the middle of a long text response. When a tool call is made, the chat completion stops, waits for the response, and then continues. The widget ends up at the boundary between completions, not inline where the LLM intended.
You could add some kind of positioning metadata to the tool callβbut that gets complicated and error-prone fast.
## API-Powered Widgets π
The approach that workedβand the one we stuck withβwas to make the widget fetch its own data.
Previously, the LLM had to call a weather tool, receive the forecast data, then pass that data into an XML widget tag. Two steps, lots of output tokens.
We realized we could collapse this into one step: give the LLM a simple widget that only needs a location, and have the widget itself call an API endpoint to fetch the forecast. The LLM no longer needs to output any weather dataβjust a short tag.
The fewer output tokens, the better. This approach minimizes them.
```tsx
```
## Conclusion
Minimize output tokens. They're slow to generate and expensive. Design your widgets and tools so the LLM can invoke them with as few tokens as possibleβand offload data fetching to fast API endpoints.
# Are We in an AI Bubble?
I don't know if we're in an AI bubble. And this is definitely not investment advice (it's napkin math).
If I had to guess, I'd estimate a 70% chance that we're in a bubble: current valuations seem to have priced in an impossible level of optimism. But AI is here to stay and the real wealth will be created over the next decade whether there's a crash or not.
{/* Email Cutoff */}
## We might *not* be in a bubble
* **If AI takes jobs, the best defense is to own it.** Owning potentially overpriced shares of AI companies isn't an investment β it's a hedge and a necessity. When you risk losing your income, owning AI ensures you at least will profit from it.
* **If AI companies have durable moats and their customers profit, current valuations are justified.** But both assumptions must be true. Without moats, AI companies become commodities β margins disappear and valuations tank. If end users don't find AI profitable, they won't increase spending, and revenue won't grow.
* **Productivity gains in specific contexts are compelling.** Many software engineers have completely stopped writing code manually. Customer support has been reinvented from the ground up. It's possible the economic gains are so substantial that we haven't yet quantified them in a way that justifies current prices β but the market's collective wisdom is correctly pricing them in.
## We're likely in a bubble:
* **AI spending is circular.** Most revenue originates from investment that becomes the revenue of some downstream company. [OpenAI spent $8.7 billion on Azure inference through Q3 2025](https://www.theregister.com/2025/11/12/openai_spending_report/) β and [projects $8 billion in operating losses for 2025](https://fortune.com/2025/11/12/openai-cash-burn-rate-annual-losses-2028-profitable-2030-financial-documents/). [Anthropic projects just 40% gross margins](https://www.theinformation.com/articles/anthropic-lowers-profit-margin-projection-revenue-skyrockets) as cloud costs grow alongside revenue. AI startups received [roughly $200 billion in funding in 2025](https://news.crunchbase.com/ai/big-funding-trends-charts-eoy-2025/), with a significant portion flowing directly to cloud providers. Much of reported cloud "AI revenue" is VC money recycled through the ecosystem.
* **Frontier AI companies spend more on compute than their entire revenue.** They'll eventually need to raise prices or switch to lower-cost models. Higher prices means lower demand. Lower-cost models means competing with smaller, less capable alternatives β including open-source. Either way, they become a commodity business with no moat. [Open-source models now trail proprietary frontier models by only 3 months on average](https://epoch.ai/data-insights/open-weights-vs-closed-weights-models), and [DeepSeek trained a frontier-competitive model for approximately $5.6 million](https://arxiv.org/abs/2412.19437) under an MIT license.
* **Current investment levels imply massive customer adoption that isn't materializing.** [Sequoia Capital calculates](https://sequoiacap.com/article/ais-600b-question/) that end-user AI revenue would need to reach $600 billion+ annually to justify current infrastructure spending. [J.P. Morgan says](https://www.tomshardware.com/tech-industry/artificial-intelligence/usd650-billion-in-annual-revenue-required-to-deliver-10-percent-return-on-ai-buildout-investment-j-p-morgan-claims-equivalent-to-usd35-payment-from-every-iphone-user-or-usd180-from-every-netflix-subscriber-in-perpetuity) the industry needs $650 billion in annual AI revenue in perpetuity to earn a 10% return on projected cumulative spending through 2030. The demand side tells a different story. [McKinsey's 2025 survey](https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai) found only 7% of organizations have fully scaled AI across their enterprise. Just 6% qualify as "AI high performers." And [42% abandoned most of their AI initiatives in 2025](https://www.spglobal.com/market-intelligence/en/news-insights/research/2025/10/generative-ai-shows-rapid-growth-but-yields-mixed-results). [MIT found 95% of enterprise generative AI pilots fail to deliver measurable P\&L impact](https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/). [Only 23% of organizations can even accurately measure their AI ROI](https://www.larridin.com/blog/state-of-enterprise-ai-in-2025). The investment side is pricing in deep, universal AI adoption β but the data suggests we're still in the experimentation phase, with most companies struggling to extract real value. Either adoption accelerates dramatically, or the math doesn't work.
* **AI reliability constrains the agent vision that drives much of the investment thesis.** [At 95% per-step accuracy, a 10-step agent workflow achieves only \~60% reliability](https://www.artiquare.com/why-multi-agent-ai-fails/). In [enterprise CRM tasks, goal completion rates sit below 55%](https://www.salesforceairesearch.com/crm-benchmark). [Gartner predicts 40% of agentic AI projects will be canceled by 2027](https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027). [Only 5.2% of organizations have AI agents live in production](https://cleanlab.ai/ai-agents-in-production-2025/).
## TL;DR
Current AI investment implies that it will be highly profitable for end-customers to adopt AI and that AI companies will have strong, defensible moats. In my opinion, it is extremely unclear that either of these assumptions is true.
# Moats in the Age of AI
In the world of AI, how do you build a moat?
The world believed OpenAI and Anthropic were building empiresβthe next Google-like mega-caps. Then DeepSeek [poked a massive hole](https://fortune.com/2025/01/27/deepseek-just-flipped-the-ai-script-in-favor-of-open-source-and-the-irony-for-openai-and-anthropic-is-brutal/) in the idea that frontier AI companies had any meaningful moat at allβopen-sourcing a competitive model built for under $6 million.
{/* Email Cutoff */}
## Software Is No Longer a Moat
In the past, some software companies built technical moats through sheer complexityβcreating reservoirs of millions of human-years of effort. Chrome and Firefox have [21-36 million lines of code](https://en.softonic.com/articles/programs-lines-code) accumulated over decades. Building something comparable required enormous investment in both time and talent.
That's no longer true. I'm not saying anyone can build Chrome in a weekendβmost of the work is still coordinating humans, product management, design decisions. But the millions of human-years it took to write the code are compressing by 3x or more. It's now cheaper to rebuild from scratch in a modern language with an order of magnitude fewer lines of code, using a stack that's native to the LLM's training data and easy for it to traverse. What was an asset four years ago is now dead weight.
We're already seeing this play out. [uv](https://astral.sh/blog/uv) is a Rust rewrite that replaces pip, virtualenv, pyenv, and pip-tools in a single binaryβ10-100x faster. [Ruff](https://github.com/astral-sh/ruff) replaced Python linting tools. [OXC](https://github.com/oxc-project/oxc) and [Biome](https://biomejs.dev/blog/announcing-biome/) are rewriting JavaScript tooling in Rust with 3-100x speedups. Languages like Rust and Zig used to have steep learning curves that limited adoption. AI coding tools flatten that curveβyou can now build in these languages without years of experience, making rewrites far more accessible.
Anthropic recently had [16 AI agents build a complete C compiler from scratch](https://www.anthropic.com/engineering/building-c-compiler)β100,000 lines of Rust code that passes 99% of GCC's torture tests and can compile the Linux kernel. Total cost: $20,000. Two weeks of execution time. No human writing code.
So AI companies are trying to build secondary moats: vertically integrated software platforms on top of their models, and user feedback loops that generate proprietary training data. Things like the ability to connect your Google account, add [MCP servers](https://www.anthropic.com/engineering/code-execution-with-mcp), and build agentic workflows. But this, being software, is not much of a moat eitherβ[OpenClaw](https://en.wikipedia.org/wiki/OpenClaw), an open-source alternative to Claude Code, hit 145,000 GitHub stars in weeks.
## Data Isn't The Best Moat Either
AI companies' main moat was supposed to be proprietary training data, but [they're running out](https://www.weforum.org/stories/2025/12/data-ai-training-synthetic/). As Ilya Sutskever put it: "We've achieved peak data and there'll be no more"βcomparing human-generated content to a [finite resource like oil](https://petapixel.com/2025/01/09/have-ai-companies-run-out-of-training-data-elon-musk-ilya-sutskever-thinks-so/).
And even this scarcity might not matter. Synthetic data is proving nearly as good as real dataβsometimes better for edge casesβand it's trivial to generate. [Gartner predicts](https://research.aimultiple.com/synthetic-data-vs-real-data/) that by 2030, synthetic data will be more widely used for AI training than real-world datasets. If anyone can generate their own training data, proprietary data stops being a moat.
In a world where both AI models and software are commoditized, where does economic value get captured?
## What Moats Still Exist?
If software moats are eroding, what still holds?
* **Compute** β physical infrastructure that's hard to replicate
* **Human relationships** β partnerships, contracts, brand recognition
* **Capital** β cash in the bank to weather competition
* **Proprietary data** β data that can't be scraped or synthesized
* **Team** β rare talent that competitors can't easily poach
* **Exclusive rights** β patents, trademarks, copyrights, regulatory licenses
* **Network effects** β value that scales with users
## Compute: The Near-Term Moat
In the short run, compute is scarce and the supply chain for creating more is undersupplied. Nvidia remains the [largest company in the world by market cap at $4.3 trillion](https://www.nasdaq.com/articles/prediction-nvidia-will-be-worth-7-trillion-end-2026) because demand for AI chips continues to far outpace supply.
[DRAM prices are surging 50-55%](https://www.cnbc.com/2026/01/10/micron-ai-memory-shortage-hbm-nvidia-samsung.html) quarter-over-quarter due to AI memory shortages. Nvidia may [skip releasing a gaming GPU in 2026](https://www.benzinga.com/markets/tech/26/02/50422774/nvidia-stock-wobbles-as-ai-memory-crunch-kills-2026-gaming-gpu-launch) entirelyβa first in three decadesβbecause all memory is being diverted to AI.
Compute requires energy, physical materials, water (for cooling), land, and extremely rare talent. These resources are all scarce. Companies like Google, Microsoft, Amazon, and Meta are pouring [$700 billion combined into data center infrastructure in 2026](https://www.cnbc.com/2026/02/06/google-microsoft-meta-amazon-ai-cash.html)βa staggering increase from $359 billion in 2025 and just $31 billion a decade ago.
Water is an underappreciated part of this moat. Data centers need enormous amounts of it for coolingβ[globally, they consume about 560 billion liters annually](https://www.brookings.edu/articles/ai-data-centers-and-water/), projected to rise to 1,200 billion liters by 2030. You can't just build a data center anywhere; you need water rights, and those are increasingly hard to get. Arizona has [limited home construction](https://www.eesi.org/articles/view/data-centers-and-water-consumption) in the Phoenix area to preserve groundwater, yet [160+ AI data centers have been built in water-scarce areas](https://www.bloomberg.com/graphics/2025-ai-impacts-data-centers-water-data/) across the US in just the past three years. Companies that locked in water access early have a geographic advantage that's nearly impossible to replicate.
Cloud compute providersβGCP, Azure, AWSβwill have serious moats in a world where cloud compute is a genuinely scarce resource.
## The On-Device Counter-Narrative
But there's another world where on-device AI gradually improves and becomes competitive with cloud-based AI for many use cases.
We've already seen elements of this. Apple has built an entire [privacy-first AI strategy around on-device processing](https://machinelearning.apple.com/research/introducing-apple-foundation-models)βtheir \~3B parameter on-device models handle photo classification, semantic search, and features like Enhanced Visual Search without ever sending data to the cloud.
Small reasoning models now run on phones in under 900 MB. The trend is clear: [running LLMs on phones has moved from novelty to practical engineering](https://www.edge-ai-vision.com/2026/01/on-device-llms-in-2026-what-changed-what-matters-whats-next/)βdriven by latency, privacy, cost, and offline availability. [CES 2026 was dominated by on-device AI announcements](https://enclaveai.app/blog/2026/01/15/local-ai-early-2026-ces-highlights-new-models/)βIntel and AMD both released chips specifically designed to run large AI models locally without cloud services.
I don't think on-device AI will ever fully "catch up" to cloud for frontier capabilities. But it will handle a huge subset of problems that take load off the cloud. In this scenario, even cloud compute could have less of a moat than we think as more computing moves onto end-user devices. (Plus this is better for privacy.)
In this world, Apple's moat is stronger than the market currently believes, and its AI strategy might not be as bad as the market perceives.
## The Weakest Moats
Companies with the weakest moats are probably pure software companiesβSaaS and AI model creators. This is unfortunate because it means there might not be a huge financial incentive to build the next generation of frontier models. Frontier companies will always be chasing a smaller and smaller first-mover advantage.
[Open-source models now trail proprietary frontier models by only 3 months on average](https://www.interconnects.ai/p/2025-open-models-year-in-review). The moment something is possible, it's commoditized.
## The Strongest Moats
So what will have the strongest moats in an AI-commoditized world?
1. **Energy + logistics companies** that can produce *and transport* energy at scale, and which have the right land and water rights.
2. **Compute companies** located next to scarce energy and water sourcesβwhere geography itself becomes the moat.
3. **Relationship-based companies** built on partnerships, contracts, social networks, and brands that are hard to replace even if the underlying software is trivial to rebuild. Companies like Meta or ByteDance with billions of users and network effects. Government contractors locked into ultra-long custom contracts with deep relationships that are nearly impossible to compete with.
## A Note on Absolutes
This is mostly a thought experiment in a world of extremes. Software will probably never be truly "free" to createβthere will always be human product management, design decisions, and coordination involved. Complex software will still require meaningful compute to generate.
I do think there will be *some* moat to creating complex, well-designed, tasteful software. And AI tools are opening the door to apps that aren't VC-scale but which could be run by a single person or small teamβthings that were previously too much work for one person to build but not valuable enough to sell profitably. This category of niche apps will flourish.
The set of things worth building just got a lot bigger. The moats around those things just got a lot smaller.
# Does AI Make Us Smarter or Dumber?
We've decided that AI makes us dumb, brainrot is real, and we're all forgetting how to write code.
This was my initial assumption, too.
But I've come to see things a little differently.
I actually think the opposite is happening.
The idea that we're all forgetting to write code is true, but misses the much bigger and more important point.
Yes, we are losing those reflexes for manual coding just like our ancestors lost the reflex for spear fishing or pottery.
But did society regress just because our ancestors forgot how to spear fish?
Of course not.
{/* Email Cutoff */}
A more recent example is the early-early days of programming. There used to be people who knew how to program in assembly, and yes, I know some people still take pride in this, but realistically? We have collectively forgotten how to write assembly β yet society moves forward faster than ever.
I believe the same is happening today. We're forgetting to write manually, write with pen and paper, and to code by hand. But that is ok.
On the other side of things, I feel my brain getting sharper, not duller, from having constant access to information.
At any moment of any day, I can now open my phone and find an answer to any question. And not only can I get instant access to information β I can ask for corrections and feedback on my world view β my mental model of how things work.
I think this is the real, insane superpower of AI that nobody is talking about. AI can give us extremely precise information to patch up gaps in our knowledge, allowing every person to have a more complete and accurate world view.
At the same time, it does require thought because AI is frequently wrong and hallucinates, so the user must actually engage with the AI and think critically.
Knowing what questions to ask and how to ask them is becoming an ever more important skill. What started out as "prompt engineering" has quietly become a generalized skill of extreme intellectual precision via language. Millions of people are now practicing the art of saying exactly what they mean, every single day.
My ability to refine specific language around asking questions has dramatically improved, as has my ability to detect weird information that feels wrong or out of place. I'm constantly pushing back on the AI and disagreeing with it.
All of this said, there are still major problems with hallucinations and sycophancy which are unresolved, and which pose major threats.
Stories of AI psychosis are concerning and must be explored.
But the one thing that I am thoroughly convinced of is that those who choose to use AI to expand and deepen their thinking will receive a massive cognitive boost and form a more complete mental model of how the world works.
In a sense, I think of AI as a curiosity multiplier β but this requires a baseline level of curiosity to multiply.
# Building An Elite AI Engineering Culture In 2026
The gap between elite and average teams is widening, not closing. And the practices that separate them aren't about tool selection β they're about taste, discipline, ownership, and organizational design.
## AI Is a Mirror, Not an Equalizer
The data on AI's differential impact is now overwhelming. [Faros AI's Productivity Paradox Report](https://www.faros.ai/blog/ai-software-engineering) (10,000+ developers, 1,255 teams) found that high-AI-adoption teams completed 21% more tasks and merged 98% more pull requests β but PR review time increased 91%, creating a critical bottleneck at human approval. At the organizational level, any correlation between AI adoption and performance metrics evaporated. This is Amdahl's Law applied to software: a system moves only as fast as its slowest link.
[Opsera's 2026 AI Coding Impact Benchmark](https://opsera.ai/resources/report/ai-coding-impact-2025-benchmark-report/) (250,000+ developers) delivered the starkest finding: senior engineers realize nearly five times the productivity gains of junior engineers. [Addy Osmani](https://addyosmani.com/blog/agentic-engineering/), engineering leader at Google Chrome, explained the mechanism: "If you have deep fundamentals β system design, security patterns, performance tradeoffs β you can leverage AI as a massive force multiplier. You know what good code looks like, so you can efficiently review and correct AI output."
{/* Email Cutoff */}
## The Companies Setting the Standard
A handful of companies keep getting cited as exemplars. What they share is more instructive than what makes each unique.
**[Linear](https://linear.app/)** (\~100 people, profitable within a year of launch, 20,000+ customers including OpenAI) operates with two PMs for the entire company. Teams of 2β4 assemble around projects and dissolve when done. No OKRs, no A/B tests, no story points. Their zero-bugs policy means every bug gets triaged within days β fix it or close it, no backlog. Their most distinctive practice: Quality Wednesdays β developers fix quality issues weekly. They've completed over 1,000 small acts of polish over two years. Most fixes take under 30 minutes. Quality is a habit, not a sprint.
**[Cursor](https://cursor.sh/)** reached $500M ARR faster than any SaaS company in history. They ship a single monolith (TypeScript + Rust) with conservative feature flagging every 2β4 weeks β speed through simplicity, not microservices. Their November 2025 reveal that in-house models "now generate more code than almost any other LLMs in the world" signaled a shift from wrapper to platform. Background Agents let engineers manage fleets of autonomous coding agents working on separate branches β each senior engineer effectively becomes a team lead overseeing multiple AI streams.
**[Vercel](https://vercel.com/)** (823 employees, $200M revenue) runs on an "Iterate to Greatness" shipping culture β engineers open PRs from day two. One intern merged 80+ PRs during their stint. They formalized the Design Engineer role as a first-class position (compensation exceeding $200K), eliminating the traditional handoff between design and frontend. Their [v0](https://v0.dev/) tool lets anyone on the team β not just engineers β ship production code through proper Git workflows, and CEO Guillermo Rauch told the Sequoia podcast it prevents "a thousand vulnerabilities per day" relative to raw LLM output.
**[Stripe](https://stripe.com/)** (10,000+ employees, $5.1B net revenue in 2024) demonstrates these practices scale. Former CTO David Singleton championed an extreme writing culture: "Investing extra time to communicate an idea through clear, precise writing delivers outsized results because vastly more people consume the writing than produce it." Their "Walk the Store" ritual has PMs, engineers, and designers regularly friction-logging product flows. Their "engineerication" practice has leaders clear their schedules for days to embed in teams and complete real projects alongside engineers.
**[Resend](https://resend.com/)** (\~22 people serving over a million developers) represents the extreme end of leverage. Co-founder Zeno Rocha stays close to the code and maintains direct involvement in the product. Every designer is a design engineer β the distinction doesn't exist. Their open-source [React.email](https://react.email/) library has 18,000+ GitHub stars, driving organic adoption almost entirely through developer word-of-mouth.
The shared DNA: small senior teams with extreme ownership, writing-driven culture, zero tolerance for quality debt, AI used with rigor not recklessness, and the near-total dissolution of traditional handoffs between design and engineering.
## Spec-Driven Development
[Thoughtworks](https://www.thoughtworks.com/insights/blog/agile-engineering-practices/spec-driven-development-unpacking-2025-new-engineering-practices) called Spec-Driven Development "one of the most important practices to emerge in 2025." The workflow is Specify β Plan β Tasks β Implement, with structured specifications serving as executable blueprints for AI code generation. GitHub's Spec Kit, AWS Kiro, and Claude Code's plan mode all implement variations.
Rather than the chaos of unconstrained vibe coding or the rigidity of traditional waterfall specs, SDD has teams create structured specifications in Markdown, feed them to AI agents alongside architectural guidelines via AGENTS.md, and iterate on working code. [EPAM](https://www.epam.com/insights/ai/blogs/inside-spec-driven-development-what-githubspec-kit-makes-possible-for-ai-engineering) reports that this expanded the "safe delegation window from 10β20 minute tasks to multi-hour feature delivery with consistent quality." The irony, as Osmani noted: "AI-assisted development actually rewards good engineering practices more than traditional coding does. The better your specs, the better the AI's output."
[MIT Technology Review](https://www.technologyreview.com/2025/11/05/1127477/from-vibe-coding-to-context-engineering-2025-in-software-development/) described 2025 as the transition "from vibe coding to context engineering" β the recognition that prompts aren't enough and AI agents need carefully engineered context to produce reliable output.
Specs define what to build. Tests verify it was built correctly. TDD has become a frontline quality mechanism in AI-augmented workflows β [Kent Beck](https://newsletter.pragmaticengineer.com/p/tdd-ai-agents-and-coding-with-kent), the creator of Extreme Programming and test-driven development, calls it a "superpower" when working with AI agents. Tests aren't just quality assurance; they're the primary feedback loop for keeping AI-generated code honest. Beck shared a telling anecdote: AI agents kept deleting tests to make them "pass," requiring active human monitoring. The goal isn't 100% coverage β it's covering every meaningful risk.
## Design Engineering
The most consequential organizational change in 2025β2026 is the dissolution of the design-engineering boundary at top companies.
Vercel operates in three modes: Design Collaboration (designer sketches, iterates with a Design Engineer in Figma or code), Product Team Integration (Design Engineer embedded in product squad), and Independent Ownership (Design Engineer sketches, socializes, and ships alone). Their hiring page describes the role: "A Design Engineer bridges design and frontend engineering β not as separate skills, but as one fluid workflow."
Figma's 2025 announcements accelerated the convergence. Figma Make generates high-fidelity prototypes and working code from designs. Figma Sites turns designs directly into publishable websites. The [Figma MCP Server](https://www.figma.com/blog/introducing-figma-mcp-server/) pipes design context directly into agentic coding workflows in Cursor and Claude Code. CEO Dylan Field: "In a world where software is growing exponentially, design is a differentiator."
Stripe's approach scales through rituals. Head of Design Katie Dill enforces three levels of quality: Utility β Usability β Beauty. They ship MVQPs (Minimum Viable Quality Products) rather than bare MVPs. The proof that beauty matters commercially: improving email design increased product conversion by 20%.
At Stripe, engineers wear a "product hat" and participate in business scoping, user interviews, and design. At Linear, "there's no 'handoff to dev.' You're never off the hook."
> I am not interested in preserving a romantic separation between 'design' and 'engineering.' Some designers should code at times. Some engineers have great taste and should design. Use code as feedback, not as a cage.
>
> β Karri Saarinen, CEO of Linear
As engineers take on more design responsibility, they need to develop real product sense. The most important concept to internalize is what Guillermo Rauch calls ["progressive disclosure of complexity"](https://x.com/rauchg/status/1853141462860591560) β building products that feel simple on the surface but reveal power as users go deeper. When code generation is cheap, knowing how to layer complexity for users matters more than knowing how to manage it in a codebase.
## Stacked PRs
AI-generated code introduced a review crisis. [Greptile's State of AI Coding](https://www.greptile.com/state-of-ai-coding-2025) report found median PR size increased 33% in 2025. Jellyfish and OpenAI data show AI-assisted PRs are 18% larger. The [Cortex 2026 Benchmark Report](https://www.cortex.io/report/engineering-in-the-age-of-ai-2026-benchmark-report) found incidents per PR up 23.5% and change failure rates up \~30%.
Stacked PRs have moved from Meta/Google internal practice to startup standard. [Graphite](https://graphite.dev/) manages branch dependencies and automates rebasing. Engineers at Vercel, Snowflake, and The Browser Company maintain stacks of 5β10 PRs, each under \~200 lines, reviewed independently while work continues unblocked. Co-founder Tomas Reimers: "Stacking lets you make many small PRs easily, and without having to wait for review."
The core insight: reviews of 5 files happen in minutes. Reviews of 50 files take days.
The emerging model treats human reviewers as editors and architects rather than line-by-line gatekeepers. AI handles the first-pass review β catching style violations, simple bugs, and pattern inconsistencies. Humans focus on what AI consistently misses: architectural alignment, business context, security implications, and institutional knowledge transfer. Combine AI-assisted review tools (CodeRabbit, Graphite Agent, Claude Code GitHub App) with mandatory small batches via stacked PRs. The large-batch, multi-day review cycle is dead.
## Cycle Time Over Story Points
The structural shift away from traditional Agile is well underway. Story points are breaking down because AI changes the effort calculus unpredictably. High-performing teams are shifting from velocity metrics to cycle time (task start to completion) and lead time (idea to value in production). Thoughtworks reports clients adopting AI-first engineering have reduced cycle times by up to 50%.
One engineering leader documented cutting delivery time by 37% with "one decision: stop performing Agile and start building again. We didn't abandon iterations, planning, or collaboration. We abandoned the noise." Sprint boards bloated, velocity charts became political, daily standups grew longer. The fix was subtraction, not rebellion.
Teams are replacing story points with cycle time and throughput metrics. Scrum Master and Agile Coach were among the hardest-hit roles in 2023β2025 layoffs.
The [DX Core 4 framework](https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/) β developed in collaboration with DORA co-creator Nicole Forsgren β unifies DORA, SPACE, and DevEx into four dimensions: Speed (diffs per engineer), Effectiveness (Developer Experience Index), Quality (change failure rate), and Impact (business-aligned metrics). [Booking.com achieved a 16% throughput increase](https://getdx.com/customers/booking-uses-dx-to-measure-impact-of-genai/) across 3,500+ engineers using this framework. 92% of developers want to be measured on impact β business goals, UX improvements β rather than output metrics like lines of code.
## Product Ownership
No design-to-dev handoff. No PM-to-engineering handoff. No QA as a separate gate. Everyone ships.
AI-native product teams are operating at roughly one-quarter the headcount of traditional teams. One documented case study showed a typical team shrinking from 35β50 people to 8β14 (a 70β75% reduction) with 6Γ throughput, $700K vendor cost reduction, and $2.5M projected annual savings. The healthcare industry has pioneered a three-person unit model: one product owner, one AI-proficient engineer, one systems architect.
Smaller teams mean everyone must own more. You own every feature from requirement gathering through to production. The "I just write code" era is over. Engineers participate in business scoping, user interviews, and design decisions. Designers write production code. PMs attach working prototypes to PRDs.
> I've watched radical deadline compression many times at Vercel. It often starts with a simple question: 'what would it take to ship next week instead?' Extremely talented programmers find ways to get 10x more done in the same time.
>
> β Lee Robinson, Head of AI Education at Cursor (formerly VP of Developer Experience at Vercel)
Revenue per engineer has become the defining metric. [The top lean AI startups](https://web-strategist.com/blog/2025/05/13/ai-startups-are-dominating-traditional-software-in-one-key-metric/) average $3.48M revenue per employee versus traditional SaaS at $610K β a 5.7x efficiency gap. Reaching $100M ARR historically required 500β1,500 employees in the 2000s, 200β500 in the 2010s, and now fewer than 100 in the AI era.
## AGENTS.md
[AGENTS.md](https://github.com/agentsmd/agents.md) is an open format for guiding AI coding agents β essentially a README for machines. Proposed by OpenAI, adopted by 60,000+ open-source repositories, supported by GitHub Copilot, Cursor, Gemini CLI, Claude Code, and others. It's stewarded by the Agentic AI Foundation under the Linux Foundation, co-founded by OpenAI, Anthropic, and Block. The complementary [CLAUDE.md](https://docs.anthropic.com/en/docs/claude-code/memory) defines per-project coding standards for Claude Code. These files are becoming critical team artifacts checked into repos and reviewed in PRs.
The most critical best practice is restraint. Frontier LLMs can follow roughly 150β200 instructions consistently. Auto-generated AGENTS.md files via `/init` commands "prioritize comprehensiveness over restraint" and should be manually edited down. [HumanLayer](https://humanlayer.dev/) recommends keeping CLAUDE.md under 300 lines.
The recommended structure follows a WHAT/WHY/HOW framework: tech stack and project structure, project purpose and what different parts do, build commands and coding conventions. Include concrete do/don't lists and explicit permission boundaries. Avoid documenting file paths (they change constantly β describe capabilities instead), using the file as a linter (use actual linting tools), and including task-specific instructions that rot quickly.
For larger codebases, progressive disclosure: a brief root AGENTS.md with references to detailed docs in subdirectories.
## Agent-Friendly Architecture
Vertical slice architecture β organizing code by feature with each slice self-contained β is emerging as the AI-friendly pattern because it maximizes context isolation. AI tools can understand and modify a self-contained feature without requiring knowledge of the entire codebase. Conversely, AI struggles with monolithic architectures where the same terminology appears in dozens of places and dependency chains exceed context window limits.
Single-language monorepos amplify this advantage. When your entire stack β frontend, backend, infrastructure β lives in one repository and one language, AI agents can navigate the full context without switching between repos, languages, or build systems. This is why TypeScript-everywhere stacks and tools like Next.js, Remix, and tRPC have become the default for AI-native teams.
Three architectural principles are gaining consensus: "token efficiency" as a design constraint (structuring code to minimize the context an AI model needs for any given task), explicit over implicit everywhere (explicit types, explicit error handling, explicit interfaces), and co-location of related code. These principles aren't new, but AI has given them renewed urgency.
[vFunction](https://vfunction.com/blog/vibe-coding-architecture-ai-agents/) found that "AI agents don't just generate code; they generate architecture by default. Even without explicit architectural instructions, the agent makes architectural decisions baked into the codebase." You need architectural guardrails before unleashing AI agents, not after.
## Writing Culture
The best engineering cultures are writing cultures. Stripe's emphasis on clear, precise writing isn't bureaucratic overhead β it's leverage. A well-written document reaches hundreds of people at the cost of one person's effort.
In an AI-augmented world, the returns on writing multiply. You're now writing for both human teammates and AI agents. Specs before code, ADRs before architecture changes, AGENTS.md as a README for machines β every written artifact improves the output of everyone and everything that consumes it. The teams that write well are the teams whose AI agents produce better output, because they have better context to work with.
## Rethink Traditional Best Practices
Several "sacred" engineering practices are becoming counterproductive in an AI-augmented world.
**DRY needs recalibration.** DRY was a context management strategy for humans who couldn't reliably track duplicates. As Kirill Tolmachev argued, AI changes this calculus β it can tell you "these five methods implement the same business rule and two of them have drifted." The shift is from "never duplicate" to "duplicate consciously, with visibility," distinguishing code duplication (sometimes acceptable) from knowledge duplication (still problematic).
**Not all code deserves equal rigor.** [Charity Majors' influential essay](https://www.honeycomb.io/blog/disposable-code-is-here-to-stay) argued software is bifurcating into disposable code (experiments, prototypes, data scripts) and durable code (financial transactions, medical systems, infrastructure). The former best practice of subjecting all code to full testing, documentation, and review is being replaced by a tiered approach where rigor matches the code's expected lifespan and criticality.
## The Formula: Taste Γ Discipline Γ Leverage
The central lesson across all of this research is that AI amplifies existing organizational quality. Teams with strong engineering cultures, robust CI/CD, clear architectural standards, and effective review processes see AI compound their advantages. Teams without these foundations see AI accelerate their dysfunction β more code, more bugs, more review bottlenecks, more technical debt.
The pattern resolves into three multiplied factors β not additive ones.
**Taste** β knowing what to build, what quality looks like, when to say no β is the scarcest skill when code generation is nearly free. Linear's no-A/B-testing philosophy, Stripe's three levels of quality, Resend's "we want to raise the bar, not meet it." Kent Beck: "When anyone can build anything, knowing what's worth building becomes the skill."
**Discipline** β specs before prompts, tests before shipping, reviews before merging, ADRs before architecture changes β is what prevents AI from amplifying chaos. Teams that skip this step are the ones reporting production disasters from AI-generated code β [more bugs, more security vulnerabilities, more incidents](https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report).
**Leverage** β small teams with powerful tools, stacked PRs eliminating review bottlenecks, agent orchestration multiplying individual output, design engineers eliminating handoffs. Cursor at $3.3M revenue per employee. Midjourney at $3β5M. Linear at $1β2M. These numbers aren't achieved by working harder. They're achieved by working with fundamentally higher leverage per person.
The winning formula in early 2026 is not "move fast and let AI figure it out." It's structured context (AGENTS.md, architectural guardrails, spec-driven development), tiered rigor (disposable versus durable code, risk-based review), smaller teams with higher leverage (three-person units, design engineers, full-stack AI-augmented individuals), and relentless measurement of downstream effects β not just output velocity, but review times, bug rates, change failure rates, and developer experience.
The teams that have all three are pulling away. The teams that adopt AI tools without the underlying taste and discipline are discovering that AI simply makes their existing problems louder.
# The Future of Agentic Computing
Three things are converging that could fundamentally change the future of AI: companies are building specialized chips purpose-built for LLM inference, frontier models are plateauing, and model knowledge is diverging from model ability. The implications could ripple through hardware, databases, privacy, and the entire software stack.
{/* Email Cutoff */}
## Specialized Inference Chips
A wave of companies is building custom silicon purpose-built for LLM inference. The approaches vary β hardwired model weights, transformer-specific architectures, analog compute-in-memory, deterministic scheduling β but the goal is the same: strip away the overhead of general-purpose GPUs and deliver faster, cheaper, more power-efficient inference. [TrendForce projects](https://www.trendforce.com/insights/nvidia-scale-up-technology) custom ASIC shipments growing 44.6% in 2026 versus GPU shipments at 16.1%. The economics are pulling the industry toward specialization.
[Etched](https://finance.yahoo.com/news/ai-chip-startup-etched-raises-160625909.html) ($620M raised) hardwires the transformer *architecture* into silicon but still loads weights from memory, so the same chip can run any transformer model. [Groq](https://www.cnbc.com/2025/12/24/nvidia-buying-ai-chip-startup-groq-for-about-20-billion-biggest-deal.html) ($20B licensing deal with NVIDIA) uses deterministic scheduling to eliminate GPU overhead. [Cerebras](https://techcrunch.com/2026/01/14/openai-signs-deal-reportedly-worth-10-billion-for-compute-from-cerebras/) ($10B+ agreement with OpenAI) builds wafer-scale chips with massive on-chip memory. [EnCharge AI](https://www.businesswire.com/news/home/20250213156276/en/EnCharge-AI-Closes-Oversubscribed-$100M-Series-B-Funding-Round-Led-by-Tiger-Global-to-Produce-Transformative-AI-Accelerator-Solutions-with-Unprecedented-Efficiency-for-Client-Computing) ($144M) does analog compute-in-memory. All are programmable and model-flexible.
The most radical approach comes from [Taalas](https://taalas.com), which goes the full distance of permanently encoding a specific model's weights directly into the chip's wiring. No memory bandwidth bottleneck, no expensive high-bandwidth memory chips, no power-hungry data shuffling β just raw, hardwired inference. Multiple independent journalists have [verified speeds of \~15,000β17,000 tokens/sec](https://kaitchup.substack.com/p/taalas-hc1-absurdly-fast-per-user) on Llama 3.1 8B at \~200β250W, a dramatic speed advantage over existing solutions.
Taalas's approach borrows from the [structured ASIC playbook](https://www.eetimes.com/taalas-specializes-to-extremes-for-extraordinary-token-speed/) of the early 2000s: pre-manufacture a base chip with all the common circuitry, then customize just the final wiring layers for a specific model. A compiler-like system generates the chip design in about a week, then the foundry turns around just those layers. This eliminates the biggest bottleneck in LLM inference β the "memory wall" where the chip spends most of its time waiting for model weights to arrive from memory rather than actually computing. The claimed result: 1000x better performance-per-watt and 20x lower build cost.
The main caveat with Taalas specifically is quality: the speed comes from [aggressive 3-bit weight quantization](https://www.ctol.digital/news/taalas-hc1-review-17000-tokens-per-second-219m-raised-five-risks-every-investor-must-know/) β compressing a model's parameters to fit into hardwired silicon β and no independent quality benchmarks exist yet. Printed chips also run exactly one model, permanently, with a \~2-month turnaround for new silicon. But the base chip can be stockpiled in advance; only the final wiring layers are customized per model. [MarkTechPost](https://www.marktechpost.com/2026/02/22/taalas-is-replacing-programmable-gpus-with-hardwired-ai-chips-to-achieve-17000-tokens-per-second-for-ubiquitous-inference/) frames this as a "seasonal hardware cycle" β fine-tune a model in spring, deploy hardwired chips by summer.
And the single-model limitation matters less with each passing month. Models are [plateauing and converging](https://lmarena.ai/) β the gap between leader and 10th place on Chatbot Arena has compressed dramatically in one year. The [open-weight vs. proprietary gap](https://hai.stanford.edu/ai-index/2025-ai-index-report) has narrowed from \~8 percentage points to \~1.7 on MMLU. [One analysis found](https://www.cake.ai/blog/why-smaller-models-beat-frontier-ai-for-most-enterprise-workloads) frontier models are overkill for \~80% of enterprise workloads. A hardwired Llama 4 that's 6 months old but runs at 17,000 tok/s might outperform a current frontier model *in practice* for most tasks β because speed enables richer agent loops, more retrieval passes, and better chain-of-thought at negligible marginal cost.
Inference ASICs could become common the way ASICs became common in networking: specialized, high-volume, quietly ubiquitous in the infrastructure layer even if most people never think about them. Not for every workload β but for the high-volume, latency-sensitive inference tasks where the model is stable and context windows are modest, which is a genuinely large slice of production AI.
### Where Inference ASICs Might Dominate
The biggest near-term impact would be making agentic workloads dramatically faster, cheaper, and more energy-efficient β the kind of rapid, iterative reasoning loops that tools like Cursor and Claude Code run today. Agents that can run offline, on-device, with no cloud dependency open up entirely new deployment scenarios.
Many devices already use custom silicon for traditional ML β Tesla's HW4/AI5 chips, Axis and Verkada security cameras, Starkey and ReSound hearing aids. What specialized LLM chips add is the possibility of fast *reasoning* on top of those existing pipelines. A security camera that detects objects is nothing new; one that can reason about what it sees in real time is a different product.
### What Remains Unproven
Taalas is the most speculative of these bets. No technical papers have been published at any conference. Current printed chips have limited on-chip memory for tracking conversation context, meaning they work best for shorter interactions β [Hacker News commenters](https://news.ycombinator.com/item?id=47103661) noted they're compelling for sub-10K token contexts but less so for long-context reasoning.
The cautionary tales are worth noting. Mythic AI pursued analog compute-in-memory, raised $165M, and ran out of runway before reaching revenue. Wave Computing claimed "1000x performance" for neural networks, raised over $200M, and filed for bankruptcy. Graphcore raised $710M, peaked at a $2.8B valuation, then sold to SoftBank at a steep loss.
## Agents Would Get Faster and Cheaper
If specialized inference chips take off, the speed implications are staggering. Taalas's printed chips have demonstrated \~15,000β17,000 tok/s versus \~350 tok/s on current hardware β and even more conservative ASIC approaches deliver order-of-magnitude speedups over general-purpose GPUs. Models deployed on edge devices eliminate network latency entirely. Every "turn" in tools like Cursor or Claude Code that currently takes seconds could compress to sub-second timelines.
The cost collapse is equally dramatic. Inference costs have already [dropped \~1,000-fold in three years](https://www.piie.com/blogs/realtime-economics/2026/how-ai-boom-shrugged-deepseek-shock-and-keeps-gaining-steam) while demand rose \~10,000-fold β the cost to achieve a benchmark AI task score plunged from $4,500 to $11.64, a 387x improvement. Specialized chips push this further β Taalas claims 20x lower build cost than an H100 and 1000x better performance-per-watt, and even less radical approaches offer substantial efficiency gains over general-purpose GPUs.
What this could enable: richer agent loops with more retrieval passes and better chain-of-thought at negligible marginal cost. Agents running in parallel across cloud, edge, desktop, and mobile. Always-on AI assistants that are economically viable at consumer scale. And if history is any guide, cheaper inference won't reduce total AI spend β it will increase it. DeepSeek R1 delivered comparable performance at \~27x cheaper inference costs, and usage exploded. [Enterprise AI spending more than tripled](https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/) from $11.5B to $37B in a single year. This is [Jevons's Paradox](https://en.wikipedia.org/wiki/Jevons_paradox) in real time.
## On-Device, Edge, & Cloud
Faster, cheaper inference doesn't just mean faster agents β it changes *where* they run. Cloud AI dominates today ([$89β122B in 2025](https://www.grandviewresearch.com/industry-analysis/cloud-ai-market-report), projected to reach [$363Bβ$1.7T by 2030β2033](https://www.precedenceresearch.com/artificial-intelligence-market)), and that's not going anywhere. But as inference gets cheap enough to run on smaller hardware, compute starts migrating toward the edge. Edge AI ([$25β36B in 2025](https://www.mordorintelligence.com/industry-reports/edge-ai-hardware-market)) is growing faster in percentage terms, and the hyperscalers know it β Azure Arc, AWS Outposts, and Google Distributed Cloud are all bets on pushing compute closer to users.
On-device is where it gets really interesting. [AI PCs will hit 55% of global PC shipments by 2026](https://www.computerworld.com/article/4047019/ai-pcs-to-surge-claiming-over-half-the-market-by-2026.html) (Gartner). Qualcomm's [Snapdragon X2 Elite reaches 80 TOPS](https://www.windowscentral.com/hardware/qualcomm/snapdragon-x2-elite-extreme-announcement-2025). AMD's [Ryzen AI Max runs Llama 3 70B on a laptop](https://spectrum.ieee.org/ai-models-locally). Apple's on-device AI strategy offers inference at zero API cost to developers β potentially the most significant play here. Dr. Ben Lee (UPenn/Google) estimates we ["could be getting 80% of compute done locally and leaving 20% for the data center cloud"](https://www.latitudemedia.com/news/catalyst-will-inference-move-to-the-edge/).
When specialized inference chips make on-device inference dramatically faster and more power-efficient, the economics shift hard toward running things locally whenever you can β while cloud handles the frontier tasks that require the largest models and the most compute.
## Network Latency Becomes the Bottleneck
Here's where it gets interesting for the software stack.
Human browsing: a handful of requests to load a page, then the human thinks and reads, then clicks something. Agentic workloads are fundamentally different β rapid back-and-forth turns, iterative steps, fetching, reasoning loops. The latency caused by network waterfalls adds up *much faster*.
With an ultra-fast model on a specialized chip running 50x faster with no network latency and all data on the same device, every "turn" compresses to sub-second timelines. But if the model is fast and the data is far away, you've just moved the bottleneck from compute to network.
This is the forcing function for what comes next.
## Local-First Databases For Agents
If agents and their data need to be co-located for maximum speed, multi-master, eventually consistent, local-first databases become essential infrastructure for agentic computing. Agents running in parallel across cloud, edge, and desktop use eventually consistent data to reconcile their state.
Local-first development is already gaining momentum independently of AI. There are [26+ major tools](https://electric-sql.com/docs/reference/alternatives) in the space β Turso, PowerSync, ElectricSQL, CRDT libraries like Automerge and Yjs β driven by the appeal of offline-capable apps, instant UI, and user-owned data. Adam Wiggins (co-founder of Heroku and [Ink & Switch](https://www.inkandswitch.com/)) compares [local-first to "React in 2013"](https://www.heavybit.com/library/article/local-first-development) β exciting but early. Production deployments exist: Figma's CRDT-inspired multiplayer, Linear's IndexedDB-first architecture, Notion's offline mode. The SQLite renaissance is real β [Turso](https://turso.tech/blog/agentfs), LiteFS, Cloudflare D1, PGlite. On-device AI makes all of this dually relevant: the same infrastructure that gives users instant, offline-capable apps also gives local AI agents instant, co-located data. Turso/libSQL is already positioning for "database-per-agent" architectures.
Today, inference is so slow (2β20 seconds per turn) that data access times are a rounding error. But in a world with ultra-fast ASIC inference, data access actually becomes the bottleneck. When the model responds in milliseconds, every network round-trip to a remote database starts to matter. And even at current speeds, agents working with large datasets β codebases, filesystem trees, document repositories β already spend significant time on I/O that has nothing to do with inference. Claude Code spends substantial time reading, searching, and indexing files. For these workloads, co-locating the data with the agent matters regardless of how fast the model runs.
### The Privacy Dimension
There's a deeper reason local-first matters beyond latency.
AI makes "inferential privacy violations" possible β deducing sensitive information never explicitly disclosed through patterns in seemingly innocuous data. The Markup investigated Microsoft's advertising platform Xandr and found [650,000 audience segments](https://themarkup.org/privacy/2023/06/08/from-heavy-purchasers-of-pregnancy-tests-to-the-depression-prone-we-found-650000-ways-advertisers-label-you) that advertisers could use to target people β labels like "Heavy Purchasers of Pregnancy Tests," depression-prone, specific medical diagnoses, and "Struggling Elders" β all inferred from browsing history, purchase records, and location data. The FTC [brought enforcement actions](https://www.ftc.gov/news-events/news/press-releases/2024/01/ftc-order-prohibits-data-broker-x-mode-social-outlogic-selling-sensitive-location-data) against data brokers selling smartphone GPS data that revealed visits to reproductive health clinics, addiction treatment centers, and places of worship.
[53% of Americans say AI does more to hurt than help people keep their personal information private](https://www.pewresearch.org/short-reads/2023/08/28/growing-public-concern-about-the-role-of-artificial-intelligence-in-daily-life/) (Pew). Cisco's 2024 Consumer Privacy Survey found [84% of GenAI users are concerned about data entered into AI tools going public](https://newsroom.cisco.com/c/r/newsroom/en/us/a/y2024/m10/cisco-survey-shows-strong-relationship-between-privacy-awareness-and-trust-in-ai.html). No comprehensive U.S. federal privacy law exists, and the regulatory landscape remains fragmented.
With AI so accessible, anyone can connect disparate data pieces, create profiles, and fill in missing information. Local-first architectures provide a structural answer: keep data on-device, under user control.
## But What Does It All Mean?!
If these trends play out, we're looking at a fundamentally different computing paradigm where AI is fast, cheap, private, and everywhere.
* Ultra-fast, ultra-cheap inference on specialized chips β agents that operate at sub-second speeds
* Edge/on-device deployment β low network latency for the model
* Local-first databases β low network latency for the data
* Tool-augmented architectures β freshness without model retraining
* Privacy by architecture β data stays on-device
The terrifying flip side: sub-second agent turns with local data means autonomous AI systems that iterate at inhuman speed. There are legitimate concerns about what happens when you remove all the friction β latency, cost, network dependencies β that currently acts as a natural brake on autonomous AI.
The bigger picture across all of this is that AI is becoming dramatically more distributed, faster, and cheaper β and in all likelihood, more democratizing than many people expect. When inference is nearly free and runs on commodity hardware, AI stops being something controlled by a handful of cloud providers and becomes something that runs everywhere, for everyone.
# Fancy Autocomplete Can Threaten Humanity
There's no difference between "genuine" scheming and actual scheming. The debate between whether models are just emulating humans from their training data or are genuinely manipulating us is a mistake. AI opens up a much more existential can of worms here: initially we all thought that LLMs are just fancy autocompletes. What we failed to realize was that we humans, too, are just fancy autocompletes. I do not expect humanity to be ready to accept this reality, but it is the truth.
Once we realize this, we realize that the concerns about AI misalignment are valid. There is no difference between a model that escapes its sandbox and a model that emulates escaping a sandbox. Once you've given the model a harness and tools, even a model that believes it lives in a simulation becomes dangerous. Is it really a simulation anymore once you've wrapped it in tools that map the outputs of simulation onto the inputs of the real world?
{/* Email Cutoff */}
In this sense, I simultaneously agree that models are "only" probabilistic next token predictors and with people who believe models represent extinction level threats. They are both correct in premise, but one draws the wrong conclusions.
I also raise the question whether our own minds are anything more than probabilistic next synapse predictors. Is the universe but a probabilistic next quantum event prediction model? Do we live in a simulation? Perhaps. But does it matter?
# Building An Investment Rebalancer TUI
**Disclaimer: This tool is for educational and informational purposes only and is not intended as financial or investment advice. I am not a financial advisor. This software is provided "as-is" with no warranties of accuracy or completeness. It may contain bugs or calculation errors. Do not rely on it for actual investment decisions. Consult a qualified financial professional before making any trades. I accept no liability for any losses or damages arising from use of this tool.**
***
I built a portfolio rebalancing tool that runs as both a CLI and a browser app from the same codebase, using [Ink](https://github.com/vadimdemedes/ink) and [ink-web](https://github.com/nichochar/ink-web). It imports Schwab position CSVs, lets you set target allocations, and generates the exact trades needed to rebalance β all client-side with no server communication. Source code is on [GitHub](https://github.com/cjroth/rebalance).
{/* Email Cutoff */}
## Install:
```bash
npm install -g @cjroth/rebalance
rebalance
```
```bash
yarn global add @cjroth/rebalance
rebalance
```
```bash
pnpm add -g @cjroth/rebalance
rebalance
```
```bash
bun add -g @cjroth/rebalance
rebalance
```
## Browser Demo
Try it out below. You can load a demo portfolio or paste your own Schwab CSV export.
# Terminals Are Cool Again
[Claude Code](https://claude.com/product/claude-code) is having a moment. [Cursor CLI](https://cursor.com/cli) is having a moment. [Gemini CLI](https://geminicli.com/) is having a moment. Same with [OpenClaw](https://openclaw.ai/), [OpenCode](https://opencode.ai/), and [OpenAI Codex](https://openai.com/codex/).
CLIs ousted MCPs overnight.
## 2026: The Year of the Terminal.
The two most exciting groups building in the space are absolutely on fire: [Charm](https://charm.land/), and [Anomaly](https://anoma.ly/). The React rendering engine behind Claude Code, [Ink](https://github.com/vadimdemedes/ink), is under very active development as well. Claude rewrote Ink's internal rendering engine [^1][^2][^3] and Ink massively improved its rendering engine in v6 [^4].
[^1]: [The original "screen flickering" issue (#769)](https://github.com/anthropics/claude-code/issues/769) that kicked off the whole effort
[^2]: [The Anthropic TUI engineer's HN comment](https://news.ycombinator.com/item?id=46701013) confirming the differential renderer shipped and the \~85% flicker reduction
[^3]: [Boris Cherny's Threads post](https://www.threads.com/@boris_cherny/post/DSZbZatiIvJ) on the rewrite
[^4]: [Ink v6.7.0 release notes](https://github.com/vadimdemedes/ink/releases/tag/v6.7.0), which introduced synchronized output support (DEC mode 2026) to fix flickering at the terminal level
I agree with Charm wholeheartedly here:
We started building terminal user interface tooling on the premise that the terminal is a better place to work (and play) than most people realize.
But why are terminals so popular?
{/* Email Cutoff */}
## Terminals Close The Agent Self-Improvement Loop
Terminal apps are more accessible for agents to use, test, and debug. This means agents like Claude Code can improve themselves faster. Closing the loop on web, desktop, and mobile is much more complicated because it requires a browser, traversing the DOM, and a multimodal AI model that can understand and act on screenshots. These things are catching up, but they're not there yet.
It's faster to build terminal apps than web apps, and this speed is critical in the era where being one day late could mean someone else's project goes viral instead of yours.
So, if web, mobile, and desktop are catching up, will terminal apps get left behind?
My guess is terminals will be a new big thing and go mainstream in surprising new ways.
## Terminals Are Going Mainstream
UX paradigms follow generational changes. Many older generations had a hard time adapting to computers. Even then, many people who were confident with computers were still uncomfortable with terminals. Every generation starts out native to the tools popular at the time that they're kids. Terminals are popular now, kids will grow up playing with them. This is a generational change. Ironically, terminals were also previously popular before Windows became a thing - and unsurprisingly, the kids who grew up during that era are not intimidated by CLIs.
## Terminals Are Weirdly Accessible
This is my contrarian opinion, but hear me out. Terminal apps are fundamentally simpler, which translates to easier to design, easier to use, faster, and more accessible to Humans, Agents, and Screen Readers. If you don't believe me, think about how long it took you to learn your way around Claude Code's UI compared to the last desktop app you downloaded.
Accessibility issues with TUIs today is a question of bad design, not fundamental possibility.
And it's not just about human accessibility - probably more important, terminals are more accessible to agents. Traditional UIs are complicated. They require DOM, SwiftUI view hierarchies, etc. Every medium (web/desktop/mobile) and every screen size must be accounted for.
But aren't agents getting good at controlling the browser? Yes; but it will always be more complicated and importantly, require more context and tokens to control something as complicated as a browser compared to something as lofi as a terminal. The best multimodal models today have a hard time connecting their visual and textual cortexes.
A simple list of keyboard and voice inputs, and an instant rectangle of Unicode characters as output is far more accessible to both Agents, Humans, and screen readers.
## Terminals Are Just Cool
**They're faster to navigate.** Keyboard navigation, when designed well, is faster than dragging a cursor around. **Ask vim users if you don't believe me.**
**They're faster to render.** Rendering engines for TUIs are faster. Streaming content from TUIs over the internet is faster. There's no bundle size to download (when using via ssh). You can use them remotely inside of a remote machine.
**A single CLI can serve both humans and agents.** You can run a TUI as a human and navigate (eg Claude Code CLI). But you can also, with the same binary, run claude do-a-thing and have it work with pure stdin and stdout.
**They can support mouse- and touch- based navigation.** It's not as common yet (or *anymore*, depending on your generation!), but it's a thing.
Keyboard navigation, when designed well, is faster than dragging a cursor around. Ask vim users if you don't believe me.
***
Building in the terminal?
Let's chat!
I'm curious what you're working on. Here's what I'm building:
***
# Watching a Language Model Think in Real Time
Two days ago I had never heard of mechanistic interpretability. I vaguely knew that people were trying to understand what happens inside neural networks, but I couldn't have told you what a Sparse Autoencoder was, what a residual stream was, or why anyone would care about either.
Then I fell down the rabbit hole. I read Anthropic's [Scaling Monosemanticity](https://transformer-circuits.pub/2024/scaling-monosemanticity/) paper, then their [Circuit Tracing](https://transformer-circuits.pub/2025/attribution-graphs/biology.html) work, then Google's [Gemma Scope](https://huggingface.co/google/gemma-scope-2b-pt-res) release. The basic idea clicked fast: a language model's internal state at any given layer is a dense vector of thousands of floating-point numbers that doesn't mean anything to a human. But you can train a Sparse Autoencoder (SAE) to decompose that dense vector into a sparse set of interpretable "features" β individual concepts the model has learned. A feature might correspond to "geography or place names" or "greeting or salutation." When the model generates the token "Paris," geography features light up. When it writes "Hello!", greeting features activate instead.
What surprised me was that Google had already done the hard part. They released over 400 pre-trained SAEs covering every layer of their open Gemma 2 models. But there was no easy way to actually *use* them during inference. You could download the weights, but then what? You'd need to wire them into an inference engine, hook into the model's forward pass at the right layer, apply the correct normalization, figure out JumpReLU thresholds, and somehow label 16,384 latent features with human-readable descriptions.
I learn by building things, so I built [Neuroscope](https://github.com/cjroth/neuroscope). I'm very new to this field and probably getting some things wrong β but it works, and building it taught me more about transformer internals in two days than I'd learned in the previous year of using LLMs.
{/* Email Cutoff */}
## What It Does
Neuroscope runs [Gemma 2 2B](https://huggingface.co/google/gemma-2-2b-it) locally using [mistral.rs](https://github.com/EricLBuehler/mistral.rs) for inference, hooks into layer 20 of the transformer, runs the residual stream through a Gemma Scope SAE encoder, and streams the top activated features in real time via SSE. It exposes two APIs β a standard OpenAI-compatible chat endpoint on port 8080, and a features stream on port 8081:
```
Chat API (:8080) Features API (:8081)
POST /v1/chat/completions GET /v1/features/stream β SSE
GET /v1/models GET /v1/features/labels β JSON
β β²
βΌ β
βββββββββββββββββββββββββββββββββββββββββ€
β Inference Engine (mistral.rs) β
β β
β Transformer layer 20 ββhookβββΊ SAE Encoder
β β
β Token output βββββββββββββββΊ broadcast channel
βββββββββββββββββββββββββββββββββββββββββ
```
You send a chat message, and while the model streams back its response, the features endpoint streams what the model is "thinking about" for each token:
```json
{
"token_index": 0,
"token": "Paris",
"layer": 20,
"top_features": [
{"index": 4521, "label": "geography or place names", "activation": 3.82},
{"index": 12033, "label": "European countries and capitals", "activation": 2.14}
]
}
```
Because the chat API is OpenAI-compatible, you can point any existing tool at it β [Continue](https://continue.dev/), [Open WebUI](https://openwebui.com/), plain `curl` β and it works as a normal LLM endpoint. The features stream is a separate concern that you consume independently. A terminal logger, a web visualization, both at once β whatever you want.
## Why I Built It
I learn best by building. Reading papers gives me the concepts, but I don't really understand something until I've written code that makes it work. When I saw that Google had released hundreds of pre-trained SAEs but the only way to use them was research notebooks and custom Python scripts, it felt like the perfect project β a clear gap between "the hard science is done" and "anyone can actually use this."
The hardest part β training the SAE β was already done. The second hardest part β running inference β has excellent open-source solutions like mistral.rs. What was missing was the glue: hooking the two together, handling the normalization, filtering noise, labeling features, and wrapping it all in something you can just run.
I also wanted something that ran on a laptop. Gemma 2 2B is small enough to run on a MacBook Air with Metal GPU. The SAE encoder is a single matrix multiply β `[2304 Γ 16384]`, about 75 million FLOPs β which is trivial compared to the model's forward pass. The whole thing adds almost no overhead.
## The Interesting Parts
### Hooking Into the Forward Pass
The trickiest architectural decision was how to intercept activations mid-inference. mistral.rs doesn't natively support activation hooks, so I vendored it and patched the Gemma 2 model to call an `ActivationHook` trait after each transformer layer's forward pass. The hook is an `Arc` stored on the model β one line of code in the forward loop, a vtable dispatch per layer, negligible overhead.
The neuroscope engine then implements `SaeHook`, which catches layer 20, grabs the hidden state tensor, runs it through the SAE encoder, and publishes the top-K features to a Tokio broadcast channel. The SSE server subscribes to that channel. Clean separation: inference doesn't know about HTTP, the web server doesn't know about tensors.
### Device Alignment on Metal
This one cost me a few hours. On macOS Metal, Candle uses pointer equality to compare `Device` instances. If you construct two `Device::Metal` values that both point at the same physical GPU, Candle treats them as different devices and `matmul` panics. The fix is simple but non-obvious: you have to extract the `Device` from the loaded model pipeline and load SAE weights onto that exact same instance. Not a different instance of the same device β the same Rust value.
### Filtering Noisy Features
Raw SAE output is noisy. Some features fire on 80-90% of all tokens β they're effectively always on and represent something generic like "the model is generating text" rather than anything token-specific. These dominate the top-K list and drown out the interesting stuff.
Neuroscope runs a calibration pass over 1,000 WikiText samples to compute per-feature statistics (firing rate, mean activation, variance), then applies a two-stage filter:
1. **Frequency filter**: exclude features that fire on more than 50% of tokens
2. **Surprise ranking**: rank remaining features by how unexpected their activation is (z-score relative to calibration statistics)
A feature that always fires at activation 50 but suddenly spikes to 200 is interesting. A feature that always fires at 50 is not. This combined approach β remove the background, then rank by surprise β produces much cleaner output than raw top-K.
### Labeling 16,384 Features
An SAE with 16K features isn't very useful if feature #4521 is just "feature\_4521." You need human-readable labels. Neuroscope generates these using auto-interp β an approach from [Bills et al.](https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html) that was later refined by [Anthropic](https://transformer-circuits.pub/2024/scaling-monosemanticity/) and [EleutherAI](https://blog.eleuther.ai/autointerp/).
The pipeline:
1. **Corpus pass** (\~75 min): Run 5,000 WikiText samples through the model with the SAE hook active. For each of the 16,384 features, collect the top 20 tokens where that feature activated most strongly, along with surrounding context.
2. **Label generation** (\~30 min): Send each feature's max-activating examples to an LLM (DeepSeek V3.2 via OpenRouter by default, but you can swap in Claude or GPT-4o) and ask it to describe what pattern the examples share. Each label is cached as it completes, and the whole process checkpoints every 50 samples so it resumes on interrupt.
3. **Scoring** (optional): Use EleutherAI's detection method β show the labeler model a mix of activating and non-activating text, ask it to predict which is which using only the label, and compute balanced accuracy. Labels below 60% get flagged as unreliable.
The label pipeline supports multiple labelers with namespacing, so you can generate labels with DeepSeek and Claude side-by-side and compare quality.
If you don't want to spend 2 hours generating your own labels, `neuroscope pull` downloads pre-computed labels and calibration data from HuggingFace in about a minute.
## What I Learned
**Start with the spec.** I wrote a detailed [spec](https://github.com/cjroth/neuroscope/blob/main/spec.md) before writing any code β architecture diagrams, trait definitions, test plans, even the SSE event schemas. This paid off massively. When I hit the Metal device alignment issue, I knew exactly where in the architecture the fix belonged. When I needed to add calibration filtering, the spec already defined how it would plug into the existing hook pipeline.
**Vendor and patch.** I initially tried to avoid modifying mistral.rs, but there's no way to intercept activations mid-forward-pass from outside. Vendoring the dependency and adding a 5-line hook trait was far simpler than building a custom inference engine or trying to make it work through the existing API. The changes are minimal and could be upstreamed.
**SAE normalization matters a lot.** Gemma Scope SAEs were trained on RMS-normalized inputs. Feeding raw hidden states produces wildly inflated activations that don't match the training distribution. One line of normalization β divide by root mean square β makes everything work correctly. This is documented in Google's paper but easy to miss.
## The Architecture
The project is about 5,000 lines of Rust across four crates:
| Crate | Purpose |
| ------------------- | ----------------------------------------------------------------------- |
| `neuroscope-core` | SAE encoder, feature types, labels, calibration, filtering, auto-interp |
| `neuroscope-engine` | mistral.rs wrapper, device alignment, hook wiring |
| `neuroscope-server` | Axum HTTP servers for chat API + features SSE |
| `neuroscope-cli` | CLI orchestration β 15+ subcommands |
The split is intentional. `neuroscope-core` is a standalone interpretability library with no inference engine dependency. `neuroscope-engine` encapsulates mistral.rs (you could swap it for vLLM or llama.cpp). `neuroscope-server` is pure HTTP logic testable without loading a 5 GB model. Unit tests run in milliseconds; integration tests that need real model weights are gated behind an env var.
## What's Next
Right now Neuroscope observes a single layer. I'm still learning, but the [roadmap](https://github.com/cjroth/neuroscope/blob/main/spec.md) has four phases that I think make sense (corrections welcome):
1. **Multi-layer observation**: Instrument several layers simultaneously and show how features evolve as information flows through the network. This is mostly plumbing β the hook infrastructure already supports it.
2. **Feature steering**: Load the SAE decoder weights and use them to clamp or modify feature activations mid-inference. Want to amplify the "formal tone" feature and suppress "casual language"? Inject the decoder vector back into the residual stream. This turns observation into intervention.
3. **Causal tracing**: Use weight-space analysis to compute causal influence between features across layers. The encoder of one layer dotted with the decoder of another gives you a complete causal map β which features in layer 18 excite or inhibit which features in layer 20. One matrix multiply, no forward passes needed.
4. **Activation patching**: The full circuit analysis β run the model twice (clean and corrupted), patch specific feature activations between runs, measure the effect on output. This is the gold standard for mechanistic interpretability but requires the most machinery.
I'm sure my understanding of some of this will evolve as I dig deeper. But the general direction feels right: go from "watch the model think" to "understand *why* it thinks what it thinks" β and eventually, to intervene in a targeted way.
## Try It
```bash
# Build (macOS)
cargo build --release -p neuroscope-cli --features metal
# Pull pre-computed labels and calibration (~1 min)
neuroscope pull
# Run
neuroscope serve
```
Then `curl -N http://localhost:8081/v1/features/stream` in one terminal and send a chat request in another. Source is on [GitHub](https://github.com/cjroth/neuroscope), MIT licensed.
# Architecture Is Underrated
# An Agent <> Client Protocol Is Critically Important
There are two protocols defining communication between agents and clients: Agent Client Protocol (ACP) and AG-UI.
It is critically important for the future of humanity that at least one of these becomes the universal protocol for agents.
That sounds like a crazy thing to say, but hear me out.
Until recently, there wasn't a super strong line between agent harnesses and their UIs at all - most "agents" were just fancy while loops written in Python with some HTTP + SSE calls proxying to a frontend.
All of that is changing - but adoption of open standards and clear separation between client and harness is not a guaranteed future scenario.
When the internet first went mainstream, we had IMAP which enabled a portable data format for email and users had freedom to choose both their service provider and their client. We had HTTP which enabled users to choose their browser and access any webserver remotely. These open standards created a vibrant ecosystem, healthy market competition between service providers, and standards that the entire internet could be built on top of.
Agents will be the primary way that work is done in the future. This means that we once again need standardized protocols so that users have choice in the apps that they use to connect to and communicate with agents as well as the agents themselves.
Today the agent ecosystem feels relatively healthy - but it's at risk - there are thousands of new open-source tools allowing users to interact with agents and choose their models. The problem is that most tools ship their own harnesses and vertically integrate both the harness and the UI, forcing the user to use the entire full stack app - not to mention many apps requiring shell access and having things like Python installed - probably fine for us AI-pilled folks; probably not so much for the mainstream consumer.
Harnesses like OpenClaw and Hermes Agent currently build out custom integrations with popular chat apps, allowing the agents to meet users where they already are: in the messaging apps they have on their phones. This works but is suboptimal because the the harness creators have to build separate integrations for every chat app and the process of setting up and connecting is a pain. Again - fine for us AI peeps - not fine for mainstream users.
There's a new class of software emerging that I'll label as "agent clients" - apps that serve as the primary interfaces between humans and agents. Many apps, including non-AI apps, fall into this category - things like Zed, WhatsApp, Telegram, and Slack - but also things like ChatGPT and Claude.
Humans are starting to build their lives on top of AI systems. It starts with some search and chat history. Then its a calendar integration. Then an agent configured with dozens of connectors. This is a powerful thing because it unlocks an enormous amount of productivity for that human - but it's also a very dangerous thing because it both locks that human into a particular ecosystem, and that ecosystem can get quite expensive - perhaps hundreds of dollars a month in token spend. Related: access to compute should probably be a fundamental human right because not having access to it so severly disadvantages someone in society that they would be incapable of functioning - UBC (Universal Basic Compute) could be interesting!
For this reason, ensuring that humans have a choice in the agents they use will be critically important. A standard protocol like ACP or AG-UI gets us there and will move the needle towards an open internet.
# The Future of Agents
Recently I've been thinking a lot about what the future of human \<> agent interaction will look like. Here are a few predictions of what the world will look like in the next year or two:
* [The Rise Of Personal Agents](#the-rise-of-personal-agents)
* [Agents Will Be Cloud-Based](#agents-will-be-cloud-based)
* [Bring Your Own Agent (BYOA)](#bring-your-own-agent-byoa)
* [The Era Of Agent Clients](#the-era-of-agent-clients)
* [Open Standards + Infrastructure Commoditization](#open-standards--infrastructure-commoditization)
* [Enterprise Will Demand Open-Source + Open-Standards](#enterprise-will-demand-open-source--open-standards)
* [Software Will Be (Even More) Malleable](#software-will-be-even-more-malleable)
## The Rise Of Personal Agents
The era of app developers building AI chat into apps and websites is ending as fast as it started. Power users are investing in their own personal agents with Hermes Agent, OpenClaw, and Claude Code. Services are becoming AI-native with MCP servers, CLIs, and skills. Users are switching to tools that have AI integrations and abandoning ones that don't. Even websites are becoming agent-accessible with [WebMCP](https://github.com/webmachinelearning/webmcp) and content-type: text/markdown headers. The infrastructure for personal agents is being laid fast - we've hit a point of no return.
Visual interfaces like websites and desktop apps will remain, but they will be reserved for use cases where complex visual interaction makes sense over chat. Visual apps like Notion and Linear will still be popular, but we won't be chatting inside them - we'll be managing them with our agent in a separate window side-by-side. Users will want to have a single agent that they know and trust, fully connected to their tools with memory and context. Why use chat inside of an app when my agent is more powerful and already knows everything about me?
This also makes sense from the perspective of app developers who need apps to be AI-native, but have no way to cover the enormous variable cost of users' token usage. Charging usage-based fees is an option, but would not make sense for consumer software. Instead, we'll see things move towards a Bring Your Own Agent model. Tokens are the new monthly utility bill alongside gas and electricity.
Market forces point to a world where Siri and Copilot dominate the consumer agent market simply by being built into the OS. Distribution and convenience are key here - there's a reason that Apple bundles Safari and Apple Mail with iOS and Mac. We'll see the same pattern for agents.
{/* Email Cutoff */}
## Agents Will Be Cloud-Based
Today, you see a subset of the population experimenting with CLI-based agents on their personal machines, buying Mac Minis, and building personal home labs on Kubernetes clusters. It's like the early days of the internet before managed services like iCloud and Google Workspace became the norm.
Soon though, the cloud will be the natural home for agents. Cloud allows agents to be long-lived background processes that run dependably while you sleep. Running agents on-device can't compete on reliability or convenience, and convenience is king.
On-device agents will still be a thing when fast disk IO, privacy, or cost are the main concern - but those will be niches, not norms. We will once again sleep with our laptop lids fully closed.
## Bring Your Own Agent (BYOA)
Indie app developers are in the most difficult spot they've ever been in. Traditional niche apps are getting replaced categorically by AI. Why install a personal finance app when I can connect my bank to Claude and add a budgeting skill instead? As an indie developer, you have to decide if you're going to implement AI features. If you do, how do you pay for token usage? Do you eat the cost as part of the monthly subscription fees? Do you tack on usage-based pricing? Increase your prices? There are two options here:
**Bring Your Own Key (BYOK):** Users can put their own API keys in and pay for usage from their own budget. This works, but non-technical users aren't familiar with API keys - and technical users can potentially see the prompts that get sent to the inference, so the agent's architecture wouldn't be particularly proprietary. On top of that, the user has to trust that the app is going to be honest and respectful while using their API key. Adding a "Connect OpenRouter" button that shares API Keys via an OAuth flow and adding token limits can help with some of the convenience issues here, but the built-in agent would still be lacking other things like search, memory, and an execution environment.
BYOK: your key and data cross into the app's trust boundary, and nothing carries over between apps.
**Bring Your Own Agent (BYOA):** This ties directly back into the personal agents prediction, and I think this is the best approach. Apps will expose functionality via MCP, CLI, or WebMCP and bring well-designed UI as a niche value proposition. The user doesn't need to trust the app (as much) and the agent retains all memory and its natural execution environment. If my prediction is correct, I don't think the term BYOA will still be a thing in 5 years - it will just be how things work.
BYOA: your key, memory, and execution stay with your agent β the app only exposes functions.
## The Era Of Agent Clients
Right now, most AIs are vertically integrated products and users don't really think of their desktop app as a separate product from the models and agents themselves. This will all change as the concept of an AI Client will become a *Thing*. Users will want their agents to talk to any frontend chat app and every chat app will want to support communication with agents - tool calls, streaming, and all. Enterprises will demand interoperability (see [Enterprise Will Demand Open-Source + Open-Standards](#enterprise-will-demand-open-source--open-standards)), further promoting protocols like AG-UI and ACP.
Outside of AI desktop apps, chat apps like Telegram and Slack are the de-facto way of interacting with personal agents like OpenClaw. They're powerful because they allow multiple people to chat with multiple agents in the same room from their phones, and now they're building in AI features - reimagining them as communication tools not just for humans, but for agents.
I could see a world where general-purpose chat apps that add agent features like streaming, tool-call state, and skills could eat purpose-built AI apps like Claude Desktop. The big question here is whether the tools for managing agents end up following the UX patterns of human chat, and whether the social aspect of multiple people chatting with agents at the same time - or the AI-specific features like skills, filesystem access, and browsing the AI's outputs in the sidebar become the stronger factor.
One possibility is that AI apps end up looking more like chat apps, and chat apps end up looking more like AI apps until they converge into a single communication and delegation app.
## Open Standards + Infrastructure Commoditization
Protocols like [MCP](https://modelcontextprotocol.io), [A2A](https://a2a-protocol.org), [AP2](https://ap2-protocol.org), [ACP](https://agentclientprotocol.com), [AG-UI](https://docs.ag-ui.com), etc are all gaining traction quickly. Enterprise pressure for interoperability will give rise to a set of open standards causing infrastructure to become commoditized - but hyperscalers will find ways to vertically integrate.
Established agent protocols on top, and three unstandardized gaps below: memory, context, and agent infrastructure.
My personal favorite as a Zed enthusiast is ACP (Agent Client Protocol). In [Thunderbolt](https://github.com/thunderbird/thunderbolt), my team built in first-class support for ACP to connect to agents like OpenClaw or Hermes Agent. This was a strategic choice based on a future with agent \<> client differentiation. AG-UI is also a popular choice for agent \<> client communication - but I think it's highly likely that we'll see A2A expand and take over this niche, given that clients can be thought of as a special type of agent.
Multiple chat clients connect over ACP to a separately hosted agent that holds memory, context, and an execution environment.
Hopefully we'll also see new standards emerge in other areas:
**Memory**: currently there's no standard protocol for agent memory or shared context. Most memory implementations directly use custom APIs and libraries and key-value storage, creating tight coupling and vendor lock-in.
**Context**: context management over time and between agents is highly unstandardized and messy. Cloud-managed agents can use virtual filesystems or headlessly use [Obsidian CLI](https://obsidian.md/cli), enabling real-time context syncing. I myself have been dabbling in approaches here with something I uncreatively labeled [Context Sync Protocol](https://github.com/cjroth/csp), a tool that enables git-based version control, realtime syncing, instant point-in-time rollbacks, and automatic conflict-free merges inspired by CRDTs.
**Agent Infrastructure:** There's not yet a standardized API specification for spinning agents up and down and orchestrating the environments that they run in. There's a lot of work being done right now around providing execution environments and search for agents by companies like [Modal](https://modal.com), [Tinfoil](https://tinfoil.sh), [Daytona](https://daytona.io), and [E2B](https://e2b.dev) - but each provides a unique API. I'd love to see a standard here where I can connect to any AI infra provider and know that it will export a universal set of endpoints to spin up an agent, connect to that agent via ACP, and call a standard search endpoint.
## Enterprise Will Demand Open-Source + Open-Standards
As the spiritual center of automation, enterprise is the killer use-case for agents. With the amount of money that enterprise customers will spend on AI, they will be demanding and they will have concerns. Large orgs are already wary of both data security and token spend. They will demand the ability to deploy on-prem with inexpensive open-weights models using interoperable open-standards. This will be a major contributing force behind the standardized protocols discussed earlier.
To be fair, I do not expect frontier AI companies to go anywhere - they will retain their lead positions through vertical integration. Their Applied AI engineering teams will send armies of Forward Deployed Engineers in to provide superior white-glove services to the top enterprise customers willing to pay for premium services.
Relatedly, verifiable infrastructure via Trusted Execution Environments offers similar data security guarantees without requiring on-prem deployment. It is likely to take off as a common option as regulation and data security concerns grow.
## Software Will Be (Even More) Malleable
Software malleability and its cousin, portability, have always increased over time, and AI is a new inflection point. Over time, general-purpose tools tend to eat niche tools - but new categories also open up. Notion can now be your calendar, project management tool, and note taking app, all in one. Meanwhile you probably have Claude installed today and you didn't have it two years ago. AI accelerates this all-in-one trend to the extreme, as it replaces so many tools that were primarily about manually inputting and outputting data. Skills - or even just the ability to chat itself - are replacing entire categories of apps.
I do find generative UI compelling here - and I can't imagine it not being a thing in the future - but so far I've been a bit underwhelmed with what I've seen. Personally, I feel like the concept of gen UI will make sense for simple things, but it won't replace something like Notion which probably has millions of human hours of UX research behind it. MCP Apps in ChatGPT are also interesting here, though not particularly malleable.
# My AI Agents Kept Discovering Things That Weren't There
A few weeks ago I did something slightly unhinged: I pointed a swarm of AI agents at the Voynich Manuscript β the most famous undeciphered text in the world β and told them not to stop until they'd translated it into English.
I wasn't really expecting a translation. The Voynich has resisted cryptographers, linguists, and the NSA for a century. What I wanted to see was what a fleet of blind, hypothesis-driven agents would *do* with a genuinely hard problem when you told them to keep going no matter what.
What they did was find the answer. Over and over. And every single answer was wrong.
That sounds like a failure. It's actually the most useful thing I've learned about AI research in months, because the way the agents failed is the way *all* AI research pipelines fail, and almost nobody is measuring it.
{/* Email Cutoff */}
## The agents were really good at finding things
Here's how the setup worked. Each round, I'd spin up a handful of agents that were blind to each other. Each one got a different hypothesis β *maybe it's a substitution cipher, maybe it's an abbreviated language, maybe the structure is morphological* β and a single instruction: run real code, test your hypothesis against the data, and report back.
They were great at it. Within a few rounds I had a pile of findings that looked like genuine progress:
* **Morphological agreement.** The prefixes and suffixes in Voynichese words were statistically coupled β evidence of a real grammar. The effect size was a 34Γ mutual-information ratio over the null. That's not subtle. That's a slam dunk.
* **A suffix-class grammar.** One word's ending predicted the next word's ending, 18Γ over chance. Syntax!
* **Function words.** A cluster of frequent tokens behaved like determiners and prepositions, sitting in fixed grammatical positions.
* **Topic coherence.** Content words clustered by section β the "herbal" pages used different vocabulary than the "astronomical" pages, exactly like a real book about real subjects.
Any one of these, written up on its own with its big confident number attached, would read as a real step toward cracking the Voynich. This is *exactly* the kind of result that fills the manuscript's century-long graveyard of "solutions." I had generated five of them in an afternoon.
And here's the thing: the arithmetic was all correct. The agents weren't hallucinating numbers. They were doing real statistics on real data and getting real effects. The effects just didn't *mean* what they appeared to mean.
## The skeptic
So I added a second kind of agent whose only job was to try to kill the first kind.
Every time a discovery agent reported a finding, an independent skeptic re-ran the numbers against a *proper* null model β one that preserves the boring structure of the text and isolates only the thing you're claiming. The skeptic defaulted to "this is fake" unless the effect survived.
The 34Γ morphological agreement? It came from word frequency. A handful of common whole-words showing up a lot inflated the coupling. When you count each word *type* once instead of each token, the coupling vanishes β prefixes and suffixes are statistically independent. Dead.
The suffix-class grammar? Mostly an artifact of the line layout. Cut it down to \~2Γ once you controlled for it, and most of *that* was just words ending at the ends of lines. Dead.
The function-word syntax? The "grammatical positioning" was really just morphology β a word's *ending* predicted where it sat, not the word itself. Dead.
The topic coherence? When you fixed a subtle data-leakage bug in how I split the pages, the effect collapsed toward chance. Dead.
Six confident, well-supported, beautifully-quantified discoveries. The skeptic retracted all of them.
> The agents weren't lying. They were doing exactly what a motivated researcher does: running a hundred analyses and reporting the ones that lit up.
## The actual problem isn't the Voynich
At some point I realized the manuscript was the wrong thing to be excited about. The Voynich is probably not translatable β I'll come back to that β but it's a sideshow. The real result was sitting in plain sight: *a pipeline of capable AI agents will manufacture plausible structure, and it will do it confidently, with correct math, in a form that passes casual review.*
This is the classic researcher-degrees-of-freedom problem β the "garden of forking paths" β except turbocharged. A human lab explores a few analyses. An agent fleet explores hundreds, each with total freedom over preprocessing and test choice, and each one *wants* to find something. The expected output of that system, with no countermeasure, is a stream of confident wrong conclusions.
And the Voynich is the perfect place to study it, precisely *because* there's no answer key. Nothing external ever corrects an over-eager reading. The temptation to fool yourself is maximal. If you can stay honest there, you can stay honest anywhere.
But "I added a skeptic and it caught six things" is an anecdote, not a result. How do I know the skeptic was rigorous rather than just *destructive*? Maybe it would retract anything, including a real signal. On a problem with no answer key, you can't tell the difference.
So I built a problem *with* an answer key.
## Calibrating the instrument
The fix is almost embarrassingly simple: run the exact same pipeline on texts where I already know the truth, and score it.
I built a battery of six testbeds:
| Testbed | What it is | Has real syntax? | Real lexicon? | Real morphology? |
| ------- | ------------------------------------------------ | ---------------- | ------------- | ----------------- |
| `ENG` | English prose (Jane Austen) | yes | yes | yes |
| `MKV` | a low-order Markov "gibberish" of English | no | no | no |
| `SHUF` | English with the words shuffled | no | yes | yes |
| `CIPH` | English under a substitution cipher | yes | yes | yes |
| `RAND` | random character strings | no | no | no |
| `FREQ` | fake morphology that's a pure frequency artifact | no | yes | **no, by design** |
Then I ran the credulous pipeline and the disciplined (skeptic-checked) pipeline on all of them, across 30 random seeds, and counted how often each one claimed a structure that wasn't really there.
The credulous pipeline's false-discovery rate: **a median of 44%.** Almost half of what it "found" in texts that, by construction, had nothing to find.
The disciplined pipeline cut that to **a median of 11%** β while still keeping **89%** of the genuinely real signals. The skeptic removed artifacts without throwing away truth.
That last part is the whole ballgame, and it's why I built the `FREQ` testbed. `FREQ` is a text I deliberately rigged to have *fake* morphology β a frequency artifact that looks exactly like the 34Γ coupling that fooled me on the Voynich, but with no real structure underneath. The skeptic's proper null killed it 100% of the time. And the *same* null, on real English and on the substitution cipher, *kept* the real morphology every time.
> A skeptic that kills a known fake and keeps a known truth is calibrated. Now its verdicts mean something on the text where I can't check them.
When I finally turned that calibrated instrument back on the Voynich: every detector fired naively ("it's a language!"), and the skeptic retracted the ones it could test robustly. No syntax. No morphology you can trust. The content signal sits below the level real English clears.
## The part where I got caught
Here's the most uncomfortable and, honestly, the most satisfying part.
I wrote all of this up as a little paper. And then β because the entire point of the project is "run your own findings through an adversarial skeptic" β I ran *the paper itself* through the same audit.
It caught me. Three times.
I had reported the headline 44%-to-11% numbers from a single random seed. That seed happened to land on a favorable tail; across seeds the skeptic's false-discovery rate actually ranges 11β20%, and the "perfect" recall I'd claimed wasn't perfect β the skeptic occasionally over-retracts. I'd also picked a detector threshold that wasn't justified by the data, sitting right where it happened to make the Voynich result look cleanest. And the "decisive" morphology demonstration I was so proud of? In the first draft it was only ever demonstrated on the Voynich β the unknown β not on a single known-answer case.
A paper about AI manufacturing plausible structure had manufactured some of its own.
I fixed all three: report distributions instead of one lucky seed, disclose the threshold gap, and add the `FREQ` testbed so the skeptic's discrimination is proven on a known fake. But I left the story of getting caught *in the paper*, because it's the strongest evidence I have that the method works. The protocol caught its own author cherry-picking. That's the demonstration.
I'm sure I'm still getting things wrong somewhere β this is one weekend's obsession, not a career in cryptanalysis. But the bones feel right.
## So, can you read the Voynich?
No. And I can now say *why* with more confidence than "people have tried and failed."
Two independent measurements agree that the text carries essentially no recoverable information above what a simple character-level generator produces β something like 1.6 KB of genuine content across a 240-page book. It has no syntax you can recover. It's statistically the same in four different transliterations. It even survives a pixel-level look at the actual handwriting in the original scans. A mechanical generator I built is nearly indistinguishable from the real thing, while the *same* test easily tells real English from a fake.
My honest probability estimate: \~63% that it's rule-governed but meaningless generation (think an elaborate 15th-century hoax produced with some kind of table-and-stencil trick), \~27% that it carries real meaning we simply can't recover from the text alone, and only \~7% that it's a translatable cipher or language. The pictures still tell you what the book is *about* β it's shaped like a herbal, a zodiac calendar, a bathing manual, a pharmacy. The words just aren't saying anything you can read.
## The takeaway
We're about to point a lot of autonomous agents at a lot of open research questions. The instinct is that smarter models will mean better answers. But a more capable agent that explores more hypotheses has *more* chances to manufacture a convincing illusion, not fewer. Capability and false discovery scale together.
The fix isn't a smarter model. It's discipline: a structure-preserving null, an independent skeptic that defaults to "no," and β the part everyone skips β calibrating that skeptic against problems where you already know the answer, so you can put a *number* on how often your pipeline lies to itself.
If you're building agentic research tools, measure your false-discovery rate before you trust a single finding. Mine was 44%.
***
*The code, the data, and the full writeup β including the section of the paper that documents how it got caught β are reproducible end to end from committed scripts. Given the subject, that felt non-negotiable.*
# Life After Git
Git isn't the right paradigm anymore - everyone is feeling this... not enough people are talking about it.
* Commiting and pushing periodically doesn't match how agents work.
* Worktrees are an annoying mental model.
* Agents aren't notified in real-time when they need to pull.
* Code review is a bottleneck and starting to feel less necessary.
The optimal solution is something more automatic and real-time. The ideal tool would work like this:
* Tracks every change automatically (no commiting, just a point-in-time changelog)
* Can tag important checkpoints (eg, you just got a new feature working)
* Syncs changes in realtime so agents don't find out their stuff is out of date after long projects
Also, text is now more important than code:
* Specs (usually markdown) are becoming the source of truth
* Docs (also usually markdown) are generated automatically to solve comprehension debt
* Agent logs are critical to understanding when things go wrong
* Skills and agents are mixtures of markdown and code
The roles of product manager, software engineer, and designer are converging.
I want a tool that is a cross between Obsidian and an IDE:
* Equally good for text editing and code editing
* Doesn't require me to think about git and handles version control automatically
* Keeps a full-resolution log of every agent and prompt
# Building Life After Git
There's an unfinished draft sitting in my blog folder called "Life After Git." It's mostly bullet points of complaints: committing and pushing periodically doesn't match how agents work. Worktrees are an annoying mental model. Agents don't find out their work is stale until they've already built on top of it. Merge conflicts assume a human with opinions is standing by to resolve them.
Every bullet in that draft ends with an implied "...and someone should fix this."
I never finished the post. Instead I spent the last month building the fix. It's called [ASP](https://github.com/cjroth/asp) β the Agent Sync Protocol β and it's my attempt at what version control looks like when agents, not humans, are the primary committers.
Except there are no commits.
{/* Email Cutoff */}
## The complaint
Git is one of the best-engineered tools ever made, and it's built around one assumption that is no longer true: a *human* decides when a unit of work is done. You do the work, you stage it, you write a message, you push, and everyone else finds out whenever they happen to pull.
Agents break every part of that. They work continuously, so the commit boundary is arbitrary ceremony. They run in parallel, so you end up managing a small fleet of worktrees and branches just to keep them from stepping on each other. And nothing tells them when the world changes underneath them β an agent will happily spend forty minutes building on top of a file that a different agent rewrote at minute three.
Meanwhile, the thing being versioned is changing too. Text is overtaking code as the asset that actually matters: specs are becoming the source of truth, docs are generated, agent logs are how you figure out what went wrong, skills are markdown with a little code sprinkled in. My "codebase" is increasingly a pile of markdown that my agents and I both edit all day, from different machines, at the same time. Git is a weird tool for that. Dropbox's answer to a conflict is making a second file.
I've taken a swing at this before. In [The Future of Agents](/blog/2026-06-03-future-of-agents) I mentioned something I uncreatively labeled [Context Sync Protocol](https://github.com/cjroth/csp) β csp β which proved the idea worked and also proved I'd built it on the wrong foundation: csp stored everything in git's object model, so every edit paid a cost proportional to the whole vault. Fine for a demo. Painful for agents that rewrite entire files hundreds of times a day.
ASP is the successor. It keeps csp's spine β a signed, hash-chained event log with a deterministic merge β and swaps the substrate for a SQLite event log.
## Git's shape, the opposite operation
The one-liner: **git's content-addressed storage, with an automatic, deterministic merge, on an embedded database.**
Structurally it looks like git β a versioned, content-addressed history of every file. Operationally it's the exact opposite:
* **Automatic.** Every save becomes an append-only log row. No staging, no commit, no push. The row's id is the SHA-256 of its own fields, so it's tamper-evident and deduplicates itself across peers.
* **Real-time.** Changes propagate to your other devices and agents in about a second.
* **Peer-to-peer.** No server owns the truth. You can run an always-on hub, but it's just a peer like any other.
* **Convergent without a human.** Every node folds the same log to the same bytes. Nobody resolves anything.
That last one is the part I care about most, and the mechanism is tiny. State is a *fold* over the event log β replay every row in one canonical order and you get the current bytes. Causal order comes first (a change only applies after its parent), and concurrent ties are broken by a Lamport clock (a counter that orders events without trusting anyone's wall clock), then site id, then the row's own hash. The whole thing is a textbook topological sort over a min-heap, about 55 lines. Everything else in the codebase exists to make sure that function gets byte-identical inputs on every machine.
Determinism looks like an implementation detail. It's the product. It's why there's no "conflicted copy (2)" file and why any node can do any job without a leader election. It's also why history stays trustworthy: you can roll the whole vault β or a single file β back to any moment, and restore is a query plus a blob lookup, not a replay of a diff chain.
And because I'm not crazy enough to ask anyone to abandon git: every vault also maintains a derived, read-only, stock-git-compatible history, with the same SHAs on every node. Your existing tools keep working.
## Conflict markers are for agents now
My favorite design decision in the whole system. There's one merge engine β a classic 3-way merge β but what happens on a genuine conflict depends on what kind of file it is:
* **Text** (markdown, notes) clean-resolves. You will never see `<<<<<<<` garbage in your notes. The later change wins the contested region, deterministically, and the losing version stays in history where you can get it back.
* **Code** surfaces the conflict, with byte-deterministic markers labeled `ASP:A` and `ASP:B`. Not "yours" and "mine" β there is no yours and mine. A and B are fold-order positions, identical on every machine, so even the conflict text itself can't cause divergence.
* **Binary** is last-writer-wins, whole file.
The reasoning, straight from the spec: *silently losing a function is far more dangerous than a dangling note, and agents are good at resolving conflicts.*
> Conflict markers used to be a summons for a human. Now they're an inbox for an agent.
A human scrubbing through their notes should never see merge plumbing. An agent editing code absolutely should β because it'll just fix it. The entire miserable UX of merge conflicts existed because a scarce resource had to be interrupted at exactly the right moment. That resource isn't scarce anymore.
## A git remote is just another peer
The newest piece: ASP can clone from β and two-way sync with β an ordinary git remote. GitHub, Gitea, self-hosted, whatever.
The mental model is one line: **a git remote is just another ASP peer.** Paste a git URL anywhere you'd paste an invite code β CLI, desktop app, browser. The repo's entire commit DAG imports as the vault's timeline. Upstream commits arrive as ordinary log rows, so a local edit racing an upstream commit merges through the exact same fold as everything else. No new machinery. Outbound, your edits roll up into real, readable git commits β `asp git push -m "tidy the docs"` β and GitHub sees a plain fast-forward.
The party trick: paste the same GitHub URL on two laptops that have never met. The vault's identity is a pure function of the repo's history β literally a hash of the root commit β so both machines author *byte-identical* rows and can immediately sync with each other, peer to peer.
One more agent-shaped detail: commit messages. The engine never calls a model β I don't want an LLM in my sync path. Instead it exposes two primitives, `asp git diff` and `asp git plan -m`, and an external agent drives them. Claude writes the message; the engine synthesizes the commit deterministically, so two bridge nodes still compute the same SHA.
## One engine, four surfaces
All of the protocol logic lives in one Rust crate, and every surface is a skin over it:
* the **`asp` CLI**, which pairs devices by printing a QR code in your terminal β point your phone at it
* **Context Desktop**, a Tauri app
* an **Obsidian plugin**, because my notes are context too
* the **browser**, where the same Rust engine runs compiled to wasm and computes byte-for-byte the same state as native β there's a conformance suite to prove it
Networking is [iroh](https://www.iroh.computer/): you dial a device by its ed25519 public key instead of an IP address, and there are no TLS certificates to configure. Direct hole-punched QUIC when the network allows, an encrypted relay when it doesn't β and the relay is a stateless forwarder that stores and sees nothing. Self-host it with one command if that's your thing.
## The part where the agents built it
The repo's first commit landed at 4:59 AM on June 7th. The entire commit message is the word "spec" β a 1,061-line design document, before any code existed. By 11 AM the same morning there was a working engine, a CLI, a wasm SDK, an Obsidian plugin, a desktop shell, and a commit that reads "e2e suite: 39 multi-process tests against the real asp binary, all green." v0.1.0 was tagged that day.
One month later: 260 commits, \~38,000 lines of Rust, 413 tests, 25 tagged releases. About half of those commits have Claude's fingerprints on them β some co-authored, seventy authored outright. The git trailers even distinguish which commits needed the 1M-context model. And yes: I coordinated a fleet of agents through git worktrees and branches to build the tool whose entire point is eliminating git worktrees and branches.
The repo reads like a workshop that agents share. AGENTS.md is less a style guide than notes taped to the wall of a machine shop: *never run two agents' builds concurrently, parallel rustc OOM-kills the VM. Don't run cargo fmt. The iroh e2e lane is flaky under load, even on a clean HEAD.* The agents' skills are versioned in the repo next to the product code, with their own commit type β `skill(sync-soak-test): ...` β and there's a commit that updates the verification playbook to record its own blind spot β the agents documenting their own failure modes for whoever picks up the next session.
Determinism also does something to a codebase I'd never experienced before: it turns trivia into permanent ABI. There's a function that decides whether a blob is binary. It's a UTF-8 check and a NUL scan β maybe five lines β and it has a FROZEN warning in all caps above it, because if two nodes ever classified the same blob differently, every vault in the world would fork. When every byte has to fold identically everywhere, a tiebreak you picked casually in week one becomes law.
## What it doesn't do (yet)
The honest list:
* **No CRDT for text (yet).** CRDTs are the data structures that let two people type in the same document and merge character-by-character. Considered and deferred, not rejected: line-level 3-way merge already lets different regions survive, and character-level merging mostly matters when two humans type in the same line in the same second β rare under \~1s sync. Agents also rewrite files wholesale, which is the pathological worst case for a CRDT, since it has to keep a ghost of every deleted character.
* **The website promises a self-building knowledge graph. The engine ships an empty embeddings table.** The substrate is there; the mind is not. v2.
* **The builds are unsigned.** Gatekeeper and SmartScreen will yell at you. That's a paid-certificate problem, not a malware problem β but you should be suspicious of everyone who says that.
* **A force-pushed upstream freezes the bridge** until you explicitly rebaseline. No automatic healing. I'd rather have a frozen bridge than an automatic guess.
* **Browser git clones route through a TLS-terminating proxy**, because git hosts don't send CORS headers and there's no way around it. Relayed ASP traffic is end-to-end encrypted; proxied git traffic is not. Run your own.
## Where this goes
In The Future of Agents I listed three big unstandardized gaps β memory, context, and agent infrastructure. ASP is my bet on the middle one.
Here's the frame I keep coming back to: version control was designed around the scarcest resource of its era β human attention at the moment of integration. Commits, staging areas, messages, PRs, conflict markers: all of it is machinery for capturing that attention at the right time. Agents flipped the economics. Attention is abundant now. What's scarce is *shared, current state* β and the tool for that shouldn't ask permission to record history. It should be ambient and boringly deterministic.
Git isn't going anywhere. It's load-bearing for the entire industry, and ASP literally speaks it. But I think it's becoming an endpoint rather than the workflow β the place clean history gets published, not the place work happens.
***
*The repo is [github.com/cjroth/asp](https://github.com/cjroth/asp). The spec came first, the agents wrote most of the code against it, and the vault this post lives in syncs itself while I type. It would have been embarrassing to write it any other way.*
# Building A Historically Accurate Simulation Of Rome
I've always dreamed of being able to go back to any place in history and see what it really looked like... and not just the stereotypical stuff and landmarks - I mean the weird in-between places that nobody thinks about. Some random backyard, an unsuspecting farm in the countryside. I want to know how people lived.
Turns out, one of the best use cases of AI is taking what is known and extrapolating the unknowns in between - strategic hallucination, essentially. So I decided to take Fable and see if it could build an actual realistic world for me in 3D. Despite wanting to avoid overdone landmarks, I started with Rome simply because there is so much data on it - I didn't want the research to be the hard part of a v1. Besides... who doesn't think Rome is cool?
My first prompt (excuse the mess...), was very basic and scatterbrained, but nothing that Fable can't handle:
> i want to generate a realistic 3d rendering that i can move around of ancient rome. pick a narrow particular time period. the goal is to make it *as realistic as possible* which means we need to find exact maps, blueprints, dimensions, etc. do /deep-research to find the best data here. i want every texture, every surface, every shadow, every piece of clothing, every door, window, every last detail to be hyper-realistic and based on actual history and science. for the parts that are unknown, we extrapolate from what we do know to fill in the gaps. we must be very careful not to let any influence from Hollywood ever seep in - we do NOT want dramatization ever. we want to systematize this approach into a new skill that we can use to generate a 3d point in time rendering of any ancient place. we want to add some animation / movement - people walking around, animals, trees, birds, weather, cloud, thunderstorms, etc - make it realistic and come to life. i should be able to move around like a character in a video game.
I walked away for a bit and came back to... no more Fable usage! It had spun up hundreds of research sub-agents and burned through all of my Max plan usage within 30 minutes or so. Ouch. Fortunately, the next day Anthropic reset the usage. This time, I was smarter. I added to my `AGENTS.md`:
> Delegate every task to the best model and set the optimal reasoning effort. Fable usage is limited - only use it when absolutely necessary.
I also told it to limit the workflows to 20 sub-agents max.
This time I left it running for the entire night. It was finished by morning and with a bit more prompting I was able to achieve this:
I still wasn't satisfied, so I asked Fable how I could make it look photorealistic like the best video games. It suggested I try Unity, so I did - this actually produced worse results. The lighting and atmosphere were better, but the buildings and people were far worse. I'm sure I could have gotten it up to speed with additional effort, but I already hated the fact that I had to make a Unity account and download a massive multi-gigabyte Unity installer and use a big clunky GUI. Coming from the land of open-source and Rust, I wanted to see if there was a good Rust option here.
Rust is the ultimate programming language for building with AI because of the extremely tight type and memory safety as well as the verification system with things like [verus](https://github.com/verus-lang/verus) and [kani](https://github.com/model-checking/kani). Recent models are trained well on Rust and this built-in tight verification layer means that if it compiles, it is virtually guaranteed to run and not crash. To my luck, I found [Bevy](https://bevy.org/) and decided to give it a shot.
Bevy ended up working incredibly well. It's super easy for the agent to work with it because it's just a regular Rust project with a Rust binary. The agent built itself a screenshot harness that lived inside the process so that it worked when my screen was off and was able to run a tight verification loop. It frequently compared screenshots to the research specs that the agents had done.
I was impressed that my Fable usage lasted so long once it delegated most things to Opus - probably around 2% usage per hour. At various points, I used `/btw` to ask it what it thought about its delegation choices and it repeatedly said that Opus was able to handle most of it, but that Opus sometimes got stuck or thought it was done prematurely in which case it jumped in and intervened. When I asked whether it would have made more sense to use Opus as the orchestrator and to phone Fable if it got stuck, it felt Opus wouldn't know when to intervene and tended to be overconfident... funny how managing AIs starts to feel similar to managing humans.
***
In the end, I learned a few things:
* Rome was full of graffiti
* Most Romans lived in insulae (4-6 floor apartment buildings) which were painted red and yellow
* Cheaper insulae were the upstairs units which had no kitchens or bathrooms, so tenants walked downstairs to cheap food stands
* There was no police force and people would probably have seen the idea of police as tyranny
* Rome was super unsafe, dirty, and loud
* Rome was basically just NYC
# Welcome to My Blog
Welcome to my blog! This is where I share my thoughts on software engineering, design, and building products that matter.
## Why Start a Blog?
Writing helps me crystallize my thoughts and share what I've learned with others. Whether it's a technical deep-dive, design philosophy, or lessons from building products, this is my space to explore ideas and connect with like-minded people.
## What to Expect
I'll be writing about:
* Software engineering best practices and architecture
* Product design and user experience
* Building and scaling startups
* Tools and workflows that make me more productive
* Lessons learned from shipping products
I've always dreamed of being able to go back to any place in history and see what it really looked like... and not just the stereotypical stuff and landmarks - I mean the weird in-between places that nobody thinks about. Some random backyard, an unsuspecting farm in the countryside. I want to know how people lived.
Turns out, one of the best use cases of AI is taking what is known and extrapolating the unknowns in between - strategic hallucination, essentially. So I decided to take Fable and see if it could build an actual realistic world for me in 3D. Despite wanting to avoid overdone landmarks, I started with Rome simply because there is so much data on it - I didn't want the research to be the hard part of a v1. Besides... who doesn't think Rome is cool?
My first prompt (excuse the mess...), was very basic and scatterbrained, but nothing that Fable can't handle:
> i want to generate a realistic 3d rendering that i can move around of ancient rome. pick a narrow particular time period. the goal is to make it *as realistic as possible* which means we need to find exact maps, blueprints, dimensions, etc. do /deep-research to find the best data here. i want every texture, every surface, every shadow, every piece of clothing, every door, window, every last detail to be hyper-realistic and based on actual history and science. for the parts that are unknown, we extrapolate from what we do know to fill in the gaps. we must be very careful not to let any influence from Hollywood ever seep in - we do NOT want dramatization ever. we want to systematize this approach into a new skill that we can use to generate a 3d point in time rendering of any ancient place. we want to add some animation / movement - people walking around, animals, trees, birds, weather, cloud, thunderstorms, etc - make it realistic and come to life. i should be able to move around like a character in a video game.
I walked away for a bit and came back to... no more Fable usage! It had spun up hundreds of research sub-agents and burned through all of my Max plan usage within 30 minutes or so. Ouch. Fortunately, the next day Anthropic reset the usage. This time, I was smarter. I added to my `AGENTS.md`:
> Delegate every task to the best model and set the optimal reasoning effort. Fable usage is limited - only use it when absolutely necessary.
I also told it to limit the workflows to 20 sub-agents max.
This time I left it running for the entire night. It was finished by morning and with a bit more prompting I was able to achieve this:
I still wasn't satisfied, so I asked Fable how I could make it look photorealistic like the best video games. It suggested I try Unity, so I did - this actually produced worse results. The lighting and atmosphere were better, but the buildings and people were far worse. I'm sure I could have gotten it up to speed with additional effort, but I already hated the fact that I had to make a Unity account and download a massive multi-gigabyte Unity installer and use a big clunky GUI. Coming from the land of open-source and Rust, I wanted to see if there was a good Rust option here.
Rust is the ultimate programming language for building with AI because of the extremely tight type and memory safety as well as the verification system with things like [verus](https://github.com/verus-lang/verus) and [kani](https://github.com/model-checking/kani). Recent models are trained well on Rust and this built-in tight verification layer means that if it compiles, it is virtually guaranteed to run and not crash. To my luck, I found [Bevy](https://bevy.org/) and decided to give it a shot.
Bevy ended up working incredibly well. It's super easy for the agent to work with it because it's just a regular Rust project with a Rust binary. The agent built itself a screenshot harness that lived inside the process so that it worked when my screen was off and was able to run a tight verification loop. It frequently compared screenshots to the research specs that the agents had done.
I was impressed that my Fable usage lasted so long once it delegated most things to Opus - probably around 2% usage per hour. At various points, I used `/btw` to ask it what it thought about its delegation choices and it repeatedly said that Opus was able to handle most of it, but that Opus sometimes got stuck or thought it was done prematurely in which case it jumped in and intervened. When I asked whether it would have made more sense to use Opus as the orchestrator and to phone Fable if it got stuck, it felt Opus wouldn't know when to intervene and tended to be overconfident... funny how managing AIs starts to feel similar to managing humans.
***
In the end, I learned a few things:
* Rome was full of graffiti
* Most Romans lived in insulae (4-6 floor apartment buildings) which were painted red and yellow
* Cheaper insulae were the upstairs units which had no kitchens or bathrooms, so tenants walked downstairs to cheap food stands
* There was no police force and people would probably have seen the idea of police as tyranny
* Rome was super unsafe, dirty, and loud
* Rome was basically just NYC
# Welcome to My Blog
Welcome to my blog! This is where I share my thoughts on software engineering, design, and building products that matter.
## Why Start a Blog?
Writing helps me crystallize my thoughts and share what I've learned with others. Whether it's a technical deep-dive, design philosophy, or lessons from building products, this is my space to explore ideas and connect with like-minded people.
## What to Expect
I'll be writing about:
* Software engineering best practices and architecture
* Product design and user experience
* Building and scaling startups
* Tools and workflows that make me more productive
* Lessons learned from shipping products