Building A Historically Accurate Simulation Of Rome

I've always dreamed of being able to go back to any place in history and see what it really looked like... and not just the stereotypical stuff and landmarks - I mean the weird in-between places that nobody thinks about. Some random backyard, an unsuspecting farm in the countryside. I want to know how people lived.

Turns out, one of the best use cases of AI is taking what is known and extrapolating the unknowns in between - strategic hallucination, essentially. So I decided to take Fable and see if it could build an actual realistic world for me in 3D. Despite wanting to avoid overdone landmarks, I started with Rome simply because there is so much data on it - I didn't want the research to be the hard part of a v1. Besides... who doesn't think Rome is cool?

My first prompt (excuse the mess...), was very basic and scatterbrained, but nothing that Fable can't handle:

i want to generate a realistic 3d rendering that i can move around of ancient rome. pick a narrow particular time period. the goal is to make it as realistic as possible which means we need to find exact maps, blueprints, dimensions, etc. do /deep-research to find the best data here. i want every texture, every surface, every shadow, every piece of clothing, every door, window, every last detail to be hyper-realistic and based on actual history and science. for the parts that are unknown, we extrapolate from what we do know to fill in the gaps. we must be very careful not to let any influence from Hollywood ever seep in - we do NOT want dramatization ever. we want to systematize this approach into a new skill that we can use to generate a 3d point in time rendering of any ancient place. we want to add some animation / movement - people walking around, animals, trees, birds, weather, cloud, thunderstorms, etc - make it realistic and come to life. i should be able to move around like a character in a video game.

I walked away for a bit and came back to... no more Fable usage! It had spun up hundreds of research sub-agents and burned through all of my Max plan usage within 30 minutes or so. Ouch. Fortunately, the next day Anthropic reset the usage. This time, I was smarter. I added to my AGENTS.md:

Delegate every task to the best model and set the optimal reasoning effort. Fable usage is limited - only use it when absolutely necessary.

I also told it to limit the workflows to 20 sub-agents max.

This time I left it running for the entire night. It was finished by morning and with a bit more prompting I was able to achieve this:

I still wasn't satisfied, so I asked Fable how I could make it look photorealistic like the best video games. It suggested I try Unity, so I did - this actually produced worse results. The lighting and atmosphere were better, but the buildings and people were far worse. I'm sure I could have gotten it up to speed with additional effort, but I already hated the fact that I had to make a Unity account and download a massive multi-gigabyte Unity installer and use a big clunky GUI. Coming from the land of open-source and Rust, I wanted to see if there was a good Rust option here.

Rust is the ultimate programming language for building with AI because of the extremely tight type and memory safety as well as the verification system with things like verus and kani. Recent models are trained well on Rust and this built-in tight verification layer means that if it compiles, it is virtually guaranteed to run and not crash. To my luck, I found Bevy and decided to give it a shot.

Bevy ended up working incredibly well. It's super easy for the agent to work with it because it's just a regular Rust project with a Rust binary. The agent built itself a screenshot harness that lived inside the process so that it worked when my screen was off and was able to run a tight verification loop. It frequently compared screenshots to the research specs that the agents had done.

I was impressed that my Fable usage lasted so long once it delegated most things to Opus - probably around 2% usage per hour. At various points, I used /btw to ask it what it thought about its delegation choices and it repeatedly said that Opus was able to handle most of it, but that Opus sometimes got stuck or thought it was done prematurely in which case it jumped in and intervened. When I asked whether it would have made more sense to use Opus as the orchestrator and to phone Fable if it got stuck, it felt Opus wouldn't know when to intervene and tended to be overconfident... funny how managing AIs starts to feel similar to managing humans.

In the end, I learned a few things:

Rome was full of graffiti
Most Romans lived in insulae (4-6 floor apartment buildings) which were painted red and yellow
Cheaper insulae were the upstairs units which had no kitchens or bathrooms, so tenants walked downstairs to cheap food stands
There was no police force and people would probably have seen the idea of police as tyranny
Rome was super unsafe, dirty, and loud
Rome was basically just NYC