Comment by 1wd

I've done a lot of advanced research in this domain. It is far more difficult than people expect for a few reasons.

The biggest issue is that the basic data model for population behavior is a sparse metastable graph with many non-linearities. How to even represent these types of data models at scale is a set of open problem in computer science. Using existing "big data" platforms is completely intractable, they are incapable of expressing what is needed. These data models also tend to be quite large, 10s of PB at a bare minimum.

You cannot use population aggregates like census data. Doing so produces poor models that don't ground truth in practice for reasons that are generally understood. It requires having distinct behavioral models of every entity in the simulation i.e. a basic behavioral profile of every person. It is very difficult to get entity data sufficient to produce a usable model. Think privileged telemetry from mobile carrier backbones at country scales (which is a lot of data -- this can get into petabytes per day for large countries).

Current AI tech is famously bad at these types of problems. There is an entire set of open problems here around machine learning and analytic algorithms that you would need to research and develop. There is negligible literature around it. You can't just throw tensorflow or LLMs at the problem.

This is all doable in principle, it is just extremely difficult technically. I will say that if you can demonstrably address all of the practical and theoretical computer science problems at scale, gaining access to the required data becomes much less of a problem.

I’m also super interested in this kind of question. The late Soviet Union and their cybernetics research were really into simulating this kind of stuff to improve the planned economy. But I’m curious if something like this can be done on a more local scale, to improve things like a single company output.

You might find early agent-based models (e.g. the Sante Fe Institute's Artificial Stock Market[0]) interesting.

IMO the short answer is that such models can be made to generate realistic trajectories, but calibrating the model the specific trajectory of reality we inhabit requires knowledge of the current state of the world bordering on omniscience.

[0]: https://www.santafe.edu/research/results/working-papers/asse...

Agent based modeling (ABM) is an attempt at this. I've wanted to forecast the economy on a per-person basis since playing Sim City as a kid (although Sim City is not an ABM to be clear). From doing a bit of research a while back it seemed like the research and real world forecasting have been done on a pretty small scale and nothing as grand as I'd hoped. It's been a while since I've looked into so I would be happy to be corrected.

Doyne Farmer's group at Oxford does 'agent-based' economics simulations in this vein. He has a new book called 'Making Sense of Chaos' that describes it.