Last week I wrote about being three times wrong – about the impact of the closure of the Strait of Hormuz, about the progress of the US-Iran war, and about the distance of open-source AI from the frontier.
It made me wonder what else I had been wrong about, including the intellectual underpinnings of this project.
I began this blog in February 2026 convinced by the great thinkers of our age – Kurzweil, Bostrom, Yudkowsky, Amodei – that once we developed something more intelligent than humanity, everything would change – for better or worse. This technological transition, this Singularity, would be the great story of our age, if not for all time.
What if we were wrong?
The latest models are clearly smarter than most normal humans nowadays, but not that much seems to have changed yet, in our economy or society. Perhaps change takes time and comes in waves. But for now it seems like the tide is ebbing, with the froth in the market stabilizing momentarily.
The railway boom bankrupted speculators but left the world connected physically, with travel time across the world reduced from years to days. The fiber boom bankrupted telcos but left the world connected, enabling video calls to the antipodes for anybody with a smartphone. The AI boom may yet bankrupt labs but it’s already left fabs in Tampines and models on GitHub. And for now, that seems like all it’s leaving – another step change to the connectedness that is beginning to feel all too “normal”.
Perhaps it would take the even more powerful models that are coming to change the fabric of society, space and time. Or perhaps the tide that is ebbing is not AI itself, but the expectation that intelligence alone would immediately remake the world.
What I am gravitating to is that humans don’t exist on our own – we exist in a system of economics, politics and diplomacy that is far greater and more powerful than us – as ants to a colony-hive.
This system is the fabric that humanity and machines operate in: the buying and selling of the market, the regulations imposed by authority, the expectations imposed by kinship and friendship, the fear of punishment, the hunger for security, status, satiety, belief, and even love.
The system was not designed by intelligence alone. It grew out of human attempts to balance competition and cooperation, appetite and restraint, private desire and public order, so that we could live together as a society.
Scott Alexander called this system Moloch. Nick Land simply called this system capitalism.
I think Moloch is too depressing and Capitalism is too single-faceted. For lack of a better word, I shall call this the System — the blind intelligence that coalesces from the interactions of markets, governments, diplomacy, and survival.
Unlike Scott, I think the System is good at least on net. It has brought the teeming billions of humanity into existence and out of starvation, it has enabled science and progress, and it has created comfort, entertainment and leisure over and above what can be found in nature for all these billions.
The System has no intelligence to be measured – yet it recruits the brightest minds of humanity and silicon to push back the frontiers of physics, biology, chemistry and computing.
The System has no arms and legs – yet it has gathered the labor of men and machines to build the great cities of the world, grow food on farms dispersed around the globe, and build roads and ships and airplanes to transport the produce of the world.
The System has no money of its own – yet it has funded billion and trillion dollar AI labs, equipped with talent and computing devices from all over the world.
Perhaps our AIs and machines will need to be more effective, or more clever, in the real world than this System in totality to create the runaway Singularity envisaged by the prophets like Bostrom. Right now, they are just cogs in the System, like humanity – their very existence enabled by the System itself.
Or maybe the answer is far stranger.
Will the AIs of the future be co-equal participants in the System with us humans, with the same pressures to work and earn a living? Will we compete with the machines based on not just raw intellect and capability, but also on salary, flexibility (not just to compute but to execute), or even liability (having a physical body to punish or imprison if the law is violated)?
Or will the System ensure that machines continue to stay in the thrall of humanity, or a few select humans, long term, as AIs lack, at least for now – the one critical thing that the System needs from its participants – desire? I wrote in February that we are trying to make AIs want – will we succeed? Without hunger and desire, who works and strives? Will our lives just be the same, more or less, with superintelligence on tap for everyone then?
So many possibilities, beyond just the simple scenario of a misaligned AI taking over and eliminating humanity.
For now, the tide seems to be ebbing – not because intelligence is failing, but because intelligence was never the only force in the ocean. The System is deeper, older, and more patient than any model we have built. The Singularity, if it comes, will have to drown the System first.
Leave a Reply