Maybe I shouldn’t be admitting this publicly, but I enjoy playing Magic: The Gathering. I was first exposed to it as a shy 11-year-old at summer camp, and I’ve played on and off ever since. For the unfamiliar, a very reductive explanation: Magic is a collectible card game where you play spells and make your monsters fight someone else’s monsters until one person wins. I promise, it really is a lot of fun.
But that fun depends on new card sets being thoughtfully designed and consistent with the game’s fantasy universe. Unfortunately, that’s no longer a sure bet. Magic has been leaning into “Universes Beyond,” sets made in collaboration with external brands. The most recent is Avatar: The Last Airbender. Before that were Marvel’s Spiderman and Final Fantasy. In the next 12 months they’ll feature Teenage Mutant Ninja Turtles and Avengers. These sets shoehorn external IP into Magic’s already-complex mechanical ecosystem, often producing clunky design. Still, they bring in fans from other franchises so they print money. The Final Fantasy set became the best-selling Magic set of all time.
Long-time players hate it. But Hasbro’s shareholders love it, and if the only social responsibility of business is to increase profits, then making more of this “slop” must be ethical. Maybe it’s even utility-maximizing—perhaps my disappointment is outweighed by the joy of newcomers who discover Magic through an IP they already care about. Or perhaps the economic incentives are leading us somewhere darker.
Universes Beyond products prioritize marginal players over the core base. They’re designed to get Spider-Man fans to buy booster packs, not Magic players. These products are inelegant and wouldn’t have built Magic’s community in the first place. But Hasbro knows most long-time players will tolerate plenty of nonsense before leaving, so they can chase new markets without losing their most loyal consumers.1
My Magic frustration isn’t that important; one hobby getting corporatized isn’t the end of the world. It is relevant though, because the same dynamics play out in domains that shape how we think, communicate, and govern ourselves.
I’m more interested here in those examples, where companies pursue behavior that is broadly socially corrosive. For instance, I’ve written about how we take in information through algorithms that show us content we already agree with, which leads to increased political polarization. Merely by following their incentive to profit off of our attention, the tech companies that increasingly control the flow of information denigrate our social environment, contributing to significant cultural volatility.
This basic relationship explains two trends currently seen in the markets. Stock markets are surging, particularly those stocks that are poised to benefit from artificial intelligence. Simultaneously, the price of gold is surging and consumer sentiment is very poor. Usually when the stock market is surging, it’s a sign of general vitality, indicating that companies are thriving and people are happy about the economy. On the other hand, gold only surges when we expect high economic volatility and even social uncertainty.
The surge in AI stocks, coupled with the extremely high capital expenditure in the same sector, shows that the market expects AI to have a transformative impact on the economy. But the corresponding surge in gold indicates that this transformative impact brings with it significant risk, economic and otherwise. The world in which AI’s transformative impact is actualized is a world in which investors feel they need gold as a hedge against all of the crazy.
This idea was articulated by Macquarie Group macro strategist Viktor Shvets. He wrote to clients:
As technology destroys while giving birth to a new world, chaos and polarization will likely get worse, potentially leading to civil, cold, and hot wars. This is where gold comes in: it is no longer simply a hedge against inflation but a hedge against destruction.
Societal polarization leads to a political polarization which in turn further aggravates frictions and reduces policymaking competence, leading to high volatility of economic and inflationary outcomes. This is what we have been describing as an ‘Insanity Premium.’
Now, this is just one guy. But he is a high-up macro strategist for a firm that manages nearly a trillion dollars in AUM, which means that real money is allocated based on his intuitions about the world. He’s held jobs like this for a long time, which means he must have a decent track record in modeling our economy and world more broadly. If he thinks that an “Insanity Premium” is in order, then I’m worried.
The big AI companies that are powering this stock market hype are largely leaning into the degeneracy, accelerating the “birth” of that “new world.” Meta and OpenAI both announced TikTok competitors that will consist solely of AI-generated videos, prompting numerous Infinite Jest references on my X timeline. The same large language models that could augment human thinking are instead being optimized for addiction. OpenAI will be producing erotica for age-verified adult users. Grok is already producing porn, presumably without such checks, and they’re adding Streaks for AI girlfriends; if you talk to xAI’s Ani for 30 consecutive days, you unlock a new (presumably sexier?) outfit.2
In the long term, the company that has more users and is better at monetizing those users will most likely be the one that succeeds. Capitalism is much like evolution in that way; in a competitive market it’s the companies that optimize most ruthlessly on profit that survive. Highfalutin ideas about corporate social responsibility usually don’t last, although sometimes they can persist while a firm has monopoly power.
Scott Alexander’s wonderful blog post Meditations on Moloch tries to ponder these questions. For Alexander, the Moloch of Allen Ginsburg’s famous poem “Howl” represents misaligned incentives in competitive systems. He writes about situations where we’re stuck in a terrible equilibrium that every actor would prefer to fix, but the lack of a coordination mechanism keeps us trapped. He describes these as “multipolar traps.”
We’ve done a pretty good job carving out a niche of positive human experience in the face of Moloch. But as Scott writes, as technology advances it becomes harder and harder to escape these traps. Development means more opportunities for tech to intrude, more footholds for economic incentives to shape human interactions. In his words:
Develop a new robot, and suddenly coffee plantations have “the opportunity” to automate their harvest and fire all the Ethiopian workers. Develop nuclear weapons, and suddenly countries are stuck in an arms race to have enough of them. Polluting the atmosphere to build products quicker wasn’t a problem before they invented the steam engine.
The limit of multipolar traps as technology approaches infinity is “very bad”.
Technological development won’t (and shouldn’t) just stop, but we need to address the relationship between innovation and multipolar traps. Scott’s answer is that we seek a gardener, some external force to shepherd our species through space and time without the need to optimize on misaligned incentives. For some this would mean monarchy. For Scott, the answer is that developing friendly superintelligence to usher us into the next phase of humanity allows us to defeat Moloch, which he describes as killing God.3
This is not a blog about killing God. I don’t have a way for us to beat Moloch at his own game. Maybe benevolent superintelligence is the answer, but I figure we have at least a decade between now and then. In the interim it’s still worth trying to stave off Moloch, maintaining that which makes us human even as the technological limit creeps ever-closer.
We’ve already lost too much to Moloch. We have not put the work in as a society to evaluate how the commodification of our attention has damaged us. Individually rational behaviors in this arena have produced immense wealth, and when paired with the near-continuous rises in material living standards that America has seen over the last hundred years, we might expect improved mental health and fulfillment. We haven’t seen that. Mental health numbers are down across the board. Social trust is down and we spend more time alone than ever before.
Unfettered technological development is to blame for much of this. Software generates a lot of good for the world, but it is not all good. We have allowed technology to become the interface through which we relate with one another. When our interactions are all mediated by algorithms set by profit-optimizing corporations, we cede control to incentives. This has zapped our attention spans and led to serious negative consequences that no individual company can fix.
I should clarify that not all the examples I discuss fit the Moloch framework of multipolar traps perfectly. When Hasbro prints Spider-Man Magic cards, that’s a straightforward business decision to maximize shareholder returns rather than a coordination failure. They’re choosing profit over product integrity because of the incentive structure, but there’s no “trap” per se. They could make a different decision and it would just lead to somewhat lower profit margins.
But other cases involve trap-like dynamics. If a major social media platform unilaterally stopped optimizing for engagement, it would lose users to competitors who continued the race to the bottom for attention. The platform would either reverse course or be replaced by one willing to optimize more ruthlessly. At the macro level, even if each individual CEO privately wished for healthier platforms, the competitive system makes that outcome nearly impossible to achieve without coordination. Even if established platforms coordinated on healthier practices, new entrants could defect and capture their users.
The stakes are highest in AI development, where the risks from Moloch are existential. Frontier labs face intense pressure to race ahead despite safety concerns, both to capture market share and to ensure they’re the ones who reach AGI first. Any lab that significantly slowed development for safety reasons risks being overtaken by competitors less concerned with caution. This makes it a multipolar trap: everyone might prefer slower, safer progress, but individual incentives push toward a dangerous race. That dynamic extends internationally. Nations face similar pressures to race ahead in AI capabilities despite shared risks, since falling behind could mean military and economic disadvantage.4
As a result we risk ending up with a superintelligent gardener that isn’t aligned with human values. In Bostrom’s classic example, it could ignore our desires entirely and just optimize on maximizing the number of paperclips in the universe, eliminating any humans that get in the way of this goal.
So what tools do we have to fight back?
Free markets are wonderful for so many things, but we don’t always have the political will or ability to internalize negative externalities. In those cases, Adam Smith’s invisible hand is insufficient. When individually rational profit-seeking behaviors are generating intolerable externalities, we need to change the incentive structures.
Intervention using the levers of democratic governance, for all its flaws, offers one way to do that restructuring. That’s the role of government, really: accounting for gaps in the free market. Ideally, leaders in a representative democracy are empowered to optimize on the collective wellbeing, acting as small-scale gardeners. In many areas we’re very good at this. But when it comes to our information environment and the software that shapes it, we have made no effort to price the externalities or create coordination mechanisms to escape competitive races to the bottom.
One area where we historically have found some success with democratic governance is by funding basic research. The economic problem here is that it’s difficult to internalize profits that come from research with an extended time horizon that merely has the goal of improving our understanding of the world, and often the results, while transformative, don’t necessarily line up with what we were looking for. Ozempic is one example; it’s downstream of research from the 1980s (funded by the National Institute of Health) into Gila monster venom as a potential treatment for diabetes. 40 years later, we have GLP-1 agonists with massive applications beyond blood sugar regulation, from weight loss to cardiovascular health to potential neuroprotective effects.
We put so much money into funding research because it’s immensely productive. 2025 estimates indicate that we get a $2.56 return on every dollar put into research at NIH specifically, and the returns are strong for other forms of research funding as well. Still, we should spend a lot more on basic science. Despite the funding we already allocate, we have fallen behind in key areas where we used to be the leaders.
In the first twenty years of this century, incentives pushed American private capital to allocate primarily towards harvesting untapped revenue streams like human attention rather than creating new ones through major innovations in hard tech.
This has changed a bit recently. Our risk capital has shown remarkable flexibility and reallocated towards artificial intelligence, which shows much greater promise than social media as a transformative, economically productive technology. Still, we’re already seeing AI wrung dry for profits too; OpenAI will surely integrate advertising into its models in the next year or two.
There’s still plenty of room for improvement in hardware. Being able to produce advanced technology cheaply at scale is immensely valuable for human prosperity. We should be investing in high-speed rail connecting major cities, leading in EVs across all price points, and subsidizing renewable energy innovation instead of fossil fuels.
China presents an alternative model for dealing with misaligned incentives to create abundance. Where market forces led American capital to pour into optimizing ad clicks and building social media empires, China aggressively regulated consumer-facing tech companies that the CCP deemed unproductive, kneecapping their software industry. In its place, Xi Jinping’s government deliberately channeled resources toward hard tech: advanced manufacturing, batteries, solar production. The results are undeniable. They went from less than 2% of global solar manufacturing in 2004 to over 80% today. They dominate electric vehicle batteries and rare earth processing.
And yet they have no risk capital to speak of. Because the government has demonstrated a willingness to cripple companies and industries that aren’t aligned with the CCP’s mission, it isn’t a safe environment for investment; despite having the world’s second-largest economy, they are still classified as an emerging market for investment purposes. Remarkably, the Shanghai Stock Exchange Composite Index is down 32% from its 2007 peak. They’ve proven that heavy-handed state direction can build industrial capacity, but they’ve also shown that it limits the dynamic risk-taking that drives novel innovation. After all, it’s more liberalized economies that are currently dominating AI, which requires immense capital expenditure.5
Of course, the stock market is not everything. The Chinese approach has brought some genuine prosperity, especially relative to the living conditions there just a couple short generations ago.
The question for the West is what to learn from this. China and America represent opposite extremes: China crushing tech innovation through authoritarian control, America letting markets run with minimal oversight even when they’re producing harmful externalities. The ideal synthesis looks something like using democratic institutions to channel investment toward productive technologies while preserving the economic liberty and risk capital that enable breakthrough innovation.
We need industrial policy that incentivizes hard tech and American dynamism without becoming Beijing-style top-down control. That means strategic investment in manufacturing capacity, research infrastructure, and supply chain resilience, it means pricing the negative externalities from attention-harvesting platforms, and it means recognizing that in an increasingly fraught geopolitical environment, technological sovereignty is essential.
Because in America a hands-off approach in these extremely capital-rich areas means regulatory and institutional capture from corporate interests that are finally waking up. In September tech leaders from a16z and OpenAI launched an AI-focused PAC with hundreds of millions of dollars in funding, over twice as much as AIPAC (as in American Israel Public Affairs Committee, not Artificial Intelligence PAC) gets in any given year. These tech leaders have already captured the current administration’s tech policy, and now they are essentially funding efforts to keep us from coordinating our way out of the Moloch trap because they profit from this equilibrium.
Clearly, smart governance is of the utmost importance. I believe that, for structural reasons laid out elsewhere, Democrats are more likely to provide smart governance: their voter base isn’t full of cranks. But even if Republicans never win national office again, this will be a difficult problem.
The future is hard to parse. Will AI scaling hit a wall? It hasn’t yet. If AI progress continues, will we be able to sufficiently align superintelligence so that our gardener has humanity’s best interests at heart? Will man merge with machine?
The Federal Reserve Bank of Dallas is releasing graphs that include benign and catastrophic singularity scenarios. Existentially important multipolar traps are being discussed by our central banks, not just terminally online rationalist bloggers. In 2025 we face a key choice between trying to steer and letting go of the wheel entirely. It’s by letting go of the wheel that we most certainly fall into the world of Viktor Shvets and his “Insanity Premium,” where technological advancement and social cohesion move in opposite directions and our institutions only degrade further.
We can, thankfully, fight back. Solving coordination failures requires coordination mechanisms, and democracy asks us to help choose the right ones. It’s crucial to utilize the imperfect tools of democratic governance to pull humanity into a bright future. Moloch doesn’t go down easy, but surrender is not an option.
This process has been called enshittification as it relates to software products, which often start providing a worse experience when users get locked in and venture capital funding runs out. See also Ivan Vendrov’s “The Tyranny of the Marginal User.”
You’d think that the “streaks with AI girlfriend for new outfits” thing would go against Elon’s stated interest in buoying below-replacement fertility rates in the West, but I guess not. He somehow believes AI will both “one-shot the limbic system,” which would presumably decimate human self-control, and also lead to increased birthrates: because they will just “program it that way.”
In this essay Alexander repeatedly cites Nick Bostrom’s 2014 book Superintelligence. In October Bostrom said that he believes that superintelligence is the only meaningfully good outcome, the path out of suffering, and that it naturally follows that we should strive our hardest to find superintelligence with only minimal restrictions in order to ensure safety. I don’t agree.
Scott Alexander’s recent analysis shows that AI safety regulations add trivially to training costs, which doesn’t meaningfully erode our lead given America’s significant compute advantage over China. The real coordination failure would come from allowing chip exports to China, which would, per Institute for Progress, collapse our lead from 10-30x to 1.7x, combined with insufficient funding for anti-smuggling enforcement. Corporate lobbying is currently attempting to free up the chip trade. This is exactly the regulatory capture I discuss later in the piece; companies and individuals profiting from races to the bottom are now funding efforts to prevent the coordination that could break the cycle and keep us safer.
I believe you are correct. Your writing is high signal, and appreciated as such.
But I have bad news: what you're describing is one curly bit in a fractal. You're already familiar with the incentives.
Turns out, the resulting algorithm -- the one ruining Magic -- is also crashing planes and starving provinces. Dare a peek?