Two letters from inside AI development caught my attention. One warns that humanity could lose control of the systems being built. The other asks what happens to the joy of human creation when those systems become better at the work we love.
Both confront an uncertain future. What interests me is the movement from that uncertainty to a response: what we ask people to fear, what we invite them to explore, and who receives the power to decide what happens next.
There is a gap between promising that AI will benefit humanity and putting the means to shape life into people’s hands. These letters brought that gap into focus.
Jacob Coxon, announcing his resignation from Anthropic, argued that the race toward self-improving AI was putting humanity in existential danger. He questioned the responsibility of both Anthropic and OpenAI and raised the possibility of “a temporary ban on improving model capabilities.” His appeal was also addressed to fellow researchers: should they continue simply because the race is already underway, or demand different conditions? His resignation post challenges the companies conducting that race.
Someone can work close to a technology, see reasons for alarm, and speak out in good faith. Coxon’s willingness to question his own industry deserves to be heard.
My concern begins with a further step: when a warning about danger becomes an argument for concentrating authority over the technology.
The possibility of catastrophe can create enormous pressure to accept whoever offers protection. Under that pressure, an essential question can disappear: what power are we handing over, to whom, and with what possibility of challenging their decisions?
Consider a safety requirement that only the wealthiest laboratories can afford to satisfy. Even if its purpose is protection, its effect could be to exclude independent researchers and smaller developers. Or consider an arrangement in which a few companies retain advanced capabilities while everyone else receives restricted access on terms those companies set. People may be promised the benefits of progress while losing the ability to participate in determining its direction.
These are consequences to examine in the actual design of a proposal. Coxon’s post does not establish that any particular regulation would produce them. A pause could also constrain the most powerful companies. The question is what the arrangement does: whose power it limits, whose power it preserves, and whether people outside those institutions retain a meaningful voice.
That is the first response to uncertainty I want to make visible: the possibility that, in seeking protection, we authorize a future increasingly decided on our behalf.
The second letter opens another question.
In an essay translated to English as “I Have to Bury My Talent in Yesterday,” DeepSeek engineer Liu Shengyu describes the prospect of AI surpassing him at the work he loves. He helped build the technology that he expects will overtake his own abilities. He continues partly because the work gives him immense pleasure, and partly because he believes the race will continue with or without him.
Liu expects that he may remain employable. What troubles him is the change from creating the code himself to directing AI agents that produce it. His expertise may still be useful, his livelihood secure, and his output greater, while the activity he loves disappears from his working life. In the translation included in the inquiry, he writes: “There are a few more gears in my hands, but a few less rhythms in my heart.”
That distinction matters. We can preserve someone’s income and still overlook what makes their life worth living. We can give someone more productive tools while narrowing the space in which they feel alive.
Liu then asks what happens to people more broadly. If students can obtain excellent results by asking AI to complete their assignments, will they develop the understanding needed to judge those results? He does not assume every existing skill must survive unchanged. He asks which skills can be relinquished and which remain essential to responsible work. More output, without the ability to understand it, could multiply mistakes as easily as achievements.
His concern then reaches the distribution of power. He imagines a society in which a small minority commands the strongest AI while everyone else receives weaker systems. To improve your position, you need access to the very capability your position prevents you from obtaining. Advantage becomes a condition for acquiring further advantage.
That is why his conclusion matters. He calls for the most advanced intelligence to be available to everyone, openly and affordably, and gives this as his reason for remaining at DeepSeek. His answer to the prospect of displacement is to widen people’s access to the means of acting. That will not automatically restore the work he loves. It could, however, help preserve people’s ability to discover what comes next for themselves.
This is the direction I support. Uncertainty becomes a reason to equip people to explore, create, and find their way. Their participation is part of how the future is discovered.
The contrast is sharp, but it does not divide the two writers neatly into fear and courage. Liu’s essay also contains fear and combative rhetoric. Coxon’s post also opens a space for questioning and refusal. Neither person can be reduced to one response, and a company’s commitment to openness must be judged by what it actually makes possible.
The distinction I care about runs through both letters, and through our responses to them: does uncertainty lead us to surrender more of our capacity to decide, or to strengthen that capacity in one another?
Here the practical gap becomes clear. A person who can afford powerful AI, adapt it to their needs, and use it independently has possibilities that someone dependent on limited access does not. If that difference grows, it can become a difference in who is able to learn, build, earn, and participate. Access to the means of advancement could increasingly depend on already being advantaged.
Calling a system open does not resolve this by itself. It must also be usable, affordable to run, and available under terms that allow meaningful independence. The test is what a person can actually do with it.
Yet access reaches only part of the question. Even with powerful AI in everyone’s hands, we could organize life around keeping up with the machine: producing more, responding faster, and accepting its suggestions before discovering what matters to us.
This is where Liu’s grief reaches beyond one profession. If a machine performs a task better, does that settle the value of a person’s engagement with it? Does it tell us what that person should care about, or what is worth bringing into the world?
For me, the question begins with the human freedom to say, “I don’t know,” without immediately replacing it with a ready-made answer. There is room there for a question to emerge before its purpose has been defined, before it has become a task for a machine to complete.
Human beings need room to discover the questions and purposes which then AI may help them explore. The quality of a life cannot be settled by how efficiently its tasks are performed.
Safety belongs within this vision. Both demonstrated harms and credible emerging risks require serious attention. The same scrutiny should reach the institutions entrusted with managing them. A claim to protect humanity does not, by itself, justify concentrating power over humanity’s choices.
I want to ask of each warning, policy, and promise:
Does it leave people better able to understand, question, create, and act? Can they challenge the terms under which the technology reaches them? Can they pursue something its providers did not anticipate?
A letter’s reception tells us something about those who adopt it. It cannot establish its author’s motives. But the choices made in response can be examined, and their consequences matter.
We do not know what the future will become. That uncertainty makes it all the more important that people retain the means to participate in its creation.
Who gets to shape the future should not be decided before the rest of us have had a chance to ask what is worth creating.
Amihai Loven
Amihai Loven
