A researcher from Anthropic has stepped away from the artificial intelligence sector, driven by deep-seated worries about the trajectory of the industry. He believes that his former laboratory and its competitors are engaged in a frantic race to build AI systems that could ultimately escape human control, highlighting escalating safety concerns within leading AI organizations.

Jacob Coulson, a researcher who specialized in training new AI models on massive datasets, announced on Tuesday that he is leaving the company. He stated that he does not want to participate in the industry-wide push to create AI systems capable of self-iterating, fearing these systems could spiral out of control and pose a threat to humanity. The 27-year-old British researcher, who studied mathematics, noted that many of his peers now use terms like "decisive moment" and "endgame" to describe the momentum toward self-improving models.

Coulson remarked, "At the current pace, many extreme scenarios could become a reality, and things might spiral out of control by the end of next year." He added that safety trade-offs seem inevitable as various companies, including emerging Chinese AI firms, compete fiercely against one another.