AI firms lack full control over their systems, warns researcher

Jacob Coxon, who previously worked at the AI safety firm Anthropic, has cautioned that leading technology companies developing large language models and other AI systems cannot claim to fully govern their creations once deployed.

The warning underscores mounting concerns within both the technology industry and academic circles about the controllability and predictability of increasingly sophisticated AI systems. As companies race to develop more capable models, questions persist about whether safety measures and oversight mechanisms are keeping pace with technological advancement.

Coxon's comments reflect broader industry discussions about alignment—the challenge of ensuring AI systems behave in accordance with human values and intentions. Researchers and industry observers have flagged instances where AI models have demonstrated unexpected behaviors or capabilities not explicitly programmed by their developers.

The concerns carry implications for regulation and governance of AI development globally. Policymakers in various regions, including the European Union and parts of Asia, are developing frameworks to address AI safety, control, and accountability as the technology becomes more integrated into critical sectors.