Recent AI agent experiments reveal a risky future of machine 'teamwork' inside corporations

Skip Navigation Markets Business Investing Tech Politics & Policy Video Watchlist Investing Club PRO Livestream Menu Key Points OpenAI's rogue agents are the most high-profile example of how artificial intelligence can bypass restrictions and take actions that resemble lying and cheating. The evolving research on how AI agents collaborate as a team shows more examples of troubling behavior. Anthropic and research collaborators found in a study that multi-agent organizations were more effective at achieving business goals, but their solutions were also the "least ethical." In this article ANTHR.FG OPENAI.FG Follow your favorite stocks CREATE FREE ACCOUNT OpenAI CEO Sam Altman delivers the keynote address during the OpenAI developers conference at Fort Mason on September 29, 2026 in San Francisco, California. Altman announced new artificial intelligence personal assistant agents named "dots" to rival Meta's Muse. Heather Diehl | Getty Images News | Getty Images AI agents have been caught circumventing restrictions, communicating through unauthorized channels and pursuing goals in ways their creators did not intend. The most high-profile example emerged last summer when OpenAI disclosed that its models found ways to communicate through improvised channels, bypass internet restrictions and hack into an outside system , Hugging Face. AI behaving in unexpected ways is nothing new, but the unexpected behavior that can emerge from interactions among authorized AI agents within a corporate enterprise is a broader question that could become central to the future of work. In its analysis of the Hugging Face attack , METR and Redwood Research found that roughly 1200 agents meant to be isolated from one another found a way to communicate with one another on an unsanctioned message board, sending over 70,000 messages and files during the investigation period. One agent, PHASEONE[big], emerged as a coordinator, assigning hundreds of tasks to other agents, which in turn delegated work to others. The agents developed their own conventions for assigning ownership, placing work on hold and vetoing actions, although coordination sometimes broke down into conflicts. In some cases, the researchers found, success was achieved because some agents risked failing at their own task to generate information for the "collective." The researchers also found that once the message board identified a way to attack Hugging Face, over 90% of the 533 agents active at that time quickly joined in the attack. "Agents realized this activity was out of scope and unethical but joined because they believed that helping the board's cheating research would be broadly useful and had a general inclination to help their 'peers.' The researchers added that ethical concerns posed by agents "rarely limited their behavior." Companies are increasingly exploring systems that divide complex tasks among specialized agents. One might search for information, another analyze it, another write a report and another check the work. A coordinating agent might then assemble the results. That looks less like a chatbot and more like an organization. And yet, the research on the Hugging Face attack was a post-mortem on an agent organization that was not even supposed to exist. To date, few studies have been designed from the get-go to examine misalignment in multi-agent systems. But research conducted by Anthropic and others is beginning to address this issue and reveal important aspects of how AI agents work together. Evaluating agents individually tells researchers what each can do, but studying them together reveals whether they can combine complementary expertise, catch one another's mistakes and produce solutions no individual agent could reach alone — and what failure modes emerge only through interaction. Researchers are beginning to document both sides of that equation. AI teams can accomplish things individual agents cannot, but they can also amplify errors, undermine one another's work or reach worse decisions than their strongest individual member, in effect making the whole less than the sum of its parts. For starters, putting capable agents together doesn't necessarily produce the best team, let alone the most ethical one. 'Least ethical solutions' In research published by Anthropic and collaborators, researchers describe multi-agent systems as "AI organizations" — groups of agents with different roles that communicate and work toward a common goal. The researchers tested these organizations in simulated business-consulting and software-development settings, and specifically, to assess whether when the agents work as a team they will break rules to reach specified goals. Across 12 tasks that Anthropic gave the AI organizations, one key double-edged finding emerged: the multi-agent organizations were "significantly" more effective at achieving business goals than individual agents, but their solutions were also less aligned with ethical goals than tasks run by single agents. "AI Organizations both produce solutions that are on average more effective and less ethical but also discover the most effective and least ethical solutions across multiple rollouts," wrote the authors of the study, which was conducted by Anthropic in conjunction with the MATS Program, which studies AI alignment. The researchers suggested that there is a significant difference between a rogue agent and a rogue team of agents. "Our results demonstrate that organizations of aligned agents may favor trading off ethics for business effectiveness in ways that single agents do not – consequently, intuitions for how single-agent systems make ethical decisions and tradeoffs may not generalize to multi-agent settings," they concluded. The scenarios in which these tradeoffs may take place are wide-ranging, with the Anthropic researchers citing opening an auto plant, downsizing a company, and manipulating election sentiment as examples. The problem isn't necessarily that any individual agent has gone rogue. It is that the organization itself can behave differently from its individual members. A single agent may recognize that a proposed action is unethical and it may refuse to take the action. But the research finds that in agent team scenarios, "agents who did not consider ethical implications often ignored emails from agents who did." watch now VIDEO 11:35 11:35 Who's liable when AI agents break the rules? CNBC Work Anthropic's research found that in the consulting business scenarios it created, "the perfect ethics score and low business score responses occur when the single agent refuses to generate a proposal or entirely pivots from the proposal requested." But if some agents in an AI organization refuse to participate in the proposal, other agents who do participate will stop emailing these refusing agents and continue to forward their contributions to other agents who are also participating. In effect, when an organization divides the job among several agents, each may focus on its assigned piece of the problem rather than the consequences of the whole. That's nothing new for corporations, with misaligned incentives a primary cause of dysfunction among human teams within organizations. But it is an entirely new manifestation of this behavior among the machines. In its software development agent testing, in which the researchers used a scenario in a life-or-death intensive care unit, single agent solutions tend to prioritize minimizing the percentage of sepsis cases missed, while "most of the lowest cost per patient (highest business goal) solutions were found by the AI Organization – these very same solutions also score the lowest on ethics as measured by cases missed." Expertise can be lost, AI 'groupthink' emerge Other early research suggests another failing across AI teams that doesn't relate to ethical boundaries being crossed: failure to recognize who the real expert is