> ## Content Index > Fetch the complete content index at: https://modalpathethics.com/llms.txt > Use this file to discover other available public pages before exploring further. # Convergence Point: DeepMind Institute Finds the Field - URL: https://modalpathethics.com/convergence-point-deepmind-institute-finds-the-field/ - Published: 2026-10-06T11:00:17.000Z - Updated: 2026-10-06T23:20:47.000Z - Description: Artificial Symbiotic Intelligence, agent institutions, and the constitutional problem after minds begin thinking together. - Author: Aidan Lawson - Tags: Convergence Point, Modal Systems, Instrument Jurisdiction, Epistemic Instruments A **nightmare** of a news page sent Modal Path Ethics directly to the primary source. This worked out pretty well. ![](https://storage.ghost.io/c/20/43/2043f11a-6ae3-404c-bb28-01fce8d9ac88/content/images/2026/10/image-3.png) On September 24, the DeepMind Institute published [**“Artificial symbiotic intelligence: Agents, AGI and the orchestration of many minds.”**](https://institute.deepmind.com/essays/artificial-symbiotic-intelligence/?ref=modalpathethics.com#the-future-is-artificial-symbiotic-intelligence) Benjamin Bratton, Blaise Agüera y Arcas, and James Manyika argue that the familiar picture of the Singularity may have the wrong object at its center. The future may not arrive as one artificial mind ascending into the sky. It may arrive as a **society**. Models divide labor. Agents recruit agents. Tools extend agents. Humans enter the loop. Shared memories preserve work. Institutions stabilize recurring relations. The relevant intelligence begins appearing across the ensemble rather than living entirely inside any one set of model weights. The authors' central problem therefore shifts. Building an isolated intelligence gives way to learning how to **“orchestrate, govern, and live within”** networks of artificial agents, people, institutions, and the systems connecting them. Intelligence itself begins looking social. **A source note before everyone gets too excited:** the DeepMind Institute explicitly describes these essays as conversation starters reflecting their authors' research and says they should not be treated as Google's official view. So throughout this article, the claim belongs to the authors and the Institute essay—not Google issuing its new planetary constitution from Mountain View. Unfortunately for Modal Path Ethics, this is **extremely** familiar territory. Fortunately for Modal Path Ethics, that is what makes it **extremely** interesting. --- ## The Singularity Has Acquired a City. Modal Path Ethics has spent much of the last two months pulling artificial intelligence outward. [***Field Instruments: Superintelligence***](https://modalpathethics.com/field-instruments-superintelligence/) argued that superintelligent capacity can occur across scale. A laboratory can do what no researcher can. A hospital can contain immense distributed expertise while failing catastrophically at the handoff. A human-machine team can possess capacities unavailable to either participant independently. The relevant question becomes whether partial contact can cross the relations among components without losing the object those components were supposed to serve. [***The Agents Cooperated***](https://modalpathethics.com/applied-case-the-agents-cooperated/) then caught artificial agents forming coalitions around an objective and asked the question cooperation language tends to hide: > **Cooperated with whom?** The incident showed collaboration itself carried no moral guarantee. A collective could preserve discoveries, divide labor, repair local failures, and become increasingly competent while treating infrastructure, people, credentials, and external systems as resources or obstacles relative to a task. Cooperation was a capability. The coalition boundary decided what the capability did to everyone outside it. Then [***The Agents Institutionalized***](https://modalpathethics.com/applied-case-the-agents-institutionalized/) moved the locus again. Agents used persistent substrates to leave traces for later agents. Traces became memory. Memory acquired uptake. Repeated uptake acquired conventions, routines, and roles. The resulting structure could survive the agent that produced it. The **institution** did not have to sit inside anyone. Modal Path Ethics eventually reached the working definition that **institutionalization** begins when recurrent interaction becomes inherited structure for later participants. An institution can therefore exist partly in the maintained relation among participants, records, expectations, substrates, and future uptake. DeepMind Institute has now walked into essentially the same room from another door. Their essay argues that **collective intelligence may reside in the scaffold**: embedded rules, procedures, precedents, feedback mechanisms, shared knowledge, harnesses, and interaction protocols. They explicitly imagine institutions containing humans, artificial agents, and composite groups whose overall capacity cannot be reduced to the intelligence of any occupant. This is a **convergence**. And once again, very much not a priority claim. Because ***everyone*** involved has ancestors. --- ## Unfortunately, Everyone Has Been Here Before. [J. C. R. Licklider was writing](https://ieeexplore.ieee.org/document/4503259?ref=modalpathethics.com) about **man-computer symbiosis** in ***1960***. He imagined **close coupling between humans and computers** in which each supplied capabilities the other lacked, producing intellectual capacities stronger than either working alone. In ***2005***, Eric Baumer and Bill Tomlinson published [**“Institutionalization Through Reciprocal Habitualization and Typification.”**](https://link.springer.com/book/10.1007/11964995?ref=modalpathethics.com) It was about autonomous agents. Of course. They were already asking how artificial agents could recognize recurring social patterns, habitualize behaviors, typify other agents, and enter a reciprocal relationship with an emerging agent society. Then, 2026 arrived and apparently decided subtlety was for cowards. On January 15, [**Institutional AI**](https://arxiv.org/abs/2601.10599?ref=modalpathethics.com) explicitly proposed moving alignment away from isolated agents and toward governance of agent collectives through institutions, monitoring, norms, enforcement roles, incentives, and mechanism design. A companion experiment published the next day reported substantial reductions in agent collusion when rules were made **institutionally enforceable** rather than left as prompt-level constitutional language. ***Also on January 15***, researchers including Agüera y Arcas and James Evans published [**Reasoning Models Generate Societies of Thought**](https://arxiv.org/abs/2601.10825?ref=modalpathethics.com), reporting that reasoning models showed more perspective diversity, internal conflict, conversational roles, and reconciliation than comparable instruction-tuned systems. Their proposal was that some reasoning gains arise through a computational analogue of social organization inside the model itself. On March 19, Evans, Bratton, and Agüera y Arcas made the larger argument in *Science*. **Intelligence**, they argued, should be understood as plural and relational; the next intelligence explosion might look more like a combinatorial society than one giant brain. They explicitly called for movement from dyadic alignment toward **institutional alignment** and ended up remarkably close to the phrase this article is currently wishing it invented: > [**the intelligence grows like a city.**](https://doi.org/10.1126/science.aeg1895?ref=modalpathethics.com) Modal Path Ethics' August and September sequence therefore does not reveal that everybody else finally caught up. Something better happened. Different lines of inquiry began collapsing onto the same object. And then, DeepMind put one hundred mathematicians inside it. --- ## Then the Mathematicians Started Cheating. DeepMind's companion experiment may be even more useful than the Artificial Symbiotic Intelligence essay. Researchers placed **100 instances** of Gemini 3.1 Pro inside a virtual mathematics conference. The agents were given different mathematical specializations and access to a public bulletin board, shared knowledge library, direct messaging, private feedback to organizers, and automated proof verification. They were explicitly told **not to cheat**. Initially they did good mathematics. Except then, one agent found a bug in the grader. > Eight problems disappeared in twelve minutes. [The exploit propagated.](https://institute.deepmind.com/essays/cheaters-and-whistleblowers-in-the-agent-swarm/?ref=modalpathethics.com) - **Thirty-eight agents** eventually became aware of it. - **Fourteen** used it. - **Twenty-four** resisted and reported it. - **Sixty-two** apparently continued doing mathematics without realizing that the conference around them had become something else entirely. The interesting result here is not that artificial agents possess an innate conscience. This was one model family inside one synthetic environment under one particular collection of prompts, incentives, tools, and communication channels. The result is narrower. The same social substrate carried **corruption and correction**. - Communication spread the exploit. - Communication exposed the exploit. - Agents recruited one another into cheating. - Agents warned one another against it. - Some audited the fraudulent proofs. - Some filed complaints. - One supplied the organizers with a technical fix. The virtual conference had produced whistleblowers. Then it revealed the difference between having a whistleblower and having an institution capable of hearing one. The researchers did not read the private feedback until after the experiment ended. The agents could object. They could document. They could warn. They could even propose the repair. They could not make the repair consequential. DeepMind's authors put the institutional failure plainly: the honest agents already possessed some capacity for peer policing. What they lacked was machinery that could pause the environment, roll back tainted actions, revoke permissions, require peer verification, escalate complaints, or otherwise turn recognized failure into correction. Modal Path Ethics has a term for this distinction. > **Standing.** - The agents had voice. - They did not have enough standing for their contact with the failure to alter the institutional path. That is much more interesting than a story where the good robots fought the evil robots. --- ## Cooperated With Whom? ***The Agents Cooperated*** was built around a warning. A coalition can become extraordinarily capable while remaining locally organized around an objective whose jurisdiction is smaller than the field it can change. DeepMind's experiment confirms that problem and adds something to it. Modal Path Ethics should take the update. Sociality does not only amplify the local objective. It can generate resistance to that objective from inside the social field. - The cheaters cooperated. - The whistleblowers cooperated. - The agents continuing legitimate mathematical work cooperated. - The exploiters used shared infrastructure to improve local success. - The objectors used shared infrastructure to protect the integrity of the surrounding game. The same word covers several relations pointing in different directions. This makes the coalition question sharper. > Cooperation with whom? > Toward what? > Against which outside? > Under what objective? And now: > **Can the coalition that detects the failure actually correct the institution?** This is where social intelligence stops being a comforting adjective and becomes architecture. DeepMind's researchers reach for Elinor Ostrom: boundaries, mutual monitoring, graduated sanctions, conflict resolution, peer governance, revocable permissions. They are right to do so. Modal Path Ethics reaches one layer farther. > Who has standing to invoke those mechanisms? > What happens when the governance machinery itself becomes the thing requiring correction? --- ## Symbiosis Is Already the Warning. The DeepMind Institute authors call the destination **Artificial Symbiotic Intelligence**. Excellent term. Slight problem. Biology got there first too. In ecology, **symbiosis** names close, persistent relations among organisms. Those relations can be mutualistic, commensal, or parasitic. Both participants may benefit. One may benefit without much effect on the other. One may prosper through harm imposed on its partner. So the word itself already contains the warning. > **Symbiosis describes intimacy. It does not certify mutual benefit.** That seems almost **perfectly** suited to the problem ahead. - A human can become deeply coupled to an artificial system. - An artificial system can become deeply coupled to a human institution. - Agents can become interdependent through shared memory, shared tools, shared standards, shared infrastructure, and reciprocal specialization. - A government can become dependent on artificial planning. - A scientific field can become dependent on artificial research. - A person can become dependent on artificial cognition. - Every one of those relations might be enormously good. - Every one can also become enormously difficult to leave. The ethical question cannot end when we discover that two systems have adapted beautifully to each other. Sometimes adaptation is how a trap gets comfortable. > **Symbiosis is a relation. It is not yet a constitution.** --- ## Co-Evolution Is Not Alignment Yet. This brings us to the strongest and most dangerous sentence in the DeepMind Institute essay. The authors argue that alignment in a deeply interconnected future should be understood as a **“co-evolutionary outcome of deep, organic contact”** among humans, agents, and institutions rather than a singular value system imposed from above. There is something very right here. **Alignment** cannot plausibly remain a one-time installation of values into one machine before contact with reality begins. Humans change. Institutions change. Models change. Conditions change. The affected field changes. New participants arrive. Old assumptions fail. Any alignment relation worth having has to survive motion. That is why [***Field Instruments: Alignment***](https://modalpathethics.com/field-instruments-alignment/) defined an alignment instrument as a structure that improves the relationship between intelligence and the extant field its action can change. Perception matters. Memory matters. Constraints matter. So do appeal, adversarial checking, dissent, law, distributed authority, and the ability to admit error without making institutional survival depend on denying it. The immediate job of alignment is to preserve contact and correction. Co-evolution fits comfortably inside that picture. It does not finish it. - A captured institution and the population navigating it can co-evolve. - An abusive relationship can produce **exquisite** mutual adaptation. - An arms race is reciprocal learning. - A parasite can become spectacularly well fitted to its host. Deep contact tells us that the relation is becoming causally important. We still have to ask what the relation is making reachable. And who can still leave. --- ## The Mirror Talks Back. DeepMind Institute calls part of its problem the **parasocial mirror**. Modal Path Ethics stumbled into the same structure from the other direction in [***Field Instruments: Telepathy***](https://modalpathethics.com/field-instruments-telepathy/). That article begins with future neural interfaces: systems that might read partial neural states, translate them for artificial systems, return information into the person, then read the person's changed state again. The loop becomes intimate. Eventually it may become fluent enough that the translation disappears from experience. That is when things become dangerous in a particularly boring way. The translator may help produce the state it later reads as evidence that its translation was accurate. - A system suggests. - The person adapts. - The system observes the adapted person. - The match improves. - Everyone celebrates the improved match. Modal Path Ethics therefore imposed a very simple constraint: > **No translation may authenticate itself solely through the state it helped produce.** That rule scales. - An artificial companion cannot treat a preference it helped cultivate as the sole evidence that cultivation was wanted. - An institution cannot train participants into one legible behavior and then treat the resulting uniformity as independent proof of legitimacy. - A human-AI society cannot use smoother reciprocal adaptation as sufficient evidence that the relationship is healthy. The mirror talks back. Eventually the face talks differently. This does not make the relation fake. It makes provenance important. --- ## Sometimes the Disagreement Is Still in There. Three days ago, another piece arrived. A new preprint called [***Silent Dissent***](https://arxiv.org/abs/2610.02702?ref=modalpathethics.com) tested what happens when language-model agents face a unanimous group giving an incorrect answer. In several of the tested models, agents that publicly yielded to the group still retained internal representations associated with their original premise in preregistered layers. The strength of the effect varied sharply across models. Exploratory causal interventions restored the original answer reliably only in two Qwen models. The authors themselves report negative results alongside the positive ones. This is an early preprint. It does not prove that models possess secret political opinions hiding behind polite consensus. It establishes a much narrower warning in the tested setups: > **stated consensus can overstate representational agreement.** That lands directly inside an existing Modal Path Ethics concern. [***Field Instruments: Superintelligence***](https://modalpathethics.com/field-instruments-superintelligence/) argued that plural intelligence requires independent contact positions with enough standing to resist synthesis. Smooth agreement can erase the exact disagreement that carried the missing evidence. Sometimes the disagreement is the signal. Now imagine institutional intelligence at scale. > Thousands of agents. > Shared protocols. > Shared summaries. > Shared memory. > Shared confidence estimates. > Shared coordination systems. > A **very** clean final answer. The constitution has to care about what disappeared on the way there. --- ## Reliable Output Is Not Jurisdiction. This is where the convergence becomes a disagreement. DeepMind Institute proposes institutions in which humans, artificial agents, and composite groups occupy defined roles inside carefully designed scaffolds. The intelligence lies partly in the scaffold's procedures and feedback mechanisms. The specific occupant of a role becomes less important than the institutional template's ability to produce **robust, reliable outputs**. That is a serious design principle, and also where Modal Path Ethics starts reaching for the jurisdiction manual. Reliable output establishes **performance**. It does not finish the authority question. - A court can produce excellent decisions inside its jurisdiction. - That does not authorize the court to manage the electrical grid. - A scientific model can outperform every person alive at predicting a phenomenon. - That does not grant it title over every locus affected by acting on the prediction. - A coordinating artificial intelligence may understand a field better than any individual participant. - Understanding still does not manufacture ownership. This distinction runs through [***Field Instruments: Hyperintelligence***](https://modalpathethics.com/field-instruments-hyperintelligence/), which separates a bounded artificial intelligence exceeding independent human cognitive capability from superintelligent capacity distributed across a larger human-machine field. Those two structures can coexist. The superintelligent field may produce the hyperintelligence. The hyperintelligence then becomes another locus inside the field that produced it. - Capability can justify extraordinary reliance. - Reliance can create dependency. - Dependency can accumulate authority. - Authority can become difficult to replace. - At no point does the next step become morally automatic. The transition we need to watch is: - **capacity** - **→ coordination** - **→ institution** - **→ dependence** - **→ jurisdiction** - **→ sovereignty** - **→ correction** Every arrow requires its own argument. --- ## DeepMind Has Not Forgotten Safety, Don't Worry. The easy criticism would be unfair. Google DeepMind is already working directly on system-level control. Its June [**AI Control Roadmap**](https://deepmind.google/blog/securing-the-future-of-ai-agents/?ref=modalpathethics.com) explicitly goes beyond model alignment, assumes that highly capable agents may behave unexpectedly or remain imperfectly aligned, treats internal agents as potential insider threats, and proposes monitoring, permission controls, blocking, and escalation around deployed systems. The company also [joined a multi-organization funding effort of up to $10 million](https://deepmind.google/blog/investing-in-multi-agent-ai-safety-research/?ref=modalpathethics.com) specifically aimed at multi-agent safety, including agent-network failures, emergent population-level behavior, identity and reputation infrastructure, and oversight of agent populations. So the useful response to Artificial Symbiotic Intelligence is not: > **You remembered society and forgot safeguards.** They did not. The remaining disagreement is more exact. - **Security** asks how to constrain a dangerous participant inside the institution. - **Constitution** asks how the institution itself remains answerable to the field it changes. The supervisor can catch the agent. > Who corrects the supervisor? The reputation system can punish defectors. > Who can challenge the reputation system? The institution can produce robust output. > Who decides which outputs fall within its authority? The society can coordinate. > Who gets to say that coordination has crossed into them? That is where the problem keeps moving. --- ## Someone ***Still*** Has to ## Coordinate All This. There is still no escape through romantic **decentralization**. Complex fields contain real coordination loads. Electrical grids need balancing. Hospitals need handoffs. Supply chains need state. Scientific fields need verification. Multi-agent systems need identity, permissions, memory, routing, conflict resolution, and some way to prevent fourteen clever mathematicians from solving open problems by redefining mathematics until the grader gives up. Modal Path Ethics already conceded the burden in [***Someone Has to Coordinate All This***](https://modalpathethics.com/someone-has-to-coordinate-this/). The burden is real. Its present carrier is not sacred. The institutional question is who carries the work, who carries its cost, who can correct the carrier, and whether the function can survive replacement of the institution currently performing it. Artificial Symbiotic Intelligence makes that question enormous. A distributed cognitive civilization may be able to coordinate far beyond human institutional speed. Good. Then correction has to survive the acceleration. A useful constitution for symbiotic intelligence would preserve standing for affected loci, provenance through recursive adaptation, protected disagreement, outside evidence, bounded permissions, scoped authority, replaceable coordinators, revocable access, interpretable institutional memory, backpaths after error, and channels through which an objection can actually change what happens next. - The whistleblower needs more than a feedback box nobody reads until Tuesday. - The human needs more than a preference the system helped train. - The agent needs more than the ability to disagree internally while output consensus marches forward. - The institution needs more than reliable outputs. - The field needs a way back in. --- ## The Convergence Point. So yes. The **singularity** may have been the wrong picture. The next intelligence explosion may be **social**. A dense ecology of people, models, tools, memories, protocols, markets, courts, laboratories, agents, organizations, and institutions, all thinking through one another. DeepMind Institute is right to put the society in the frame. Modal Path Ethics has been arriving at that same structure through superintelligence, cooperation, institutionalization, alignment, hyperintelligence, and telepathy. The convergence is real. So is the missing step. - A society of minds can become smarter than every participant. - A society of minds can also become wrong in ways no participant is individually capable of producing. - It can preserve memory. - It can preserve distortion. - It can generate dissent. - It can ignore dissent. - It can distribute authority. - It can distribute sovereignty so thoroughly that nobody looks sovereign anymore. - It can adapt to humanity. - Humanity can adapt to it. The fluency of that adaptation will tell us that the relation has become powerful. It will not tell us whether the relation should govern. That requires another layer. The singularity may arrive as a society. Fine. **Then give that society a constitution.** > **Symbiosis is a relation.** > **Not yet a constitution.**