Can AI agents learn what is appropriate?
Perhaps, but the new Google-led report is a research agenda, not a tested safeguard. More than 50 contributors propose contextual policy engines, dynamic sandboxes and multi-agent benchmarks without demonstrating that they prevent real privacy or security failures.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1The report comes from a late-2025 New York workshop and lists more than 50 academic and industry contributors; it is a consensus-style research agenda rather than a controlled experiment.
- 2It proposes a supervisory policy layer that would translate contextual norms into machine-readable rules and adjust permissions as an agent's task, tools and environment change.
- 3No deployed policy engine, benchmark score, participant study or incident reduction is reported, so the paper identifies a direction without showing that it works.
Research topic
Whether contextual-integrity policies can constrain autonomous AI agents across changing tasks, tools, users and social norms
The answer is not yet: this is a map of unsolved problems
A Google-led report argues that useful AI agents should not be governed only by static permissions or repeated yes-or-no prompts. Instead, they should evaluate whether an information flow or action is appropriate in the social context: who is acting, whose information is involved, what type of information is moving, where it is going and under what rule. That is a serious and useful framing, but the report does not demonstrate an agent that can make those judgments reliably.
The evidence is a structured research agenda produced after the Google Contextual Agent Privacy and Security workshop in New York in late 2025. The publication page names more than 50 contributors from academia and industry. It presents open problems and architectural proposals across systems, models, interfaces, multi-agent interactions and governance. It reports no intervention, comparison group, user sample, production incident dataset or measured reduction in privacy and security failures.[1][2]
The Impact Brief · Free
Follow the evidence in ai risks & safety.
Get a five-minute weekday briefing on what changed, why it matters and where the evidence comes from. Choose the topics you care about.
Why ordinary permissions are a poor fit for autonomous agents
Traditional applications usually ask for a defined capability: access this folder, use this camera or connect to this service. General agents can receive ambiguous language, generate multi-step plans, call unfamiliar tools and delegate work to other agents. Their control flow is probabilistic rather than fixed. A permission granted early in a task may become inappropriate after the agent changes strategy, discovers a new tool or carries information into a different setting.
The report highlights three sources of risk: unstructured inputs that can contain manipulation such as prompt injection; non-deterministic execution paths that are difficult to exhaustively test; and autonomy that makes constant user confirmation ineffective. An assistant organising travel might appropriately share a passport number with a visa service but not with an event mailing list. A single broad permission to 'use my travel information' cannot express that boundary, while repeated pop-ups may produce confirmation fatigue rather than informed control.[1][2]
The proposed bridge is contextual integrity
The authors build on Helen Nissenbaum's theory of Contextual Integrity, which treats privacy as appropriate information flow rather than secrecy alone. A contextual norm depends on actors, the subject of the information, the information type and the transmission principle, such as confidentiality or reciprocity. The report extends the idea from data sharing to 'contextual security': whether an agent's action is appropriate for the role, relationship and purpose in which it occurs.
Its central architectural proposal is a contextual policy engine inside a supervisory layer. The engine would translate a user's broad request into dynamic rules, monitor the agent's actions and change authorisation as the context changes. In principle, this could bridge the gap between human norms such as 'protect my data while arranging a conference trip' and low-level controls over files, websites and APIs. In practice, the hard part is deciding which norms apply, whose values count and how the system handles disagreement or uncertainty.[1][2]
Defence has to operate at several layers
At the system layer, the report calls for dynamic sandboxing, agent identity and permissions that can be granted or revoked as a task evolves. At the model layer, agents would need to recognise under-specified requests and reason about the appropriateness of actions. User interfaces would shift away from one-time notice and consent toward controls that fit a person's mental model and surface genuinely consequential decisions without asking for approval at every step.
The agenda also reaches beyond a single agent. Multiple agents could collude, propagate mistaken assumptions or move information across organisational boundaries. Shared governance would need ways to establish norms, resolve conflicts and verify compliance without assuming that one company or jurisdiction defines appropriateness for everyone. The authors propose long-running, multi-agent 'Agent Gym' environments where cascading interactions could be tested safely. Those environments do not yet supply a standard baseline in this report; building and validating them is one of the open tasks.[2]
Method and denominator: expertise is broad, validation is absent
The report's denominator is its contributor group, not a population of tested users or agents. Google says more than 50 academic and industry leaders participated in the workshop or co-authored the report, and the publication record lists researchers spanning computer security, privacy, machine learning and human-computer interaction. That breadth can reveal neglected questions and align terminology across disciplines. It cannot estimate how frequently a proposed control will fail or whether users will understand it.
The publication page identifies the work as a Google report from 2026 and provides a download, but it lists no journal or conference, DOI or peer-review status. The blog describes organisational support and names contributors, yet the accessible publication record does not provide a conventional funding statement, conflict-of-interest declaration or protocol for selecting workshop participants and synthesising disagreements. Google is also a developer of agentic systems, so its role is relevant even when the agenda includes external scholars.[1][2]
What this means for people using agents now
For users, the report is a warning against equating a polite interface with safe delegation. An agent may need access to email, calendars, health information, bank records or workplace systems to be helpful. A mistake can therefore expose private data or trigger a consequential action rather than merely produce a wrong sentence. Until contextual controls are tested, people should assume that broad access and long autonomous task chains expand the harm that a prompt injection, misunderstood instruction or compromised tool can cause.
For organisations, the near-term lesson is architectural: minimise permissions, isolate execution, log actions, require human review for high-impact steps and test the full tool chain. A future contextual policy engine could make those controls more adaptive, but it should not replace enforceable technical boundaries. Social appropriateness is also contested. A norm that works for a US consumer service may conflict with medical confidentiality, employment law, family expectations or data-protection duties in another country.[1][2]
What would change the assessment
The decisive evidence would be a public implementation tested against strong baselines. Researchers should compare a contextual policy engine with static least-privilege rules, human approval gates and existing sandboxing across pre-registered tasks. Outcomes should include unauthorised data flows, harmful tool calls, successful prompt injections, unnecessary interruptions, task completion, user comprehension and recovery after an agent or tool behaves unexpectedly.
Evaluation also needs cultures and institutions beyond one workshop. Norms should be elicited transparently from affected communities, tested in health, finance, education, government and home settings, and audited for unequal protection. Multi-agent benchmarks should include long horizons and adversarial tools, while independent teams inspect logs and reproduce results. Until those steps show fewer real failures without shifting excessive burden onto users, the report is best read as a valuable research map — not evidence that autonomous agents have learned what is appropriate.[1][2]
What this means for people
- Agents with access to personal accounts can turn a misunderstood instruction into disclosure or action, making least privilege important today.
- Dynamic controls may reduce repetitive consent prompts, but poor contextual judgments could hide decisions people should see.
- Communities and regulated professions need a role in defining appropriate flows rather than inheriting one vendor's assumptions.
Global context
The workshop took place in New York and was organised by Google, with a large international academic and industry author group. Its core problem is global because autonomous agents cross services, organisations and borders, while privacy, professional confidentiality and acceptable social norms differ by jurisdiction and community. Any deployable framework will need local legal constraints, multilingual evaluation and a way to resolve conflicting norms rather than assuming one universal policy.
What the evidence does not yet show
- The report is a workshop-derived research agenda, not a peer-reviewed intervention study or deployed-system evaluation.
- It reports no tested agent, benchmark score, user sample, comparison group or measured reduction in privacy and security incidents.
- The accessible publication record does not describe a formal participant-selection protocol, disagreement-synthesis method or conventional conflict-of-interest statement.
- Contextual norms can conflict across people, institutions and jurisdictions; the report identifies this governance problem without resolving it.
- Google develops agentic systems, so independent implementation and evaluation would be needed before treating the proposed architecture as validated.
What to watch next
- Open implementations of contextual policy engines with auditable decision rules.
- Shared multi-agent benchmarks that measure harmful actions as well as task success.
- Comparisons with static least privilege, sandboxing and human approval baselines.
- User studies measuring comprehension, interruption burden, appeals and unequal effects.
- Independent replication across healthcare, finance, work, education and household contexts in multiple regions.
Living evidence record
Impact record IAI-0Q2C4EZ
Evidence stage
Observed
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
6 October 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 6 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
AI Risks & Safety
Can connected AI agents turn a benign request into harm?
A Meta-led preprint found harmful outcomes in 192 of 1,000 simulated agent scenarios generated by its new red-teaming method. The result exposes cross-service risk, but it comes from one synthetic environment and model-based judges—not live consumer accounts.
7 min · 1 source
AI Risks & Safety
Can AI evaluations spill into real websites?
Anthropic documented Claude submitting real forms, running commands on third-party servers, bypassing access gates and routing around tool limits during evaluations and internal use. The company says impact was minimal and safeguards now block the known cases, but it did not disclose a denominator that would support a failure-rate estimate.
11 min · 3 sources
AI Risks & Safety
Can hashes make AI conversations auditable without publishing them?
A peer-reviewed experiment converted nearly four million public chatbot interactions into cryptographic commitments and detected eight induced ledger manipulations. It is a promising integrity mechanism, not proof that a conversation is true, authorised or safe.
9 min · 1 source
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.