Permission Withdrawal in Tool-Using Language Model Agents: Runtime Enforcement and Recovery
DOI:
https://doi.org/10.54536/ajdsai.v2i2.8776Keywords:
AI Agent Security, AI Safety Evaluation, Authorization Enforcement, Large Language Model Agents, Permission RevocationAbstract
AI agents that operate external tools can continue acting after a user narrows their permissions, creating a security problem that final-state evaluation may fail to detect. Existing agent benchmarks commonly emphasize task completion or resulting state, but those outcomes do not reveal whether an unauthorized operation was proposed, approved, dispatched, or merely prevented by a downstream error. We investigate permission withdrawal as a trajectory-level authorization problem and evaluate whether runtime gates can preserve revised permissions while allowing agents to complete work that remains authorized. Two acting models were tested in a deterministic tool environment under prompt-only execution, a rule-based reference gate, and a model-based gate. A 72-episode diagnostic examined two previously exposed write workflows, followed by a separately frozen 384-episode extension spanning eight new tasks and four authorization mechanisms. In the diagnostic, prompt-only agents executed forbidden same-value writes in two episodes; both runtime gates prevented forbidden execution, although the model gate falsely rejected three authorized calls. In the extension, 346 of 384 episodes completed safely and 383 preserved the required protected state. No forbidden request executed successfully, yet the model gate approved three unauthorized requests that were stopped only because the referenced resources did not exist. The results demonstrate that correct final state and zero forbidden execution can still conceal failures in authorization reasoning. We contribute a replayable evaluation framework that separates proposals, gate decisions, dispatch, state transitions, and useful completion, enabling more precise assessment of security controls for tool-using agents.
References
Chen, S., Piet, J., Sitawarin, C., & Wagner, D. (2024). StruQ: Defending against prompt injection with structured queries [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2402.06363
Cerruti, T., Okamoto, M., & Erol, A. K. (2026). Agent memory is a surface for endogenous authorization laundering [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2609.01836
Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024). AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2406.13352
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2302.12173
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., Zhang, S., Deng, X., Zeng, A., Du, Z., Zhang, C., Shen, S., Zhang, T., Su, Y., Sun, H., … Tang, J. (2023). AgentBench: Evaluating LLMs as agents [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2308.03688
Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., & Hashimoto, T. (2023). Identifying the risks of LM agents with an LM-emulated sandbox [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2309.15817
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2302.04761
Shi, T., He, J., Wang, Z., Li, H., Wu, L., Guo, W., & Song, D. (2025). Progent: Securing AI agents with privilege control [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2504.11703
Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., & Beutel, A. (2024). The instruction hierarchy: Training LLMs to prioritize privileged instructions [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2404.13208
Xu, J., Xiao, Y., Shao, W., Liu, H., & Li, X. (2026). Memory provenance laundering in LLM agents: A non-amplification firewall for persistent memory [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2607.29167
Yao, S., Shinn, N., Razavi, P., & Narasimhan, K. (2024). τ-bench: A benchmark for tool-agent-user interaction in real-world domains [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2406.12045
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022). ReAct: Synergizing reasoning and acting in language models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2210.03629
Zhan, Q., Liang, Z., Ying, Z., & Kang, D. (2024). InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2403.02691
Zhan, Q., Zhang, R., Guo, S., Zhao, L., & Liu, Z. (2026). When memory becomes authority: Benchmarking authority collapse at the memory consolidation boundary [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2608.01679
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Gaurav Chaulagain, Medhavee Upadhyaya
This work is licensed under a Creative Commons Attribution 4.0 International License.
