About the Workshop
Add your questions for Discussion →Language model security remains fundamentally misunderstood. While researchers have catalogued countless adversarial attacks and proposed numerous defenses, we've barely scratched the surface of why these vulnerabilities exist. The mathematical foundations that enable them, the internal mechanisms that process malicious inputs, and the gap between our benchmarks and actual security threats remain opaque.
This workshop will bring together researchers to share research and discuss the root causes of model vulnerability and how we might design secure and robust architectures from first principles.
Emphasizing foundational understanding over incremental improvements, we ask:
- What mathematical and computational properties make language models inherently vulnerable?
- How can interpretability inform our view of attack surfaces and defense mechanisms?
- Which evaluation frameworks can bridge the gap between benchtop metrics and real-world security failures?
- What blind spots persist in our current research programs and conceptual frameworks?
Our goal is to catalyze rigorous, cross-disciplinary discussion that advances the theoretical, empirical, and evaluative foundations of language model security.
Workshop Format
The workshop consists of four thematic blocks. Each block includes an expert keynote (45 minutes), two contributed talks (15 minutes), and an extended guided discussion (30 minutes) among participants, presenters, and domain experts. Our format prioritizes deep engagement and discussion over talk density.
Trade-offs in System-level defences against Prompt Injection
AI Security in Industry
Securing Real-World AI Agents
Securing AI Agents with Information Flow Control
Invited Speakers
In this talk I will discuss what we currently know about prompt injection defences today, followed by what the next generation of defences is likely to look like.
LLMs are widely used, in both industry and by end users. Yet, on both ends of these spectrums, there are very different challenges, which we will discuss in this talk. On the one hand, companies must ensure the availability, confidentiality, and integrity of systems - on a level that supports both governance and audit-ability. On the other hand, end-users face different dilemmas when interacting with LLM. They are often unaware of pitfalls and engage, potentially unaware, in security-relevant behavior. Lastly, we briefly discuss AI security incident reporting as a possible solution to the open questions raised in both areas.
AI agents introduce a paradigm shift in software security, moving from predictable, deterministic systems to components with non-deterministic behavior. This presents critical challenges, especially when agents are given autonomy and access to sensitive real-world computer systems (such as coding IDEs). This talk provides insight into Snyk's security research, detailing ongoing work to both exploit and secure real-world agents. Based on extensive red teaming against Model Context Protocol (MCP) systems, we illustrate key vulnerabilities and present a framework for effectively protecting against sophisticated attacks.
Indirect prompt injection attacks are the result of information flow security violations: we knowingly allow untrusted data to taint the context of an agent without constraining its actions. In this talk, I will describe how information flow control offers a robust, deterministic defense against prompt injection attacks. In a nutshell, by attaching confidentiality and integrity labels to data ingested by an agent and propagating these labels as the data is processed, information flow analysis can determine when confidential or untrusted data influences the actions of an agent. A runtime monitor can then enforce security policies before executing consequential actions (e.g., for tool-calling agents, based on the label of the context that generated a tool call and the labels of the arguments of the call).
Schedule
Location: IT University, Rued Langgaards Vej 7, 2300 Copenhagen, Aud. 2 (16 min away form Bella Center)
Contributed Talks
We are excited to share that the following researchers which be presenting their work as lightning talks at the workshop. Thank you to all who submitted talks and supported the review process.
Topics of Interest
- Mathematical foundations of adversarial robustness in language models
- Interpretability-driven security analysis
- Novel evaluation frameworks for real-world security
- Computational properties underlying vulnerabilities
- Secure-by-design architectures
- Sociotechnical aspects of LLM security
- Cross-domain transfer of security principles
Organizers
Volunteers
Discussion Chairs
Guided discussion in each thematic block will be moderated by a group of discussion chairs consisting of the speaker, organizers and volunteers who will guide the conversation between the workshop participants.
Supported By
Contact
For questions about the workshop, please contact:
egor [dot] zverev [at] ist.ac.at
EurIPS 2025 Workshop on Foundations of Language Model Security
December 6, 2025 • Copenhagen, Denmark