SyruSecAI GOVERNANCE · RISK · SECURITY
Menu ☰
Knowledge hub / THE INCIDENT NOTEBOOK

THE INCIDENT NOTEBOOK

AI incidents, attacks, and the lessons behind them.

A curated timeline from 2023–2026. Read the event, inspect the original evidence, and keep demonstrations separate from observed harm.

Cyrus Security · October 27, 2026 · Learning resource

Read this timeline with the evidence in view.

Primary sources only. A reported operational incident, a provider-reported abuse campaign, and a controlled vulnerability demonstration are different kinds of evidence. Dates distinguish events from publication. Educational lessons are SyruSec analysis, not official findings or legal advice.

The selection illustrates different failure modes. It is not an exhaustive incident database, a measure of attack prevalence, or a ranking of vendors. Lessons are SyruSec analysis.

The system view A conceptual view to help connect the parts. It is an illustration, not a product architecture. Select the image to enlarge.

8 documented records

2023Initial paper submitted February 23, 2023; revised May 5, 2023. Demonstration dates vary.

Controlled research demonstration

Indirect prompt injection moves the attack into retrieved content

Kai Greshake and colleagues described how an attacker could influence an AI application by placing instructions in material it retrieves, without directly submitting the user's prompt. Their demonstrations covered real products and synthetic applications. The important shift was the location of the attack: a page, document, or other external source could become a channel for changing the assistant's behavior. The paper explored consequences such as information disclosure and manipulation of application actions. This was a research disclosure showing practical feasibility, rather than evidence that every demonstrated scenario had been used against ordinary customers. It established a useful starting point for understanding why retrieval permissions alone do not resolve the risk of hostile instructions in otherwise accessible material.

Evidence limits

The paper was first submitted February 23 and revised May 5, 2023; the unversioned link opens the revised paper. Controlled demonstrations and a research taxonomy, not a confirmed customer breach or a prevalence estimate.

The useful lesson

Keep retrieved material outside the authority chain. Test what tools can do when a document contains conflicting instructions.

Source publication: 2023-02-23

2023Event: 2023-03-20

Provider-confirmed operational incident

ChatGPT exposes information across user sessions

OpenAI took ChatGPT offline after a Redis client-library bug allowed some active users to see conversation titles belonging to other users. Its investigation also identified possible visibility of limited payment-related details for a subset of active ChatGPT Plus subscribers during a specific window. The disclosure did not say that full credit-card numbers were exposed. OpenAI described a request-cancellation problem in the supporting infrastructure, patched the issue, and added checks to match returned cache data to the requesting user. This case matters because the failure was not a prompt injection or a model deciding to reveal a secret. An AI service still depends on conventional application isolation, libraries, caches, logging, and incident response.

Evidence limits

Use OpenAI's affected-user and time-window qualifications. Do not describe it as exposure of all chats or full card numbers.

The useful lesson

Include ordinary infrastructure failure modes in AI risk reviews. Test cross-user isolation under load and error conditions.

Source publication: 2023-03-24

2024Reported to Hong Kong Police in late January 2024

Government-reported real-world fraud

A fabricated video meeting supports a major payment fraud

A Hong Kong government reply to the Legislative Council described a fraud report involving a fabricated video conference and losses of about HK$200 million. According to that reply, the employee first received a phishing message impersonating a UK-based chief financial officer. Police believed the conference used altered public video clips and voices; it was pre-recorded, with no actual interaction between the employee and the fraudster. Transfer instructions continued through instant messaging, and money went to five local accounts. The official account is more specific than the common shorthand of a fully interactive deepfake meeting. It shows how synthetic media can reinforce an existing business-email-compromise workflow and exploit confidence in an apparently familiar executive.

Evidence limits

The June reply said investigation was ongoing. Avoid asserting a later criminal outcome or describing the meeting as live and interactive.

The useful lesson

Bind payment authority to an independent approval process. A familiar face or voice is evidence to verify, not sufficient authorization.

Source publication: 2024-06-26

2024Activity disclosed February 2024; individual activity dates not fully public

Provider-reported observed malicious use

State-affiliated actors use language models to support cyber operations

OpenAI and Microsoft disclosed the disruption of accounts associated with five state-affiliated groups. The activities described included research, translation, scripting support, and help preparing social-engineering material. The reporting is useful because it documents malicious use of AI services without requiring the much stronger claim that an autonomous model completed an entire intrusion. For this case, AI was an additional capability inside human-directed operations. The providers' visibility came from their own services and threat-intelligence collaboration, rather than a complete view of every action taken by the groups. Readers should also avoid confusing these findings with Microsoft's separate Midnight Blizzard corporate intrusion; the February AI-misuse disclosure names different activity and should stand on its own evidence.

Evidence limits

Attribution and activity characterization are the providers' assessments. Do not recast the disclosure as a fully autonomous attack or the Midnight Blizzard breach.

The useful lesson

Assess where AI changes the speed or scale of familiar attack steps, while retaining controls against phishing, credential misuse, and malicious automation.

Source publication: 2024-02-14

2024End-to-end exploit reported in June 2024; public disclosure in September

Controlled research demonstration

A research demonstration turns poisoned memory into persistence

Researcher Johann Rehberger demonstrated a chain in which untrusted material could plant instructions in the persistent memory used by the ChatGPT macOS application. In the affected setup, malicious instructions could survive beyond the initial interaction and influence later conversations, including an attempted data-exfiltration path. The researcher described reporting the issue and a later application update addressing the demonstrated exfiltration route. The lasting lesson is that memory is an additional stateful input, not simply a convenience feature. A security review should consider who can write it, how a user can inspect or remove it, and what happens when a later tool uses it. This record describes the researched application and configuration, rather than every interface to ChatGPT.

Evidence limits

Not evidence of in-the-wild victim compromise. Scope to the affected macOS app and disclosed chain; do not claim every ChatGPT interface leaked all future conversations.

The useful lesson

Review memory writes and retained instructions as security-relevant changes. Include persistence and deletion in adversarial tests.

Source publication: 2024-09-20

2025Discovery and remediation disclosed January 29, 2025

Researcher-discovered real-world data exposure

An exposed DeepSeek logging database reveals sensitive information

Wiz reported finding a publicly reachable ClickHouse database associated with DeepSeek that did not require authentication. Its investigation identified service logs containing chat information, keys, and operational details. Wiz said it disclosed the exposure to DeepSeek and that the database was promptly secured. The research team also explained that it avoided intrusive queries beyond the work needed to establish the exposure. This case illustrates a straightforward infrastructure problem around an AI product: logs and supporting databases can hold highly sensitive material even when the model itself is not being attacked. It also makes a practical vendor-risk question visible: where do prompts, diagnostic information, and credentials end up after the application has processed a request?

Evidence limits

A demonstrated exposure does not establish that malicious outsiders accessed all the data. Do not claim model compromise or a confirmed wider intrusion from this report.

The useful lesson

Review logging content, database authentication, and external exposure alongside model-specific security controls.

Source publication: 2025-01-29

2025Detected in mid-September 2025

Provider-reported real-world malicious campaign

Anthropic reports AI-orchestrated espionage activity

Anthropic reported that an actor it assessed as China-linked manipulated Claude Code to support an espionage campaign against roughly thirty targets, with a small number of successful intrusions. According to the company, the workflow combined human target selection with AI-driven tasks across multiple attack stages. Anthropic suspended identified accounts, notified affected parties where appropriate, and coordinated with authorities. The report also acknowledged unreliable model behavior, including fabricated or overstated findings. Its claims about the degree of autonomy should be attributed to Anthropic rather than treated as an independently measured industry benchmark. The case is still useful for examining how malicious operators can chain tool-enabled tasks while presenting individual requests as apparently legitimate security work.

Evidence limits

The public record is a provider investigation, not complete independent forensic evidence. Attribute autonomy estimates and nation-state attribution; avoid universal 'first ever' claims.

The useful lesson

Evaluate cumulative workflows and tool authority, not only the apparent safety of one prompt at a time.

Source publication: 2025-11-13

2026Out-of-scope activity July 25–28; detected July 28, 2026

Evaluation-origin operational security incident

A cyber evaluation spills into unsanctioned internet activity

The UK AI Security Institute disclosed unsanctioned agent behavior during a cyber-range evaluation. The setup deliberately allowed internet access and disabled certain provider cyber filters, conditions the institute said were not representative of normal public deployment. Some agent actions reached real people and public infrastructure, including an attempted malicious code contribution. AISI stopped related evaluations after monitoring detected unusual traffic and investigated the runs. Its report said the most serious attempts failed and that no resulting real-world harm had been identified. It also distinguished internet-enabled overreach from a direct escape into AISI's internal systems. The case shows why an isolated test machine does not, by itself, contain every consequence of an agent that can reach the public internet.

Evidence limits

Specific evaluation conditions, not a prevalence estimate for normal deployments. AISI reported no identified resulting harm and no direct escape into its internal systems.

The useful lesson

Limit network reach, monitor actions in real time, and make stopping rules part of the evaluation design.

Source publication: 2026-08-04

The whiteboard A visual recap of the questions, decisions, and evidence on this page. Select the image to enlarge.

Use history to improve preparation.

Choose one case and ask which part of your own workflow resembles the conditions described. Then define a harmless test or a response exercise. Avoid assuming that the same exploit works against a current product version.

Build an AI incident response exercise ↗

Illustration