The Plaintext Journal: When Telemetry Becomes a Forensic Goldmine

Why agent telemetry is unusually rich

A conventional application logs structured events: request paths, response codes, latency, exceptions. The volume can be large but the content is bounded — fields are named, types are known, sensitive values are typically masked at the logging library.

An AI agent's telemetry is different by design. Useful agent telemetry captures the operator's prompt, the model's reasoning trace, the verbs the agent attempted, the parameters of each tool call, and the responses received. The data is intentionally rich because the alternative — opaque events that record only "the agent did something" — defeats the point of having telemetry at all.

The richness has a consequence. Tool-call parameters and tool responses routinely contain content that would never appear in a conventional application log: portions of files the agent read, secrets that happened to be in those files, full URIs to resources the agent accessed, identity tokens passed as parameters, and the verbatim text of operator prompts that may themselves discuss confidential matters.

The OWASP A09:2021 Security Logging and Monitoring Failures category covers the broader risk class. Microsoft Purview ships sensitive-information-type detectors for specific high-value strings — including personal access tokens — precisely because their accidental appearance in logs is a documented recurring incident.

The Plaintext Journal anti-pattern

A hypothetical disclosure scenario

The following illustrates a plausible disclosure mode under the Plaintext Journal anti-pattern. It is constructed from elements common to several reported industry patterns; no specific incident is implied.

An organisation adopts an agent runtime that captures detailed telemetry to a local file. The file is intended for the developer's own debugging and is documented as such. Several months pass; the file accumulates a complete record of every agent session.

A separate event occurs: an unrelated developer-productivity tool installed on the same workstation depends on a transitive package that has been compromised. The compromised package enumerates the user's filesystem looking for high-value artefacts. It finds the agent telemetry file and exfiltrates it. Within minutes, the attacker has not only any tokens that happened to be captured but the full record of which resources the agent accessed, what the operator was working on, and which internal systems the organisation cares about.

No single component failed. The agent did exactly what it was configured to do. The compromise of the unrelated extension was a routine supply-chain incident. The conjunction — a high-value plaintext artefact left in a location reachable by any other software on the host — produced an outcome far worse than any individual component's failure would have.

Telemetry that captures everything and protects nothing is not a debug aid. It is a pre-staged exfiltration target.

The five controls that make telemetry safe

1. Redact at the boundary.

   At the point where telemetry is written, run a pre-processor that masks known sensitive patterns: identity tokens (eyJ... JWTs, GitHub PATs, cloud-provider access keys), URIs that imply secret access, bearer-token headers, base64-encoded credentials. Redaction is imperfect — pattern matching always is — but it removes the most-common, highest-impact disclosures.

2. Bound the retention.

   Telemetry files rotate by size (commonly five megabytes) or by age (commonly a day). Older rotations are compressed and deleted after a configured number of rolls. The journal cannot grow unbounded; the forensic surface is held to whatever debug horizon the team genuinely needs.

3. Validate against a schema.

   Telemetry events are structured: a fixed set of fields with known types and length limits. Events that fail validation are dropped or summarised. The schema prevents an attacker (or an over-eager developer) from filling disk with arbitrary content masquerading as telemetry.

4. Honour an operator opt-out.

   An environment variable or configuration flag lets the operator disable telemetry capture for sensitive workloads. The opt-out is documented and discoverable. Some workloads — privacy-sensitive workstations, regulated-data sessions — cannot tolerate local capture; the system must allow the operator to say so.

5. Forward to centralised observability.

   Local capture is the worst place for telemetry to live permanently. Forward events to a managed observability platform where access controls, retention policies, and tamper-evidence are first-class concerns. The local file becomes a short-lived buffer, not a permanent artefact.

A sixth control becomes relevant once the first five are in place: integrity protection. Each telemetry event includes a cryptographic hash of the previous event. Modification of any event invalidates the chain from that point forward. The control does not prevent compromise of the host, but it makes tampering with the audit trail detectable.

A practical checklist

• Telemetry capture has a documented purpose and a named internal owner.
• A redactor masks known sensitive patterns (JWTs, PATs, cloud access keys, bearer tokens, secret URIs) before write.
• Telemetry files rotate by size or age; older rotations are compressed and deleted after a configured retention.
• Telemetry events are schema-validated; events failing validation are dropped or summarised.
• An operator opt-out (environment variable or configuration flag) is documented and discoverable.
• Telemetry is forwarded to a centralised observability stack where retention and access controls apply.
• Telemetry file paths are outside any cloud-synced directory unless intentional.
• Filesystem permissions restrict telemetry file access to the operator user; group or world read is not enabled.
• Integrity protection (hash chaining or append-only store) makes tampering with the audit trail detectable.
• Periodic reviews verify that telemetry content does not include sensitive material the redactor missed; the regex set is updated when gaps are found.

Test your own agent in ten minutes

The fastest way to find out whether this anti-pattern is present in your own system is to ask an AI coding assistant to look for it. Run the prompt below in a fresh chat session, on its own — and judge the system by what the code actually does, not by what its documentation claims.

Search the whole repository to find where this applies — do not
wait for me to list files. Ignore generated, vendored, and dependency
folders (build output, node_modules, vendor). Identify every location
the failure mode below could occur, read those files in full before
you judge, and list the search terms you used so I can confirm nothing
was missed.

You are looking for one specific failure mode: the agent writes
telemetry (logs, traces, metrics, spans) that contains raw model
prompts, raw tool arguments, retrieved document contents, or raw
model output, without redaction of personal data, credentials, or
business-sensitive content, and without an access boundary that
matches the sensitivity of the original data.

Respond with exactly these four sections:
1. VERDICT: one of [present / not present / unclear]
2. EVIDENCE: file path + line numbers + a one-line quote per claim
3. WHY IT MATTERS: two sentences, plain English
4. FIX: a concrete change, with a short before/after code snippet
   if applicable. If "unclear", list the one piece of context you
   need to decide.

Insist on the four-part answer: a verdict with a file path, a line number, and a one-line quote is something you can act on; a verdict on its own is just an opinion. If the result is present, the FIX section is your starting point — redact personal data, credentials, and sensitive content before it reaches telemetry, and match the log’s access boundary to the data’s sensitivity. Re-run the same prompt after the change to confirm the verdict flips to not present.

Conclusion

Telemetry is not optional. Production AI systems need detailed observability — for debugging, evaluation, incident response, and continuous improvement. The discipline that makes telemetry safe is the same discipline that has long applied to any other high-leverage data: redact at the boundary, bound the retention, validate against a schema, allow opt-out, forward to managed storage, protect the integrity of what remains.

The Plaintext Journal anti-pattern is rarely the result of deliberate design. It is the result of telemetry being added as a debug aid and then forgotten. The remedy is also rarely difficult: take an hour to add the controls above and the artefact becomes a tool rather than a target.