ChatGPT cannot reliably decode Base64

Ask ChatGPT to decode a Base64 string and you may get confident English that has nothing to do with the bytes. Reddit threads are full of screenshots where the model invents a sentence instead of running the algorithm. Benchmarks such as Base64Bench show even frontier models miss perfect encode/decode a large fraction of the time.

Why LLMs fail at Base64

Base64 is a deterministic mapping of bits to characters. Language models sample tokens; they approximate patterns from training data. Short, common strings sometimes work. Long, random, or security-sensitive payloads often do not—and the model rarely admits uncertainty.

That matters when the payload is:

• An Authorization: Basic … header
• A JWT segment
• A PEM key body
• A data URI embedded in HTML/CSS
• Anything an AI agent will send to a production API

What to do instead

1. Use a deterministic encoder/decoder (this site runs in your browser).
2. Verify model output with Verify Base64: round-trip and optional plaintext match.
3. If a string was encoded twice, unwrap nested Base64.
4. For agents and MCP tools, see Base64 for AI agents.

Core rule

Never ask a model to be the Base64 library. Ask it to call one.

Encoding and decoding are not reasoning tasks. Treat them like arithmetic: use a tool, then continue.