How AI works · the unit of language
Token
30-second gist~30s read
A token is the chunk of language an AI works with. It's not quite a word and not quite a letter — usually a piece of a word. The AI reads tokens, predicts tokens, and bills you per token.
Roughly: one token ≈ four characters in English, or about three-quarters of a word. A 500-word email is around 700 tokens.
If you want more
Why "tokens" instead of "words"?
Computers don't read like we do. Languages don't even agree what a "word" is — Chinese has no spaces, German jams things together. So AI builders chose a smaller, more flexible unit: the token. Common words ("the", "and") are usually one token. Rarer words split: "playing" might become "play" + "ing".
This is why some languages cost more to use. A sentence in French may take 30% more tokens than the same sentence in English. A sentence in Tamil or Burmese can take three or four times more, because the model wasn't trained on as much text in those languages and has to break the words into smaller pieces.
Why this affects your wallet
Most AI services charge per million tokens — both for what you send (the input) and what comes back (the output). A long PDF you paste in racks up tokens. So does a long answer. There's also a hard context window limit: a model can only "see" so many tokens at once, after which it starts forgetting earlier parts of the conversation.