Tokenizer Playground

Real-time tokenization with popular LLM models (GPT-4, GPT-3.5, GPT-5, Claude, Llama 2, BERT, T5). Experiment with different tokenizers, view token statistics, and estimate costs.

OpenAI GPT-4 and GPT-3.5 tokenizer (cl100k_base — accurate)

TOKENS

0

CHARACTERS

0

WORDS

0

EST. COST

$0.0000

How to use:

  • • Select a tokenizer from the dropdown (GPT-4, Claude, Llama 2, etc.)
  • • Type or paste text in the input field to see real-time tokenization
  • • View token statistics including count, characters, words, and estimated cost
  • • Switch between Text, Token IDs, or Hide display modes
  • • Copy tokenized text to clipboard or export results as JSON

Accuracy Notes

  • • GPT-4, GPT-3.5, GPT-5, Claude: Token counts use the real cl100k_base tokenizer — accurate for GPT-4 and a very close approximation for Claude.
  • • Llama 2, BERT, T5: These use simplified approximations. A browser-compatible SentencePiece/WordPiece library is not available; counts may differ from the real model.
  • • Cost estimates are based on typical published pricing and may not reflect current rates.

❓Tokenizer Playground FAQ