
A counting rule not a value measure
A token is a piece of text. A token measures the text volume, the amount of text processed. Input and output. It affects processing time and cost. It does not measure the work of AI or quality of answer. It is not a universal measure, it differs between systems.
An analogy can be assembling a car from individual components estimating workload and cost (all though in this case the parts vary in complexity):
Tokens: the parts being handled
AI model: the factory and its machinery
Token count: how many parts processed
Token price: an price for each part processed
Different factories may count the components differently and put different value on the parts. Like one factory counts the seat as one and another the cushions and the frame.
This pricing scheme has no information on how many workers or machines involved or how good the car is.
Image generated with Firefly