This is really helpful, thanks. I followed up until the part about token IDs.

Token text, even in binary, is of variable length, making it hard to work with, but token IDs are just numbers.

How and why is token text converted to an ID? Is the ID for a specific word always the same?