nanoswe-tokenizer-261002

A BPE tokenizer with a 32,768-token vocabulary, in the Hugging Face tokenizers format (tokenizer.json). It was trained on the training split of nanoswe/nanoswe-trajs-261002.

sha256 of tokenizer.json: 181a58da5d7e4f67b453ad63a422336749c0959ed31d8792b29d6516ba10bd01