...
The tokenizer language is a built-in language in camel-core, which is most often used only with the Splitter EIP to split a message using a token-based strategy.
For more details see Splitter. The tokenizer language is intended to tokenize text documents using a specified delimiter pattern. It can also tokenize XML documents with some limited capability. For a truly XML-aware tokenization, the use of the xml tokenizer language is recommended.