Splits given text to elements (tokens).

Syntax

<tokenize delimiters="delimiters"
          trimtokens="trimtokens"
          allowemptytokens="allowemptytokens">
    content to tokenize
</tokenize>

Attributes

Name Required Default Description
delimiters no new line character Tells which characters are used as token delimiters.
trimtokens no yes Tells whether to trim resulting tokens.
allowemptytokens no no Tells whether to include empty tokens in the resulting list (consisting only of whitespaces).