Class Tokenizer


  • public class Tokenizer
    extends Object
    Simple tokenizer class. Used to parse data. This class is taken from org.eclipse.osgi.framework.internal.core
    Since:
    1.0.0
    • Constructor Detail

      • Tokenizer

        public Tokenizer​(String value)
        Construct the tokenizer from the given string.
        Parameters:
        value - String
    • Method Detail

      • getToken

        public String getToken​(String terminals)
        Returns the next token delimited by any character in the provided terminals String.
        Parameters:
        terminals - String
        Returns:
        String
      • getString

        public String getString​(String terminals,
                                String preserveEscapes)
        Returns the next token delimited by any character in the provided terminals String. If the text content being considered is within quotes, then return the entire string within quotes. If the text content within contains escape characters, preserve it if it is provided as the second parameter.
        Parameters:
        terminals - String
        preserveEscapes - String
        Returns:
        String
      • getString

        public String getString​(String terminals)
        Returns the next token delimited by any character in the provided terminals String. If the text content being considered is within quotes, then return the entire string within quotes. If the text content within contains escape characters, those are ignored.
        Parameters:
        terminals - String
        Returns:
        String
      • getChar

        public char getChar()
        Returns the next character to be processed by the tokenizer.
        Returns:
        char