Uses of Interface
org.simmetrics.tokenizers.Tokenizer

Packages that use Tokenizer

Package	Description
org.simmetrics.builders
org.simmetrics.metrics
org.simmetrics.tokenizers

Uses of Tokenizer in org.simmetrics.builders

Methods in org.simmetrics.builders with parameters of type Tokenizer

Modifier and Type	Method and Description
StringDistanceBuilder.CollectionDistanceTokenizerStep<T>	StringDistanceBuilder.CollectionDistanceInitialSimplifierStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the distance.
StringDistanceBuilder.CollectionDistanceTokenizerStep<T>	StringDistanceBuilder.CollectionDistanceSimplifierStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the distance.
StringDistanceBuilder.CollectionDistanceTokenizerStep<T>	StringDistanceBuilder.CollectionDistanceInitialTokenizerStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the distance.
StringDistanceBuilder.CollectionDistanceTokenizerStep<T>	StringDistanceBuilder.CollectionDistanceTokenizerStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the distance.
StringMetricBuilder.CollectionMetricTokenizerStep<T>	StringMetricBuilder.CollectionMetricInitialSimplifierStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the metric.
StringMetricBuilder.CollectionMetricTokenizerStep<T>	StringMetricBuilder.CollectionMetricSimplifierStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the metric.
StringMetricBuilder.CollectionMetricTokenizerStep<T>	StringMetricBuilder.CollectionMetricInitialTokenizerStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the metric.
StringMetricBuilder.CollectionMetricTokenizerStep<T>	StringMetricBuilder.CollectionMetricTokenizerStep.tokenize(Tokenizer tokenizer) Adds a tokenization step to the metric.

Uses of Tokenizer in org.simmetrics.metrics

Methods in org.simmetrics.metrics with parameters of type Tokenizer

Modifier and Type	Method and Description
static StringMetric	StringMetrics.createForListMetric(Metric<List<String>> metric, Simplifier simplifier, Tokenizer tokenizer) Deprecated. Use StringMetricBuilder in favor of directly constructing a metric.
static StringMetric	StringMetrics.createForListMetric(Metric<List<String>> metric, Tokenizer tokenizer) Deprecated. Use StringMetricBuilder in favor of directly constructing a metric.
static StringMetric	StringMetrics.createForMultisetMetric(Metric<com.google.common.collect.Multiset<String>> metric, Simplifier simplifier, Tokenizer tokenizer) Deprecated. Use StringMetricBuilder in favor of directly constructing a metric.
static StringMetric	StringMetrics.createForMultisetMetric(Metric<com.google.common.collect.Multiset<String>> metric, Tokenizer tokenizer) Deprecated. Use StringMetricBuilder in favor of directly constructing a metric.
static StringMetric	StringMetrics.createForSetMetric(Metric<Set<String>> metric, Simplifier simplifier, Tokenizer tokenizer) Deprecated. Use StringMetricBuilder in favor of directly constructing a metric.
static StringMetric	StringMetrics.createForSetMetric(Metric<Set<String>> metric, Tokenizer tokenizer) Deprecated. Use StringMetricBuilder in favor of directly constructing a metric.

Uses of Tokenizer in org.simmetrics.tokenizers

Classes in org.simmetrics.tokenizers that implement Tokenizer

Modifier and Type	Class and Description
class	AbstractTokenizer Convenience tokenizer.

Methods in org.simmetrics.tokenizers that return Tokenizer

Modifier and Type	Method and Description
static Tokenizer	Tokenizers.chain(List<Tokenizer> tokenizers) Chains tokenizers together.
static Tokenizer	Tokenizers.chain(Tokenizer tokenizer, Tokenizer... tokenizers) Chains tokenizers together.
static Tokenizer	Tokenizers.filter(Tokenizer tokenizer, com.google.common.base.Predicate<String> predicate) Constructs a new filtering tokenizer.
static Tokenizer	Tokenizers.pattern(Pattern pattern) Returns a tokenizer that splits a string into tokens around the pattern as if calling pattern.split(input,-1).
static Tokenizer	Tokenizers.pattern(String regex) Returns a tokenizer that splits a string into tokens around the pattern as if calling Pattern.compile(regex).split(input,-1).
static Tokenizer	Tokenizers.qGram(int q) Returns a q-gram tokenizer for a variable q.
static Tokenizer	Tokenizers.qGramWithFilter(int q) Returns a q-gram tokenizer for a variable q.The tokenizer will return an empty collection if the input is empty or shorter then q.
static Tokenizer	Tokenizers.qGramWithPadding(int q) Returns a q-gram tokenizer for a variable q.
static Tokenizer	Tokenizers.qGramWithPadding(int q, String padding) Returns a q-gram tokenizer for a variable q.
static Tokenizer	Tokenizers.qGramWithPadding(int q, String startPadding, String endPadding) Returns a q-gram tokenizer for a variable q.The q-gram is extended beyond the length of the string with padding.
static Tokenizer	Tokenizers.transform(Tokenizer tokenizer, com.google.common.base.Function<String,String> function) Constructs a new transforming tokenizer.
static Tokenizer	Tokenizers.whitespace() Returns a tokenizer that splits a string into tokens around whitespace.

Methods in org.simmetrics.tokenizers with parameters of type Tokenizer

Modifier and Type	Method and Description
static Tokenizer	Tokenizers.chain(Tokenizer tokenizer, Tokenizer... tokenizers) Chains tokenizers together.
static Tokenizer	Tokenizers.chain(Tokenizer tokenizer, Tokenizer... tokenizers) Chains tokenizers together.
static Tokenizer	Tokenizers.filter(Tokenizer tokenizer, com.google.common.base.Predicate<String> predicate) Constructs a new filtering tokenizer.
static Tokenizer	Tokenizers.transform(Tokenizer tokenizer, com.google.common.base.Function<String,String> function) Constructs a new transforming tokenizer.

Method parameters in org.simmetrics.tokenizers with type arguments of type Tokenizer

Modifier and Type	Method and Description
static Tokenizer	Tokenizers.chain(List<Tokenizer> tokenizers) Chains tokenizers together.