Uses of Package
opennlp.tools.tokenize
Packages that use opennlp.tools.tokenize
Package
Description
Experimental package related to converting various corpora to OpenNLP Format.
Experimental package related to the
Arvores Deitadas corpus
format.Experimental package related to the corpus format used by the "brat rapid annotation tool" (brat).
Experimental package related to the CoNNL-U format.
Experimental package related to the Irish Sentence Bank format.
Experimental package related to the
letsmt
corpus format.Experimental package related to the
MASC
corpus format.Experimental package related to the
MUC
corpus format.Package related to identifying sentence boundaries.
Contains classes related to finding token or words in a string.
This package contains classes for generating sequence features.
-
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.parser
-
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.tokenizerClassDescriptionA marker interface for evaluating
tokenizers
.TheTokenizerModel
is the model used by a learnableTokenizer
.ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formatsClassDescriptionA
Detokenizer
merges tokens back to their detokenized representation.ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.ad
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.bratClassDescriptionThe interface for tokenizers, which segment a string into its tokens.The
TokenizerModel
is the model used by a learnableTokenizer
. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.conllu
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.convertClassDescriptionA
Detokenizer
merges tokens back to their detokenized representation.ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.irishsentencebank
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.letsmt
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.masc
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.muc
-
Classes in opennlp.tools.tokenize used by opennlp.tools.sentdetect
-
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenizeClassDescriptionA
Detokenizer
merges tokens back to their detokenized representation.This enum contains an operation for every token to merge the tokens together to their detokenized form.A basicTokenizer
implementation which performs tokenization using character classes.Interface for context generators required forTokenizerME
.The interface for tokenizers, which segment a string into its tokens.A marker interface for evaluatingtokenizers
.The factory that providesTokenizer
default implementation and resources.TheTokenizerModel
is the model used by a learnableTokenizer
.ATokenSample
is text with token spans.A basicTokenizer
implementation which performs tokenization using white spaces. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang
-
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang.en
-
Classes in opennlp.tools.tokenize used by opennlp.tools.util.featuregen