Overview of Context Dependent Tokenizer
This annotator creates annotations from one or more tokens, using surrounding tokens as clues. An example of an annotation created from multiple tokens is a range that includes two numbers and a dash (e.g. 2-3).
Analysis engines (annotators)
ContextDependentTokenizerAnnotator.xml
Include this analysis engine in your pipeline if you wish to have the following annotations created:
- DateAnnotation
- FractionAnnotation
- MeasurementAnnotation
- PersonTitleAnnotation
- RangeAnnotation
- RomanNumeralAnnotation
- TimeAnnotation
Parameters
(none)
TestTAE.xml
This is an aggregate analysis engine that can be used to run a short pipeline that takes plain text as input and annotates for tokens, sentences, and for the annotations created by the context dependent tokenizer annotator. This aggregate does not override any parameters or resource bindings.