This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Skip to end of metadata
Go to start of metadata

Overview of Context Dependent Tokenizer

This annotator creates annotations from one or more tokens, using surrounding tokens as clues. An example of an annotation created from multiple tokens is a range that includes two numbers and a dash (e.g. 2-3).

Analysis engines (annotators)


Include this analysis engine in your pipeline if you wish to have the following annotations created:

  • DateAnnotation
  • FractionAnnotation
  • MeasurementAnnotation
  • PersonTitleAnnotation
  • RangeAnnotation
  • RomanNumeralAnnotation
  • TimeAnnotation



This is an aggregate analysis engine that can be used to run a short pipeline that takes plain text as input and annotates for tokens, sentences, and for the annotations created by the context dependent tokenizer annotator. This aggregate does not override any parameters or resource bindings.

  • No labels