A Python NLP framework
Python GateNLP is a natural language processing (NLP) and text processing framework implemented in Python.
It provides very flexible representations of documents, stand-off annotaitons with arbitrary types and features, grouped into arbitrary annotations sets, spans, corpora, annotators, pipelines and more. Documents, annotations and corpora can be easily and interactively visualized in notebooks. It provides the ability to use existing NLP tools for annotating documents out of the box: Spacy, Stanza as well as online services like Gate Cloud, ELG, Google NLP, IBM Watson and others. The result of these tools get represented as GateNLP annotations making it easy to write code that works with all of these tools in the same way or compares or combines the results of these tools.
In addition, GateNLP provides its own annotator tools: string-based and token based gazetteers, regular-expression-based annotators, and a very powerful and flexible rule-based annotator (PAMPAC) which allows to match complex pattern of annotations and text.
Python GateNLP represents documents and stand-off annotations very similar to the Java GATE framework: Annotations describe arbitrary character ranges in the text and each annotation can have an arbitrary number of features. Documents can have arbitrary features and an arbitrary number of named annotation sets, where each annotation set can have an arbitrary number of annotations which can overlap in any way. Python GateNLP documents can be exchanged with Java GATE by using the bdocjs/bdocym/bdocmp formats which are supported in Java GATE via the Format Bdoc Plugin
Other than many other Python NLP tools, GateNLP does not require a specific way of how text is split up into tokens, tokens can be represented by annotations in any way, and a document can have different ways of tokenization simultanously, if needed. Similarly, entities can be represented by annotations without restriction: they do not need to start or end at token boundaries and can overlap arbitrarily.
GateNLP provides ways to process text and create annotations using annotating pipelines, which are sequences of one or more annotators. There are gazetteer annotators for matching text against gazetteer lists and annotators for a rule-like matching of complex annotation and text sequences (see PAMPAC).
There is also support for creating GateNLP annotations with other NLP packages like Spacy or Stanford Stanza.
The GateNLP document representation also optionally allows to track all changes done to the document in a "change log". Such changes can later be applied to other Python GateNLP or to Java GATE documents.
This library also implements the functionality for the interaction with a Java GATE process in two different ways: * The Java GATE Python plugin can invoke a process running Python GateNLP to annotate GATE documents. * Python code can remote-control a Jave GATE instance via the GateNLP GateWorker
Install GateNLP with all optional dependencies:
pip install -U gatenlp[all]
For more details see Installation
Overview of the documentation:
NOTE: most of the documentation pages below can be viewed as HTML, as a Jupyter notebook (NB), and the Jupyter notebook can be downloaded for running on your own computer (NB-DL).
- Getting Started / NB / NB-DL
- The Document class and classes related to components of a document:
- The Changelog class for recording changes to a document
- A comparison with the Java GATE API
- The module for running Java GATE code from Python
- Modules for interaction with other NLP packages and converting their documents
- Connecting to annotation services on the web (clients): annotators which connect to a web/cloud service to annotate documents.
- Modules related to NLP processing:
- Corpora and Document Sources / Destinations / NB / NB-DL
- Processing / NB / NB-DL
- Tokenizers / NB / NB-DL
- Matching strings and token sequences:
- Token/String Gazetteers / NB / NB-DL
- Regular Expressions Annotator / NB / NB-DL
- Complex Annotation Patterns for matching text and annotation sequences:
- PAMPAC / NB / NB-DL
- PAMPAC Reference
- Viewers and object representation / NB / NB-DL
- Overview of supported formats for loading and saving documents
- Miscellaneous Recipes and Examples
- Change Log: show major changes in each release since 1.0.1