efxtools.sample.ae
Class EfXRBSentenceSplitter
java.lang.Object
org.apache.uima.analysis_component.AnalysisComponent_ImplBase
org.apache.uima.analysis_component.Annotator_ImplBase
org.apache.uima.analysis_component.JCasAnnotator_ImplBase
efxtools.filtering.FilteringAnalysisEngine
efxtools.sample.ae.EfXRBSentenceSplitter
- All Implemented Interfaces:
- org.apache.uima.analysis_component.AnalysisComponent
public class EfXRBSentenceSplitter
- extends FilteringAnalysisEngine
This analysis engine represents a rule-based sentence splitter that, in general,
can be used to split any text of any western language into sentences. It does not
any input annotations, but only processes the unstructured input text.
The sentence splitter sequentially determines the end of each sentence, assuming
that the previously found sentence end has been correct. The rules used to classify
whether the a character denotes the end of a sentence can be described in a
high-level manner as follows:
- IF the current character is a period, which...
... does not belong to a single character, AND
... does not belong to an ordinal number, AND
... does not belong to an abbreviation AND
... does not belong to a URL AND
... does not belong to an ellipsis that is not followed by an upper-case character
AND
... is not followed by a right parenthesis AND
... is not followed by a comma
THEN the sentence ends after the period or its possibly succeeding quotation
mark.
- IF the current character is a question or an exclamation mark, which...
... does not belong to a URL AND
... is not followed by any sentence end mark AND
... is not followed by right parenthesis AND
... is not followed by a comma
THEN the sentence ends after the period or its possibly succeeding quotation
mark.
- IF the following character is a line break, which is...
... preceded by a line that is completely in brackets OR
... preceded by a line that is completely in upper-case OR
... preceded by a line that does not start with a word OR
... preceded by a line that ends with a URL OR
... preceded by a line that ends with a sequence of hyphens OR
... followed by a line that starts with a bulletpoint OR
... followed by a line that is a blank line
THEN the sentence ends after the preceding line.
NOTICE: This version of the sentence splitter is a mostly simplified, but also a
little extended version of the sentence splitter developed within the InfexBA
project. While the InfexBA sentence splitter had been designed with German news
articles in mind, this sentence splitter aims to be much more more generally
applicable. It has been successfully applied to both German and English texts as
well as both user reviews and news articles.
- Author:
- Author of EfXTools
|
Method Summary |
void |
initialize(org.apache.uima.UimaContext aContext)
|
| Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase |
getRequiredCasInterface, process |
| Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase |
getCasInstancesRequired, hasNext, next |
| Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase |
batchProcessComplete, collectionProcessComplete, destroy, reconfigure |
| Methods inherited from class java.lang.Object |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
PARAM_LEXICON_PREFIX
public static final java.lang.String PARAM_LEXICON_PREFIX
- The name of the parameter for the URL prefix lexicon
- See Also:
- Constant Field Values
PARAM_LEXICON_SUFFIX
public static final java.lang.String PARAM_LEXICON_SUFFIX
- The name of the parameter for the URL suffix lexicon
- See Also:
- Constant Field Values
PARAM_LEXICON_ABBREVIATIONS
public static final java.lang.String PARAM_LEXICON_ABBREVIATIONS
- The name of the parameter for the abbreviation lexicon
- See Also:
- Constant Field Values
EfXRBSentenceSplitter
public EfXRBSentenceSplitter()
initialize
public void initialize(org.apache.uima.UimaContext aContext)
throws org.apache.uima.resource.ResourceInitializationException
- Specified by:
initialize in interface org.apache.uima.analysis_component.AnalysisComponent- Overrides:
initialize in class org.apache.uima.analysis_component.AnalysisComponent_ImplBase
- Throws:
org.apache.uima.resource.ResourceInitializationException