Tokenizer for a natural language processing system
Number of patents in Portfolio can not be more than 2000
United States of America Patent
Stats
-
Aug 15, 2006
Grant Date -
Jan 30, 2003
app pub date -
Mar 30, 2001
filing date -
Jul 20, 2000
priority date (Note) -
In Force
status (Latency Note)
Importance
Abstract
The present invention is a segmenter used in a natural language processing system. The segmenter segments a textual input string into tokens for further natural language processing. In accordance with one feature of the invention, the segmenter includes a tokenizer engine that proposes segmentations and submits them to a linguistic knowledge component for validation. In accordance with another feature of the invention, the segmentation system includes language-specific data that contains a precedence hierarchy for punctuation. If proposed tokens in the input string contain punctuation, they can illustratively be broken into subtokens based on the precedence hierarchy.
First Claim
Family
International Classification(s)
Cited Art Landscape
Patent Citation Ranking
Advertisement
Advertisement
Advertisement
Recipient Email Address
Recipient Email Address
Comment
Recipient Email Address
Add to Portfolio(s)
To add this patent to one, or more, of your portfolios, simply click the add button.
This Patent is in these Portfolios:
Add to additional portfolios:
Last Refreshed On:
Changes done successfully