System for chinese tokenization and named entity recognition
A system (100, 200) for tokenization and named entity recognition of ideographic language is disclosed. In the system, a word lattice is generated for a string of ideographic characters using finite state grammars (150) and a system lexicon (240). Segmented text is generated by determining word boun...
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Patent |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | LOUDON GARETH WU HORNG JYH PAUL LI HAIZHOU BAI SHUANHU |
| description | A system (100, 200) for tokenization and named entity recognition of ideographic language is disclosed. In the system, a word lattice is generated for a string of ideographic characters using finite state grammars (150) and a system lexicon (240). Segmented text is generated by determining word boundaries in the string of ideographic characters using the word lattice dependent upon a contextual language model (152A) and one or more entity language models (152B). One or more named entities is recognized in the string of ideographic characters using the word lattice dependent upon the contextual language model (152A) and the one or more entity language models (152B). The contextual language model (152A) and the one or more entity language models (152B) are each class-based language models. The lexicon (240) includes single ideographic characters, words, and predetermined features of the characters and words. |
| format | Patent |
| fullrecord | <record><control><sourceid>epo_EVB</sourceid><recordid>TN_cdi_epo_espacenet_US6311152B1</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>US6311152B1</sourcerecordid><originalsourceid>FETCH-epo_espacenet_US6311152B13</originalsourceid><addsrcrecordid>eNrjZLAJriwuSc1VSMsvUkjOyMxLLU5VKMnPTs3LrEosyczPU0jMS1HIS8xNTVFIzSvJLKlUKEpNzk_PywRJ8jCwpiXmFKfyQmluBgU31xBnD93Ugvz41OKCxOTUvNSS-NBgM2NDQ0NTIydDYyKUAADArS_5</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>patent</recordtype></control><display><type>patent</type><title>System for chinese tokenization and named entity recognition</title><source>esp@cenet</source><creator>LOUDON GARETH ; WU HORNG JYH PAUL ; LI HAIZHOU ; BAI SHUANHU</creator><creatorcontrib>LOUDON GARETH ; WU HORNG JYH PAUL ; LI HAIZHOU ; BAI SHUANHU</creatorcontrib><description>A system (100, 200) for tokenization and named entity recognition of ideographic language is disclosed. In the system, a word lattice is generated for a string of ideographic characters using finite state grammars (150) and a system lexicon (240). Segmented text is generated by determining word boundaries in the string of ideographic characters using the word lattice dependent upon a contextual language model (152A) and one or more entity language models (152B). One or more named entities is recognized in the string of ideographic characters using the word lattice dependent upon the contextual language model (152A) and the one or more entity language models (152B). The contextual language model (152A) and the one or more entity language models (152B) are each class-based language models. The lexicon (240) includes single ideographic characters, words, and predetermined features of the characters and words.</description><edition>7</edition><language>eng</language><subject>CALCULATING ; COMPUTING ; COUNTING ; ELECTRIC DIGITAL DATA PROCESSING ; PHYSICS</subject><creationdate>2001</creationdate><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20011030&DB=EPODOC&CC=US&NR=6311152B1$$EHTML$$P50$$Gepo$$Hfree_for_read</linktohtml><link.rule.ids>230,308,776,881,25542,76289</link.rule.ids><linktorsrc>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20011030&DB=EPODOC&CC=US&NR=6311152B1$$EView_record_in_European_Patent_Office$$FView_record_in_$$GEuropean_Patent_Office$$Hfree_for_read</linktorsrc></links><search><creatorcontrib>LOUDON GARETH</creatorcontrib><creatorcontrib>WU HORNG JYH PAUL</creatorcontrib><creatorcontrib>LI HAIZHOU</creatorcontrib><creatorcontrib>BAI SHUANHU</creatorcontrib><title>System for chinese tokenization and named entity recognition</title><description>A system (100, 200) for tokenization and named entity recognition of ideographic language is disclosed. In the system, a word lattice is generated for a string of ideographic characters using finite state grammars (150) and a system lexicon (240). Segmented text is generated by determining word boundaries in the string of ideographic characters using the word lattice dependent upon a contextual language model (152A) and one or more entity language models (152B). One or more named entities is recognized in the string of ideographic characters using the word lattice dependent upon the contextual language model (152A) and the one or more entity language models (152B). The contextual language model (152A) and the one or more entity language models (152B) are each class-based language models. The lexicon (240) includes single ideographic characters, words, and predetermined features of the characters and words.</description><subject>CALCULATING</subject><subject>COMPUTING</subject><subject>COUNTING</subject><subject>ELECTRIC DIGITAL DATA PROCESSING</subject><subject>PHYSICS</subject><fulltext>true</fulltext><rsrctype>patent</rsrctype><creationdate>2001</creationdate><recordtype>patent</recordtype><sourceid>EVB</sourceid><recordid>eNrjZLAJriwuSc1VSMsvUkjOyMxLLU5VKMnPTs3LrEosyczPU0jMS1HIS8xNTVFIzSvJLKlUKEpNzk_PywRJ8jCwpiXmFKfyQmluBgU31xBnD93Ugvz41OKCxOTUvNSS-NBgM2NDQ0NTIydDYyKUAADArS_5</recordid><startdate>20011030</startdate><enddate>20011030</enddate><creator>LOUDON GARETH</creator><creator>WU HORNG JYH PAUL</creator><creator>LI HAIZHOU</creator><creator>BAI SHUANHU</creator><scope>EVB</scope></search><sort><creationdate>20011030</creationdate><title>System for chinese tokenization and named entity recognition</title><author>LOUDON GARETH ; WU HORNG JYH PAUL ; LI HAIZHOU ; BAI SHUANHU</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-epo_espacenet_US6311152B13</frbrgroupid><rsrctype>patents</rsrctype><prefilter>patents</prefilter><language>eng</language><creationdate>2001</creationdate><topic>CALCULATING</topic><topic>COMPUTING</topic><topic>COUNTING</topic><topic>ELECTRIC DIGITAL DATA PROCESSING</topic><topic>PHYSICS</topic><toplevel>online_resources</toplevel><creatorcontrib>LOUDON GARETH</creatorcontrib><creatorcontrib>WU HORNG JYH PAUL</creatorcontrib><creatorcontrib>LI HAIZHOU</creatorcontrib><creatorcontrib>BAI SHUANHU</creatorcontrib><collection>esp@cenet</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>LOUDON GARETH</au><au>WU HORNG JYH PAUL</au><au>LI HAIZHOU</au><au>BAI SHUANHU</au><format>patent</format><genre>patent</genre><ristype>GEN</ristype><title>System for chinese tokenization and named entity recognition</title><date>2001-10-30</date><risdate>2001</risdate><abstract>A system (100, 200) for tokenization and named entity recognition of ideographic language is disclosed. In the system, a word lattice is generated for a string of ideographic characters using finite state grammars (150) and a system lexicon (240). Segmented text is generated by determining word boundaries in the string of ideographic characters using the word lattice dependent upon a contextual language model (152A) and one or more entity language models (152B). One or more named entities is recognized in the string of ideographic characters using the word lattice dependent upon the contextual language model (152A) and the one or more entity language models (152B). The contextual language model (152A) and the one or more entity language models (152B) are each class-based language models. The lexicon (240) includes single ideographic characters, words, and predetermined features of the characters and words.</abstract><edition>7</edition><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_epo_espacenet_US6311152B1 |
| source | esp@cenet |
| subjects | CALCULATING COMPUTING COUNTING ELECTRIC DIGITAL DATA PROCESSING PHYSICS |
| title | System for chinese tokenization and named entity recognition |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-04T17%3A04%3A34IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-epo_EVB&rft_val_fmt=info:ofi/fmt:kev:mtx:patent&rft.genre=patent&rft.au=LOUDON%20GARETH&rft.date=2001-10-30&rft_id=info:doi/&rft_dat=%3Cepo_EVB%3EUS6311152B1%3C/epo_EVB%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |