Teradata Scoring SDK Text Tokenizer Additional Notes - Teradata Vantage

Machine Learning Engine Analytic Function Reference

Product
Teradata Vantage
Release Number
8.10
1.1
Published
October 2019
Language
English (United States)
Last Update
2019-12-31
dita:mapPath
ima1540829771750.ditamap
dita:ditavalPath
jsj1481748799576.ditaval
dita:id
B700-4003
lifecycle
previous
Product Category
Teradata Vantageā„¢

If the function uses a dictionary table, specify it with the ModelTable and ModelTag syntax elements of AMLGenerator. If the function uses a dictionary file or CRF model file, specify it with the InstalledFile and InstalledFileTag syntax elements of AMLGenerator. If you specify no tables, the function uses the default embedded dictionaries for English or Chinese text.

Example 1: Chinese

InstalledFile ('crf_model_file')
InstalledFileTag ('CRF')

Example 2: Chinese

ModelTable ('dict_table')
ModelTableTag ('DICT')
InstalledFile ('crf_model_file')
InstalledFileTag ('CRF')
InstalledFile2 ('dict_file')
InstalledFile2Tag ('DICT')

Example 3: English or Japanese

CRF file is not supported.

ModelTable('dict_table')
ModelTableTag('DICT')
InstalledFile ('dict_file')
InstalledFileTag ('DICT')