7.00.02 - TextClassifierTrainer Arguments - Aster Analytics

Teradata Aster® Analytics Foundation User GuideUpdate 2

Product
Aster Analytics
Release Number
7.00.02
Release Date
September 2017
Content Type
Programming Reference
User Guide
Publication ID
B700-1022-700K
Language
English (United States)
InputTable
Specifies the name of the table that contains the documents to use to train the model.
TextColumn
Specifies the name of the column that contains the text of the training documents.
CategoryColumn
Specifies the name of the column that contains the category of the training documents.
ModelFile
Specifies the name for the model file to be generated.
ClassifierType
Specifies the classifier type of the model, KNN algorithm or maximum entropy model.
ClassifierParameters
[Optional] Applies only if the classifier type of the model is KNN. Specifies parameters for the classifier.

The name must be 'compress' and value must be in the range (0, 1). The n training documents are clustered into value*n groups (for example, if there are 100 training documents, then ClassifierParameters('compress:0.6') clusters them into 60 groups), and the model uses the center of each group as the feature vector.

NLPParameters
[Optional] Specifies natural language processing (NLP) parameters for preprocessing the text data and produce tokens.

Each name:value pair must be one of the following:

  • tokenDictFile:token_file

    where token_file is the name of an Aster Database file in which each line contains a phrase, followed by a space, followed by the token for the phrase (and nothing else).

  • stopwordsFile:stopword_file

    where stopword_file is the name of an Aster Database file in which each line contains exactly one stop word (a word to ignore during tokenization, such as a, an, or the).

  • useStem:{'true' | 'false'}

    which specifies whether the function stems the tokens. Default: 'false'.

  • stemIgnoreFile:stem_ignore_file

    where stem_ignore_file is the name of an Aster Database file in which each line contains exactly one word to ignore during stemming. Specifying this parameter with 'useStem:false' causes an exception.

  • useBgram:{'true' | 'false'}

    which specifies whether the function uses Bigram, which considers the proximity of adjacent tokens when analyzing them. Default: 'false'.

  • language:{ 'en' | 'zh_CN' | 'zh_TW' }

    which specifies the language of the input text—English (Default), Simplified Chinese, or Traditional Chinese, respectively. For zh_CN and zh_TW, the function ignores the parameters useStem and stemIgnoreFile.

Example:
NLPParameters ('tokenDictFile:token_dict.txt', 
'stopwordsFile:fileName', 
'useStem:true', 
'stemIgnoreFile:fileName', 
'useBgram:true', 
'language:zh_CN')
FeatureSelectionMethod
[Optional] Specifies the feature selection method, DF (document frequency). The values min and max must be in the range (0, 1). The function selects only the tokens that appear in at least min*n documents and at most max*n documents, where n is the number of training documents. For example, FeatureSelection ('DF:[0.1:0.9]') causes the function to select only the tokens that appear in at least 10% but no more than 90% of the training documents. If min exceeds max, the function uses min as max and max as min.