| | |
Methods defined here:
- __init__(self, data=None, text_column=None, category_column=None, classifier_type='maxEnt', classifier_parameters=None, nlp_parameters=None, feature_selection=None, model_file=None, data_sequence_column=None)
- DESCRIPTION:
The TextClassifierTrainer function trains a machine-learning
classifier for text classification and installs the model file on
the ML Engine. The model file can then be input to the function
TextClassifier.
PARAMETERS:
data:
Required Argument.
Specifies the name of the teradataml DataFrame that contains the
documents to use to train the model.
text_column:
Required Argument.
Specifies the name of the column that contains the text of the
training documents.
Types: str
category_column:
Required Argument.
Specifies the name of the column that contains the category of
the training documents.
Types: str
classifier_type:
Required Argument.
Specifies the classifier type of the model, KNN algorithm or
maximum entropy model.
Default Value: "maxEnt"
Permitted Values: maxEnt, knn
Types: str
classifier_parameters:
Optional Argument.
Applies only if the classifier type of the model is KNN.
Specifies parameters for the classifier. The name must be
"compress" and value must be in the range (0, 1). The n training
documents are clustered into value*n groups (for example, if
there are 100 training documents, then classifier_parameters
("compress:0.6") clusters them into 60 groups), and the model
uses the center of each group as the feature vector.
Types: str OR list of strs
nlp_parameters:
Optional Argument.
Specify natural language processing (NLP) parameters for
preprocessing the text data and produce tokens:
• tokenDictFile: token_file - token_file is name of ML
Engine file in which each line contains a phrase, followed
by a space, followed by the token for the phrase (and
nothing else).
• stopwordsFile:stopword_file - stopword_file is the name
of an ML Engine file in which each line contains
exactly one stop word (a word to ignore during tokenization,
such as a, an, or the).
• useStem:{true|false} - Specifies whether the function
stems the tokens. The default value is "false".
• stemIgnoreFile:stem_ignore_file - stem_ignore_file is
the name of an ML Engine file in which each line
contains exactly one word to ignore during stemming.
Specifying this parameter with "useStem:false" causes an
exception.
• useBgram:{ true | false } - Specifies whether the function
uses Bigram, which considers the proximity of adjacent
tokens when analyzing them. The default value is "false".
• language:{ en | zh_CN | zh_TW } - Specifies the language
of the input text — English (en), Simplified Chinese (zh_CN),
or Traditional Chinese (zh_TW). The default value is en.
For the values zh_CN and zh_TW, the function ignores the
parameters useStem and stemIgnoreFile.
Example: nlp_parameters("tokenDictFile:token_dict.txt",
"stopwordsFile:fileName",
"useStem:true",
"stemIgnoreFile:fileName",
"useBgram:true", "language:zh_CN")
Types: str OR list of strs
feature_selection:
Optional Argument.
Specifies the feature selection method, DF (document frequency).
The values min and max must be in the range (0, 1). The function
selects only the tokens that appear in at least min*n documents
and at most max*n documents, where n is the number of training
documents. For example, FeatureSelection ("DF:[0.1:0.9]") causes
the function to select only the tokens that appear in at least
10% but no more than 90% of the training documents. If min
exceeds max, the function uses min as max and max as min.
Types: str
model_file:
Required Argument.
Specifies the name of the model file to be generated.
Types: str
data_sequence_column:
Optional Argument.
Specifies the list of column(s) that uniquely identifies each row of
the input argument "data". The argument is used to ensure
deterministic results for functions which produce results that vary
from run to run.
Types: str OR list of Strings (str)
RETURNS:
Instance of TextClassifierTrainer.
Output teradataml DataFrames can be accessed using attribute
references, such as TextClassifierTrainerObj.<attribute_name>.
Output teradataml DataFrame attribute name is:
result
RAISES:
TeradataMlException
EXAMPLES:
# Load example data.
load_example_data("textclassifiertrainer", "texttrainer_input")
# Create teradataml DataFrame objects.
# The input table contains text of the training documents and the
# category of the training documents.
texttrainer_input = DataFrame.from_table("texttrainer_input")
# Example 1 - The function outputs a binary file with the name
# specified by "model_file" argument.
TextClassifierTrainer_out = TextClassifierTrainer(data = texttrainer_input,
text_column = "content",
category_column = "category",
classifier_type = "knn",
classifier_parameters = ["compress:0.9"],
nlp_parameters = ["useStem:true", "stopwordsFile: stopwords.txt"],
feature_selection = "DF:[0.1:0.99]",
model_file = "knn.bin"
)
# Print the result teradataml DataFrame
print(TextClassifierTrainer_out)
- __repr__(self)
- Returns the string representation for a TextClassifierTrainer class instance.
|