| | |
Methods defined here:
- __init__(self, data=None, text_column=None, category_column=None, classifier_type='maxEnt', classifier_parameters=None, nlp_parameters=None, feature_selection=None, model_file=None, data_sequence_column=None, to_lower_case=True, punctuation=None)
- DESCRIPTION:
The TextClassifierTrainer function trains a machine-learning
classifier for text classification and installs the model file on
the ML Engine. The model file can then be input to the function
TextClassifier.
PARAMETERS:
data:
Required Argument.
Specifies the name of the teradataml DataFrame that contains the
documents to use to train the model.
text_column:
Required Argument.
Specifies the name of the column that contains the text of the
training documents.
Types: str
category_column:
Required Argument.
Specifies the name of the column that contains the category of
the training documents.
Types: str
classifier_type:
Optional Argument.
Specifies the classifier type of the model, KNN algorithm or
maximum entropy model.
Default Value: "maxEnt"
Permitted Values: maxEnt, knn
Types: str
classifier_parameters:
Optional Argument.
Applies only if the classifier type of the model is KNN.
Specifies parameters for the classifier.
Permitted Values:
* compress: The value must be in the range (0, 1). The n
training documents are clustered into value*n
groups and the model uses the center of each group as
the feature vector.
For example,
if there are 100 training documents, then
classifier_parameters ("compress:0.6") clusters
them into 60 groups.
* kvalues: The value must be int value in range
[1, max(classes, ceil(sqrt(rows)))],
where:
* 'classes' is number of classes in "data" teradataml
Dataframe.
* 'rows' is number of rows in "data" teradataml
Dataframe.
Value specifies number of nearest neighbors to
consider when deciding label of unseen document.
Function selects best specified value for deciding
label of unseen document.
* power: The value must be int value in range [0, 10]. The
value specifies power to apply to weight corresponding
to each vote considered when deciding label of unseen
document.
Note:
All above listed parameter values can be used when teradataml
is connected to Vantage 1.3, otherwise only 'compress' value
is supported.
Types: str OR list of strs
nlp_parameters:
Optional Argument.
Specifies natural language processing (NLP) parameters for
preprocessing the text data and produce tokens:
* tokenDictFile: token_file - token_file is name of ML
Engine file in which each line contains a phrase, followed
by a space, followed by the token for the phrase (and
nothing else).
* stopwordsFile:stopword_file - stopword_file is the name
of an ML Engine file in which each line contains
exactly one stop word (a word to ignore during tokenization,
such as a, an, or the).
* useStem:{true|false} - Specifies whether the function
stems the tokens. The default value is "false".
* stemIgnoreFile:stem_ignore_file - stem_ignore_file is
the name of an ML Engine file in which each line
contains exactly one word to ignore during stemming.
Specifying this parameter with "useStem:false" causes an
exception.
* useBgram:{ true | false } - Specifies whether the function
uses Bigram, which considers the proximity of adjacent
tokens when analyzing them. The default value is "false".
* language:{ en | zh_CN | zh_TW } - Specifies the language
of the input text - English (en), Simplified Chinese (zh_CN),
or Traditional Chinese (zh_TW). The default value is en.
For the values zh_CN and zh_TW, the function ignores the
parameters useStem and stemIgnoreFile.
Example: nlp_parameters("tokenDictFile:token_dict.txt",
"stopwordsFile:fileName",
"useStem:true",
"stemIgnoreFile:fileName",
"useBgram:true", "language:zh_CN")
Types: str OR list of strs
feature_selection:
Optional Argument.
Specifies the feature selection method, DF (document frequency).
The values min and max must be in the range (0, 1). The function
selects only the tokens that appear in at least min*n documents
and at most max*n documents, where n is the number of training
documents. For example, FeatureSelection ("DF:[0.1:0.9]") causes
the function to select only the tokens that appear in at least
10% but no more than 90% of the training documents. If min
exceeds max, the function uses min as max and max as min.
Types: str
model_file:
Required Argument.
Specifies the name of the model file to be generated.
Types: str
data_sequence_column:
Optional Argument.
Specifies the list of column(s) that uniquely identifies each row of
the input argument "data". The argument is used to ensure
deterministic results for functions which produce results that vary
from run to run.
Types: str OR list of Strings (str)
to_lower_case:
Optional Argument.
Specifies whether to convert input text to lowercase.
Note:
"to_lower_case" argument support is only available when teradataml
is connected to Vantage 1.3 version.
Default Value: True
Types: bool
punctuation:
Optional Argument.
Specifies a regular expression that represents the punctuation
characters to remove from the input text.
Note:
"punctuation" argument support is only available when teradataml
is connected to Vantage 1.3 version.
Types: str
RETURNS:
Instance of TextClassifierTrainer.
Output teradataml DataFrames can be accessed using attribute
references, such as TextClassifierTrainerObj.<attribute_name>.
Output teradataml DataFrame attribute name is:
result
RAISES:
TeradataMlException, TypeError, ValueError
EXAMPLES:
# Load example data.
load_example_data("textclassifiertrainer", "texttrainer_input")
# Create teradataml DataFrame objects.
# The input table contains text of the training documents and the
# category of the training documents.
texttrainer_input = DataFrame.from_table("texttrainer_input")
# Example 1 - The function outputs a binary file with the name
# specified by "model_file" argument.
TextClassifierTrainer_out1 = TextClassifierTrainer(data = texttrainer_input,
text_column = "content",
category_column = "category",
classifier_type = "knn",
classifier_parameters = ["compress:0.9"],
nlp_parameters = ["useStem:true", "stopwordsFile: stopwords.txt"],
feature_selection = "DF:[0.1:0.99]",
model_file = "knn.bin"
)
# Print the result teradataml DataFrame
print(TextClassifierTrainer_out1)
# Example 2 - This example uses parameters 'kvalues' and 'power' as input
# to argument "classifier_parameters" and outputs a binary file with the name
# specified by "model_file" argument.
# Note:
# This Example will work only when teradataml is connected
# to Vantage 1.3 or later.
TextClassifierTrainer_out2 = TextClassifierTrainer(data = texttrainer_input,
text_column = "content",
category_column = "category",
classifier_type = "knn",
classifier_parameters = ["compress:0.9", "kvalues:{1,3}", "power:2"],
nlp_parameters = ["useStem:true", "stopwordsFile: stopwords.txt"],
feature_selection = "DF:[0.1:0.99]",
model_file = "knn.bin",
to_lower_case=False,
punctuation='[a-z]'
)
# Print the result teradataml DataFrame
print(TextClassifierTrainer_out2)
- __repr__(self)
- Returns the string representation for a TextClassifierTrainer class instance.
- get_build_time(self)
- Function to return the build time of the algorithm in seconds.
When model object is created using retrieve_model(), then the value returned is
as saved in the Model Catalog.
- get_prediction_type(self)
- Function to return the Prediction type of the algorithm.
When model object is created using retrieve_model(), then the value returned is
as saved in the Model Catalog.
- get_target_column(self)
- Function to return the Target Column of the algorithm.
When model object is created using retrieve_model(), then the value returned is
as saved in the Model Catalog.
- show_query(self)
- Function to return the underlying SQL query.
When model object is created using retrieve_model(), then None is returned.
|