| | |
Methods defined here:
- __init__(self, data=None, formula='normal', data_sequence_column=None, data_partition_column=None, data_order_column=None)
- DESCRIPTION:
TF (Term Frequency) is used in conjuction with function TF-IDF
(Term Frequency - Inverse Document Frequency).
TF-IDF is a technique for weighting words in a document. The
resulting weights can be used together in a vector space model as
input for various document clustering or classification algorithms.
To compute TF-IDF values, the TFIDF function relies on the TF
function, which computes the TF value of the input.
PARAMETERS:
data:
Required Argument.
Specifies the input teradataml DataFrame that contains the document id, the
term, and optionally the count or number of appearances of
the term in the document, in that order.
data_partition_column:
Required Argument.
Specifies Partition By columns for data.
Values to this argument can be provided as list, if multiple columns
are used for ordering.
Types: str OR list of Strings (str)
data_order_column:
Optional Argument.
Specifies Order By columns for data.
Values to this argument can be provided as a list, if multiple
columns are used for ordering.
Types: str OR list of Strings (str)
formula:
Optional Argument.
Specifies the formula for calculating the term frequency (tf) of term
t in document d:
• normal: Normalized frequency (default):
tf (t, d) = f ((t, d) / sum { w,w ? d }. This value is rf
divided by the number of terms in the document.
• bool: Boolean frequency: tf ((t, d) = 1 if t occurs in d;
otherwise, tf ((t, d) = 0.
• log: Logarithmically-scaled frequency:
tf ((t, d) = log (f ((t, d) + 1) where f ((t, d) is the
number of times t occurs in d (that is, the raw frequency,
rf).
• augment: Augmented frequency (to prevent bias towards
longer documents): tf ((t, d) = 0.5 + (0.5 × f ((t, d) /
max { f (w, d) : w ? d }). This value is rf divided by
the maximum raw frequency of any term in the document.
Note: When using the output of a previous run of the TFIDF
function on a training document set to predict TFIDF scores
on an input document set, use the same "formula" value for the
input document set that you used for the training document set.
Default Value: "normal"
Permitted Values: bool, log, augment, normal
Types: str
data_sequence_column:
Optional Argument.
Specifies the list of column(s) that uniquely identifies each row of
the input argument "data". The argument is used to ensure
deterministic results for functions which produce results that vary
from run to run.
Types: str OR list of Strings (str)
RETURNS:
Instance of TF.
Output teradataml DataFrames can be accessed using attribute
references, such as TFObj.<attribute_name>.
Output teradataml DataFrame attribute name is:
result
RAISES:
TeradataMlException
EXAMPLES:
# Load the data to run the example.
load_example_data("TF","tfidf_input1")
# Create teradataml DataFrame.
tfidf_input1 = DataFrame.from_table("tfidf_input1")
# Example 1 -
tf_result = TF(data=tfidf_input1,
data_partition_column='docid',
formula='normal',
data_sequence_column='docid'
)
# Print the result DataFrame
print(tf_result.result)
- __repr__(self)
- Returns the string representation for a TF class instance.
|