Description
TF (Term Frequency) is used in conjuction with function TF-IDF (Term Frequency
- Inverse Document Frequency). TF-IDF is a technique for weighting words in a
document. The resulting weights can be used together in a vector space model
as input for various document clustering or classification algorithms. To
compute TF-IDF values, the TF_IDF function relies on the TF function, which
computes the TF value of the input.
Usage
td_tf_mle (
data = NULL,
formula = "normal",
data.sequence.column = NULL,
data.partition.column = NULL,
data.order.column = NULL
)
Arguments
data |
Required Argument.
Specifies the input tbl_teradata that contains the document ID and the term.
|
data.partition.column |
Required Argument.
Specifies Partition By columns for "data".
Values to this argument can be provided as a vector, if multiple
columns are used for partition.
Types: character OR vector of Strings (character)
|
data.order.column |
Optional Argument.
Specifies Order By columns for "data".
Values to this argument can be provided as a vector, if multiple
columns are used for ordering.
Types: character OR vector of Strings (character)
|
formula |
Optional Argument.
Specifies the formula for calculating the term frequency (tf) of term
t in document d.
Four formulas are supported:
normal: Normalized frequency (default): tf (t, d)
= f ((t, d) / sum w where w belongs to d.
This value is rf divided by the number
of terms in the document.
bool: Boolean frequency: tf ((t, d) = 1
if t occurs in d; otherwise, tf ((t, d) = 0.
log:
Logarithmically-scaled frequency: tf ((t, d) = log (f ((t, d) + 1)
where f ((t, d) is the number of times t occurs in d (that is, the
raw frequency, rf).
augment: Augmented frequency (to prevent bias
towards longer documents): tf ((t, d) = 0.5 + (0.5 x f ((t, d) / max
f (w, d) ) where w belongs to d. This value is rf divided by the maximum raw
frequency of any term in the document.
Default Value: "normal"
Permitted Values: bool, log, augment, normal
|
data.sequence.column |
Optional Argument.
Specifies the vector of column(s) that uniquely identifies each row
of the input argument "data". The argument is used to ensure
deterministic results for functions which produce results that vary
from run to run.
Types: character OR vector of Strings (character)
|
Value
Function returns an object of class "td_tf_mle" which is a named list
containing object of class "tbl_teradata".
Named list member can be referenced directly with the "$" operator
using the name: result.
Examples
# Get the current context/connection
con <- td_get_context()$connection
# Load example data.
loadExampleData("tf_example", "tfidf_input1")
# Create object(s) of class "tbl_teradata".
tfidf_input1 <- tbl(con, "tfidf_input1")
# Example 1 - Calculate TF values using input tbl_teradata containing tokens and their count in
# all documents.
tf_out <- td_tf_mle(tfidf_input1, data.partition.column="docid")