Description
TF (Term Frequency) is used in conjuction with function TF-IDF (Term Frequency
- Inverse Document Frequency). TF-IDF is a technique for weighting words in a
document. The resulting weights can be used together in a vector space model
as input for various document clustering or classification algorithms. To
compute TF-IDF values, the TF_IDF function relies on the TF function, which
computes the TF value of the input.
Usage
td_tf_mle (
data = NULL,
formula = "normal",
data.sequence.column = NULL,
data.partition.column = NULL,
data.order.column = NULL
)
Arguments
data |
Required Argument.
The input tbl_teradata that contains the document id and the term.
|
data.partition.column |
Required Argument.
Partition By columns for data.
Values to this argument can be provided as vector, if multiple
columns are used for partition.
Types: character OR vector of Strings (character)
|
data.order.column |
Optional Argument.
Order By columns for data.
Values to this argument can be provided as vector, if multiple
columns are used for ordering.
Types: character OR vector of Strings (character)
|
formula |
Optional Argument.
Specifies the formula for calculating the term frequency (tf) of term
t in document d.
Four formulas are supported:
normal: Normalized frequency (default): tf (t, d)
= f ((t, d) / sum w where w belongs to d.
This value is rf divided by the number
of terms in the document.
bool: Boolean frequency: tf ((t, d) = 1
if t occurs in d; otherwise, tf ((t, d) = 0.
log:
Logarithmically-scaled frequency: tf ((t, d) = log (f ((t, d) + 1)
where f ((t, d) is the number of times t occurs in d (that is, the
raw frequency, rf).
augment: Augmented frequency (to prevent bias
towards longer documents): tf ((t, d) = 0.5 + (0.5 x f ((t, d) / max
f (w, d) ) where w belongs to d. This value is rf divided by the maximum raw
frequency of any term in the document.
Default Value: "normal"
Permitted Values: bool, log, augment, normal
|
data.sequence.column |
Optional Argument.
Specifies the vector of column(s) that uniquely identifies each row
of the input argument "data". The argument is used to ensure
deterministic results for functions which produce results that vary
from run to run.
Types: character OR vector of Strings (character)
|
Value
Function returns an object of class "td_tf_mle" which is a named list
containing Teradata tbl object.
Named list member can be referenced directly with the "$" operator
using name: result
Examples
# Get the current context/connection
con <- td_get_context()$connection
# Load example data
loadExampleData("tf_example", "tfidf_input1")
# Create remote tibble objects.
tfidf_input1 <- tbl(con, "tfidf_input1")
#TF values using inputs
tf_out <- td_tf_mle(tfidf_input1, data.partition.column="docid")