Teradata Package for R Function Reference | 17.00 - 17.00 - TF - Teradata Package for R

Teradata® Package for R Function Reference
Product
Teradata Package for R
Release Number
17.00
Release Date
July 2021
Content Type
Programming Reference
Publication ID
B700-4007-090K
Language
English (United States)

Description

TF (Term Frequency) is used in conjuction with function TF-IDF (Term Frequency - Inverse Document Frequency). TF-IDF is a technique for weighting words in a document. The resulting weights can be used together in a vector space model as input for various document clustering or classification algorithms. To compute TF-IDF values, the TF_IDF function relies on the TF function, which computes the TF value of the input.

Usage

  td_tf_mle (
      data = NULL,
      formula = "normal",
      data.sequence.column = NULL,
      data.partition.column = NULL,
      data.order.column = NULL
  )

Arguments

data

Required Argument.
Specifies the input tbl_teradata that contains the document ID and the term.

data.partition.column

Required Argument.
Specifies Partition By columns for "data".
Values to this argument can be provided as a vector, if multiple columns are used for partition.
Types: character OR vector of Strings (character)

data.order.column

Optional Argument.
Specifies Order By columns for "data".
Values to this argument can be provided as a vector, if multiple columns are used for ordering.
Types: character OR vector of Strings (character)

formula

Optional Argument.
Specifies the formula for calculating the term frequency (tf) of term t in document d.
Four formulas are supported:

  • normal: Normalized frequency (default): tf (t, d) = f ((t, d) / sum w where w belongs to d. This value is rf divided by the number of terms in the document.

  • bool: Boolean frequency: tf ((t, d) = 1 if t occurs in d; otherwise, tf ((t, d) = 0.

  • log: Logarithmically-scaled frequency: tf ((t, d) = log (f ((t, d) + 1) where f ((t, d) is the number of times t occurs in d (that is, the raw frequency, rf).

  • augment: Augmented frequency (to prevent bias towards longer documents): tf ((t, d) = 0.5 + (0.5 x f ((t, d) / max f (w, d) ) where w belongs to d. This value is rf divided by the maximum raw frequency of any term in the document.
    Default Value: "normal"
    Permitted Values: bool, log, augment, normal

data.sequence.column

Optional Argument.
Specifies the vector of column(s) that uniquely identifies each row of the input argument "data". The argument is used to ensure deterministic results for functions which produce results that vary from run to run.
Types: character OR vector of Strings (character)

Value

Function returns an object of class "td_tf_mle" which is a named list containing object of class "tbl_teradata".
Named list member can be referenced directly with the "$" operator using the name: result.

Examples

    # Get the current context/connection
    con <- td_get_context()$connection
    
    # Load example data.
    loadExampleData("tf_example", "tfidf_input1")
    
    # Create object(s) of class "tbl_teradata".
    tfidf_input1 <- tbl(con, "tfidf_input1")

    # Example 1 - Calculate TF values using input tbl_teradata containing tokens and their count in
    # all documents.
    tf_out <- td_tf_mle(tfidf_input1, data.partition.column="docid")