Teradata R Package Function Reference - 16.20 - TextChunker - Teradata R Package

Teradata® R Package Function Reference

prodname
Teradata R Package
vrm_release
16.20
created_date
February 2020
category
Programming Reference
featnum
B700-4007-098K

Description

The TextChunker function divides text into phrases and assigns each phrase a tag that identifies its type.

Usage

  td_text_chunker_mle (
      data = NULL,
      word.column = NULL,
      pos.column = NULL,
      data.sequence.column = NULL,
      data.partition.column = NULL,
      data.order.column = NULL
  )

Arguments

data

Required Argument.
Specifies the tbl_teradata that contains the text to be scanned.

data.partition.column

Required Argument.
Specifies Partition By columns for data.
Values to this argument can be provided as a vector, if multiple columns are used for partition.
Types: character OR vector of Strings (character)

data.order.column

Required Argument.
Specifies Order By columns for data.
Values to this argument can be provided as a vector, if multiple columns are used for ordering.
Types: character OR vector of Strings (character)

word.column

Required Argument.
Specifies the name of the input tbl_teradata column that contains the words to chunk into phrases. Typically, this is the word column of the output tbl_teradata of the td_pos_tagger_mle function.
Types: character

pos.column

Required Argument.
Specifies the name of the input tbl_teradata column that conatins the part-of-speech (POS) tag of words. Typically, this is the pos_tag column of the output tbl_teradata of the td_pos_tagger_mle function.
Types: character

data.sequence.column

Optional Argument.
Specifies the vector of column(s) that uniquely identifies each row of the input argument "data". The argument is used to ensure deterministic results for functions which produce results that vary from run to run.
Types: character OR vector of Strings (character)

Value

Function returns an object of class "td_text_chunker_mle" which is a named list containing Teradata tbl object.
Named list member can be referenced directly with the "$" operator using name: result.

Examples

    # Get the current context/connection
    con <- td_get_context()$connection
    
    # Load example data.
    loadExampleData("text_chunker_example", "posttagger_output")
    
    # Create remote tibble objects.
    posttagger_output <- tbl(con, "posttagger_output")
    
    # Example 1 - This example uses the persisted output of td_pos_tagger_mle
    # as input.
    text_chunker_out1 <- td_text_chunker_mle(data=posttagger_output,
                                data.partition.column='paraid',
                                data.order.column=c('paraid','word_sn'),
                                word.column='word',
                                pos.column='pos_tag',
                                data.sequence.column='paraid')
                                
    # Load example data.
    loadExampleData("pos_tagger_example", "paragraphs_input")
    
    # Create remote tibble objects.
    paragraphs_input <- tbl(con, "paragraphs_input")
    
    # Example 2 - This example uses output of td_sentence_extractor_mle and
    # td_pos_tagger_mle as input.
    td_sentence_extractor_out <- td_sentence_extractor_mle(data = paragraphs_input,
                                                         text.column = "paratext",
                                                         accumulate = "paraid")
  
    sentenceextractor_out <- td_sentence_extractor_out$result 
  
    pos_tagger_out <- td_pos_tagger_mle(data=sentenceextractor_out,
                                      text.column='sentence',
                                      accumulate='sentence_sn')
  
    text_chunker_out2 <- td_text_chunker_mle(data=pos_tagger_out$result,
                                data.partition.column='word_sn',
                                data.order.column='word_sn',
                                word.column='word',
                                pos.column='pos_tag',
                                data.sequence.column='word_sn')