Optional Syntax Elements for TD_Ngramsplitter - Teradata VantageCloud Lake
Lake - Analyze Your Data with ClearScape Analytics™
- Deployment
- VantageCloud
- Edition
- Lake
- Product
- Teradata VantageCloud Lake
- Release Number
- Published
- February 2025
- ft:locale
- en-US
- ft:lastEdition
- 2026-02-20
- dita:mapPath
- tcl1683670667798.ditamap
- dita:ditavalPath
- pny1626732985837.ditaval
- dita:id
- tcl1683670667798
- Delimiter
- Specify the character or string that separates words in the input text.
- Default: ' ' (space)
- OverLapping
- Specify whether the function allows overlapping n-grams.
- Default: 'true' (Each word in each sentence starts an n-gram, if enough words follow it in the same sentence to form a whole n-gram of the specified size. For information on sentences, see the Reset syntax element description.)
- ConvertToLowerCase
- Specify whether the function converts all letters in the input text to lowercase.
- Default: 'true'
- Reset
- Specify, in a string, the characters that can end a sentence. At the end of a sentence, the function discards any partial n-grams and searches for the next n-gram at the beginning of the next sentence. An n-gram cannot span sentences.
- Default: '.,?!'
- Punctuation
- Specify, in a string, the punctuation characters for the function to remove before evaluating the input text.
- Punctuation characters can be from both Unicode and Latin character sets.
- Default: '`~#^&*()-'
- OutputTotalGramCount
- Specify whether the function returns the total number of n-grams in the document (that is, in the row) for each length n specified in the Grams syntax element. If you specify 'true', the TotalCountColName syntax element determines the name of the output table column that contains these totals.
- The total number of n-grams is not necessarily the number of unique n-grams.
- Default: 'false'
- TotalCountColName
- Specify the name of the output table column that appears if the value of the OutputTotalGramCount syntax element is 'true'.
- Default: 'totalcnt'
- Accumulate
- Specify the names of the input table columns to copy to the output table for each n-gram. These columns cannot have the same names as those specified by the syntax elements NGramColName, GramLengthColName, and TotalCountColName.
- Default: All input columns for each n-gram
- NGramColName
- Specify the name of the output table column that is to contain the created n-grams.
- Default: 'ngram'
- GramLengthColName
- Specify the name of the output table column that is to contain the length of n-gram (in words).
- Default: 'n'
- FrequencyColName
- Specify the name of the output table column that is to contain the count of each unique n-gram (that is, the number of times that each unique n-gram appears in the document).
- Default: 'frequency'