TextTokenizer Syntax - Teradata Vantage

Machine Learning Engine Analytic Function Reference

Product
Teradata Vantage
Release Number
8.10
1.1
Published
October 2019
Language
English (United States)
Last Update
2019-12-31
dita:mapPath
ima1540829771750.ditamap
dita:ditavalPath
jsj1481748799576.ditaval
dita:id
B700-4003
lifecycle
previous
Product Category
Teradata Vantageā„¢

Version 3.8

SELECT * FROM TextTokenizer (
  ON { table | view | (query) } PARTITION BY ANY
  [ ON dict_table AS Dict DIMENSION ]
  USING
  TextColumn ('text_column') ]
  [ InputLanguage ({ 'en' | 'zh_CN' | 'zh_TW' | 'jp' | }) ]
  [ InputModelFile ('input_model_file') ]
  [ OutputDelimiter ('delimiter') ]
  [ OutputByWord ({'true'|'t'|'yes'|'y'|'1'|'false'|'f'|'no'|'n'|'0'}) ]
  [ Accumulate ({ 'accumulate_column' | accumulate_column_range }[,...]) ]
  [ UserDictionaryFile ('user_dictionary_file') ]
) AS alias;