1.1 - 8.10 - HMMDecoder Example: Part-of-Speech Tagging - Teradata Vantage

Teradata Vantage™ - Machine Learning Engine Analytic Function Reference

Product
Teradata Vantage
Release Number
1.1
8.10
Release Date
October 2019
Content Type
Programming Reference
Publication ID
B700-4003-079K
Language
English (United States)

In this example, the parts of speech correspond to the hidden states of the HMM function, A (adjective) and N (noun).

Input

  • InitialStateProbs: initial
  • TransitionProbs: state_transition
  • EmissionProbs: emission
  • ObservationTable: phrases, a set of phrases whose parts of speech are unknown
InitialStateProbs: initial
model tag probability
1 A 0.25
1 N 0.75
TransitionProbs: state_transition
model from_tag to_tag probability
1 A A 0
1 A N 1
1 N A 0.5
1 N N 0.5
EmissionProbs: emission
model tag word probability
1 A clown 0
1 N clown 0.4
1 A crazy 1
1 N crazy 0
1 A killer 0
1 N killer 0.3
1 A problem 0
1 N problem 0.3
ObservationTable: phrases
model phrase_id word
1 1 clown
1 1 crazy
1 1 killer
1 1 problem
1 2 nice
1 2 weather

SQL Call

SELECT * FROM HMMDecoder(
  ON initial AS InitialStateProbs PARTITION BY model 
    ORDER BY model, tag
  ON state_transition AS TransitionProbs PARTITION BY model
  ON emission AS EmissionProbs PARTITION BY model
  ON phrases AS ObservationTable PARTITION BY model
    ORDER BY model, phrase_id ASC
  USING
  InitStateModelColumn('model')
  InitStateColumn('tag')
  InitStateProbColumn('probability')
  TransAttributeColumn('model')
  TransFromStateColumn('from_tag')
  TransToStateColumn('to_tag')
  TransProbColumn('probability')
  EmitModelColumn('model')
  EmitStateColumn('tag')
  EmitObsColumn('word')
  EmitProbColumn('probability')
  ModelColumn('model')
  SeqColumn('phrase_id')
  ObsColumn('word')
) AS dt ORDER BY 1, 2, 3;

Output

 model phrase_id word    tag 
 ----- --------- ------- --- 
 1     1         clown   n  
 1     1         crazy   a  
 1     1         killer  n  
 1     1         problem n  
 1     2         nice    a  
 1     2         weather a

Download a zip file of all examples and a SQL script file that creates their input tables from the attachment in the left sidebar.