In this example, the parts of speech correspond to the hidden states of the HMM function, A (adjective) and N (noun).
Input
- InitialStateProbs: initial
- TransitionProbs: state_transition
- EmissionProbs: emission
- ObservationTable: phrases, a set of phrases whose parts of speech are unknown
InitialStateProbs: initial| model |
tag |
probability |
|---|
| 1 |
A |
0.25 |
| 1 |
N |
0.75 |
TransitionProbs: state_transition| model |
from_tag |
to_tag |
probability |
|---|
| 1 |
A |
A |
0 |
| 1 |
A |
N |
1 |
| 1 |
N |
A |
0.5 |
| 1 |
N |
N |
0.5 |
EmissionProbs: emission| model |
tag |
word |
probability |
|---|
| 1 |
A |
clown |
0 |
| 1 |
N |
clown |
0.4 |
| 1 |
A |
crazy |
1 |
| 1 |
N |
crazy |
0 |
| 1 |
A |
killer |
0 |
| 1 |
N |
killer |
0.3 |
| 1 |
A |
problem |
0 |
| 1 |
N |
problem |
0.3 |
ObservationTable: phrases| model |
phrase_id |
word |
|---|
| 1 |
1 |
clown |
| 1 |
1 |
crazy |
| 1 |
1 |
killer |
| 1 |
1 |
problem |
| 1 |
2 |
nice |
| 1 |
2 |
weather |
SQL Call
SELECT * FROM HMMDecoder(
ON initial AS InitialStateProbs PARTITION BY model
ORDER BY model, tag
ON state_transition AS TransitionProbs PARTITION BY model
ON emission AS EmissionProbs PARTITION BY model
ON phrases AS ObservationTable PARTITION BY model
ORDER BY model, phrase_id ASC
USING
InitStateModelColumn('model')
InitStateColumn('tag')
InitStateProbColumn('probability')
TransAttributeColumn('model')
TransFromStateColumn('from_tag')
TransToStateColumn('to_tag')
TransProbColumn('probability')
EmitModelColumn('model')
EmitStateColumn('tag')
EmitObsColumn('word')
EmitProbColumn('probability')
ModelColumn('model')
SeqColumn('phrase_id')
ObsColumn('word')
) AS dt ORDER BY 1, 2, 3;
Output
model phrase_id word tag
----- --------- ------- ---
1 1 clown n
1 1 crazy a
1 1 killer n
1 1 problem n
1 2 nice a
1 2 weather a
Download a zip file of all examples and a SQL script file that creates their input tables.