These examples demonstrate how to categorize text in articles composed by various employees stored in a TeradataML DataFrame. Refer to TextAnalyticsAI Example Setup for the prerequisite steps.
obj_aws.classify(column="articles",data=data, accumulate="articles",
labels=["sports", "politics", "entertainment", "technology"], multi_label=True)
Output:
| articles | Labels |
|---|---|
| Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing | [technology] |
| Climate change poses significant challenges globally affecting weather patterns ecosystems and human | [nan] |
| The recent 2020 United States election have shown a shift in the political landscape With more youn | [politics] |
| The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i | [sports] |
| The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o | [environment] |
obj_azure.classify(column="articles",data=data, accumulate="articles",
labels=["sports", "politics", "entertainment", "technology"], multi_label=True)
Output:
| articles | Labels |
|---|---|
| Climate change poses significant challenges globally affecting weather patterns ecosystems and human | [sports, politics, entertainment, technology] |
| The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i | [sports] [technology] |
| The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o | [sports, nan, nan, nan] |
| The recent 2020 United States election have shown a shift in the political landscape With more youn | [sports, politics] |
| Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing | [sports, nan, nan, technology] |
Classify the text in the 'articles' column of a teradataml DataFrame using Google Gemini. The text for classification is provided as a column name within the teradataml DataFrame. You can specify a list of labels in the 'labels' argument for the grouped label classification process.
obj_gcp.classify(column="articles",data=data, accumulate="articles",
labels=["sports", "politics", "entertainment", "technology"])
Output:
| articles | Labels |
|---|---|
| Climate change poses significant challenges globally affecting weather patterns ecosystems and human | ['nan] |
| The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i | ['sports'] |
| The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o | ['politics'] |
| The recent 2020 United States election have shown a shift in the political landscape With more youn | ['politics'] |
| Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing | [technology] |
Example setup: Import required packages and set up input data
>>> import os
>>> import teradatagenai
>>> from teradatagenai import TeradataAI, TextAnalyticsAI, load_data
>>> from teradataml import DataFrame
>>> load_data('employee', 'employee_data')
>>> data = DataFrame('employee_data')
>>> df_classify_articles = data.select(["employee_id", "articles"])
Complete the example setup requirements.
Example 1: Perform classification in the 'articles' column of a teradataml DataFrame using Hugging Face model 'facebook/bart-large-mnli'
A list of labels are passed and the model classifies articles according to the labels.
Create an LLM endpoint.
>>> model_name = 'facebook/bart-large-mnli'
>>> model_args = {'transformer_class': 'AutoModelForSequenceClassification',
'task' : 'zero-shot-classification'}
>>> ues_args = {'env_name' : 'demo_env'}
>>> llm = TeradataAI(api_type = "hugging_face",
model_name = model_name,
model_args = model_args,
ues_args = ues_args)
Create a TextAnalyticsAI object.
>>> obj = TextAnalyticsAI(llm=llm)
>>> obj.classify("articles",
df_classify_articles,
labels = ["Medical", "Hospitality", "Healthcare",
"historical-news", "Games",
"Environment", "Technology",
"Games"],
delimiter = "#")
Output:
| text | Labels |
|---|---|
| The recent 2020 United States election have shown a shift in the political landscape With more youn | {'sequence': 'The recent 2020 United States election have shown a shift in the political landscape With more youn', 'labels': ['historicalNews', 'Games', 'hospital', 'Environment', 'technology', 'Medical', 'healthcare'], 'scores': [0.022191415727133852, 0.18781757354736328, 0.18410125374794006, 0.157247394323349, 0.08474085479974747, 0.08211186528205872, 0.0820668488740921]} |
| The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o | {'sequence': 'The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o', 'labels': ['Environment', 'historicalNews', 'Games', 'Medical', 'hospital', 'technology', 'healthcare'], 'scores': [0.8475484848022461, 0.07286490499973297, 0.03330129012465477, 0.014091618359088898, 0.012927068397402763, 0.010838040150702, 0.008428608998656273]} |
| Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing | {'sequence': 'Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing', 'labels': ['technology', 'Environment', 'Games', 'historicalNews', 'Medical', 'hospital', 'healthcare'], 'scores': [0.6401532292366028, 0.21753348410129547, 0.045335788279771805, 0.04430738463997841, 0.024098196998238564, 0.0182021614164114, 0.010369792580604553]} |
| The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i | {'sequence': 'The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i', 'labels': ['Games', 'historicalNews', 'Environment', 'Medical', 'technology', 'healthcare', 'hospital'], 'scores': [0.7579749822616577, 0.10658492147922516, 0.04014146327972412, 0.029473936185240746, 0.023510770872235298, 0.022025825455784798, 0.020288145169615746]} |
| Climate change poses significant challenges globally affecting weather patterns ecosystems and human | {'sequence': 'Climate change poses significant challenges globally affecting weather patterns ecosystems and human', 'lables': ['Environment', 'historicalNews', 'Games', 'hospital', 'Medical', technology', 'healthcare'], 'scores': [0.7869754433631897, 0.08319485187530518, 0.06480303406715393, 0.024431703612208366, 0.015910524874925613, 0.013035607524216175, 0.011648806743323803]} |
Example 2: Extend Example 1 to use user defined script for inferencing
>>> base_dir = os.path.dirname(teradatagenai.__file__)
>>> classify_script = os.path.join(base_dir, 'example-data', 'classify_text.py')
>>> obj.classify("articles",
df_classify_articles,
labels = ["Medical", "Hospitality", "Healthcare",
"historical-news", "Games",
"Environment", "Technology",
"Games"],
script = classify_script,
delimiter = "#")
Output:
| text | Labels |
|---|---|
| Climate change poses significant challenges globally affecting weather patterns ecosystems and human | Environment |
| Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing | Technology |
| The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i | Games |
| The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o | Environment |
| The recent 2020 United States election have shown a shift in the political landscape With more youn | historical-news |
obj_nim.classify(column="articles",data=data, accumulate="articles",
labels=["sports", "politics", "entertainment", "technology"])
Output:
| articles | Labels |
|---|---|
| Renewable energy sources such as solar wind and hydroelectric power play a crucial role in reducing | [\technology\ \sports\ |
| Climate change poses significant challenges globally affecting weather patterns ecosystems and human | [\technology\ \politics\ |
| The recent 2020 United States election have shown a shift in the political landscape With more youn | [\politics |
| The 2020 Tokyo Olympics was postponed to 2021 due to the COVID-19 pandemic. This was the first time i | [\sports\ \technology\ |
| The 2019 Amazon rainforest wildfires were a severe environmental crisis The fires burned thousands o | [\technology\ \entertainment\ \politics\ \environment |