Responses may vary based on the selected language model. The examples provided in this guide are for demonstration purposes only and do not indicate the performance of teradatagenai text analytics functions. Any variations or limitations in response accuracy are due to the selected language model.
Import the required packages and set up the input data
import os
import teradatagenai
from teradataml import DataFrame
from teradatagenai import TeradataAI, TextAnalyticsAI, load_data
load_data('employee', 'employee_data')
df = DataFrame('employee_data')
Set up 'employee_data' table columns
The TextAnalyticsAI function examples conduct text analytics on various columns of table ‘employee_data’. You can locate the corresponding csv file in the example-data directory within the package, and use any any other datasets.
df_articles = df.select(["employee_id", "employee_name", "articles"]) df_classify_articles = df.select(["employee_id", "articles"]) df_reviews = df.select(["employee_id", "employee_name", "reviews"]) df_quotes = df.select(["employee_id", "employee_name", "quotes"]) df_employeeData = df.select(["employee_id", "employee_name", "employee_data"])
Set up the TeradataAI class with all the required credentials and model specifications
The following sections illustrate examples for AWS Bedrock, Azure AI, Google Gemini, and Hugging Face.
Add the parameters according to Download and Set up the Environment.
Initialize the TeradataAI class See Examples: How to use TeradataAI.
Examples
os.environ["AWS_DEFAULT_REGION"] = "us-west-2"
os.environ["AWS_ACCESS_KEY_ID"] = "<Enter AWS Access Key ID>"
os.environ["AWS_SECRET_ACCESS_KEY"] = "<Enter AWS Secret Key>"
os.environ["AWS_SESSION_TOKEN"] = "<Enter AWS Session key>"
llm_aws = TeradataAI(api_type = "aws",
model_name = "anthropic.claude-v2")
obj_aws = TextAnalyticsAI(llm=llm_aws)
os.environ['AZURE_OPENAI_API_KEY'] = <provide your azure AI API key>
os.environ['AZURE_OPENAI_ENDPOINT'] = <provide your azure AI engine name>
os.environ['AZURE_OPENAI_API_VERSION'] = <provide your azure AI version>
os.environ['AZURE_OPENAI_DEPLOYMENT_ID'] = <provide your azure AI deployment id>
llm_azure = TeradataAI(api_type="azure",
model_name="gpt-3.5-turbo")
obj_azure = TextAnalyticsAI(llm=llm_azure)
os.environ['GOOGLE_CLOUD_PROJECT'] = <GCP project name>
os.environ['GOOGLE_CLOUD_REGION'] = <GCP cloud region>
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = <GCP access token>
llm_gcp = TeradataAI(api_type="gcp",
model_name="gemini-1.5-pro-001")
obj_gcp = TextAnalyticsAI(llm=llm_gcp)
model_name = 'sentence-transformers/all-MiniLM-L6-v2'
model_args = {'transformer_class': 'AutoModelForTokenClassification', 'task' : 'token-classification'}
llm_hugging_face = TeradataAI(api_type = "hugging_face",
model_name = model_name,
model_args = model_args)
obj = TextAnalyticsAI(llm=llm_hugging_face)
os.environ["NVIDIA_API_KEY"] = "<NVIDIA NIM API key>"
llm_nim = TeradataAI(api_type = "nim",
api_base = "<nim base url>",
model_name = "meta/llama-3.1-8b-instruct")
obj_nim = TextAnalyticsAI(llm=llm_nim)
Create LLM endpoint for "api_type" as 'onnx' for the ONNX model "bge-m3" with "model_id" as 'td-bge-m3' already stored in the default table.
from teradatagenai import TeradataAI
llm = TeradataAI(api_type = "onnx",
model_name = "bge-m3",
model_id = "td-bge-m3")
obj = TextAnalyticsAI(llm=llm_onnx)
configure.byom_install_location = "mldb"