There are no specific security considerations required for this feature. If the external large language model URL is inaccessible, work with your Operations team and network team to allowlist the URL to establish a connection with the model. Enable the access to the LLM endpoint and remove the access when the access is complete.
| API Type | Endpoint for enabling |
|---|---|
| AWS | *.amazonaws.com |
| Azure | *.openai.azure.com |
| Google Cloud Provider | gcp_location + -aiplatform.googleapis.com |
| NIM | DNS or IP of GPU nodes hosting NIM server |
| LiteLLM | Endpoint for LiteLLM server |