With DataHub running, open the DataHub GUI and login. In this example this is running at localhost:9002
- Start the new connection wizard by clicking on the ingestion plug icon and then selecting "Create new source"
-
Scroll the list of available sources and select Custom Source. This will allow you to paste in a recipe to configure the connection to Teradata.
-
A recipe is needed to configure the connection to Teradata and define the options required such as whether to capture table and column lineage, profile the data or retrieve usage statistics. Below is a simple recipe to get you started. The host, username and password should be changed to match your environment.
pipeline_name: my-teradata-ingestion-pipeline
source:
type: teradata
config:
host_port: "myteradatainstance.teradata.com:1025"
username: myuser
password: mypassword
#database_pattern:
# allow:
# - "my_database"
# ignoreCase: true
include_table_lineage: true
include_usage_statistics: false
stateful_ingestion:
enabled: true
Pasting the recipe into the window should look like this:
-
Click Next and then setup the required schedule.
-
Click Next to Finish Up and give the connection a name. Click Advanced so that the correct CLI version can be set. Set this CLI value to same as the DataHub version which you can find by typing datahub version in the command line. In this example the DataHub version is 1.6.0 and so the CLI value is set to 1.6.0.
- Click Save & Run to save the new source and run it immediately. You can also click Save to save the source and run it later.
Clicking on "Success" after a sucessful execution will bring up a dialogue similar to this one where you can see the Databases, Tables and Views that have been ingested into DataHub.
- The metadata can now be explored in the GUI by browsing:
- DataSets provides a list of the datasets (tables and views) loaded
- Entities captured from the database
- Schema of an entity showing column/field names, data types and usage if it has been captured
- Lineage providing a visual representation of how data is linked between tables and views