Assume that the following foreign table has been created on CSV data in Amazon S3 bucket:
CREATE FOREIGN TABLE riverflowcsv
, EXTERNAL SECURITY DEFINER TRUSTED AUTH_OBJECT
(
Location VARCHAR(2048) CHARACTER SET UNICODE CASESPECIFIC,
PAYLOAD DATASET INLINE LENGTH 64000 STORAGE FORMAT CSV)
USING
(
LOCATION('/s3/s3.amazonaws.com/td-usgs/CSVDATA/')
PATHPATTERN ('$data/$siteno/$year/$month/$day')
);
- Create a dataframe on foreign table.
# Create dataframe on foreign table which contains data in CSV format on S3 >>> riverflow = DataFrame.from_table("riverflowcsv") - Display the columns in the DataFrame.
# As seen from create table statement, table has two columns 'Location' and 'Payload' >>> riverflow.columns
['Location', 'Payload']
- Check the types of the columns.
# Let's check their types >>> riverflow.dtypes
Location str Payload str
- Print the content of the table.
# Let's print the DataFrame. # Equivalent SQL: # SELECT * FROM riverflow >>> riverflow.head().to_pandas()
Location PAYLOAD 0 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 1 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 2 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 3 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 4 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 5 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 6 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 7 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 8 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi... 9 /S3/s3.amazonaws.com/td-usgs/CSVDATA/09380000/... Temp,Flow,site_no,datetime,Conductance,Precipi...