When dealing with character data, it is helpful to be able to determine what actual data type the data could be stored in within the database. The Text Field Analyzer can distinguish whether a field is a numeric type, a date, a time, a timestamp, or character data. Text field analysis can be applied to any type of character data. Non-character data types go unprocessed and are passed along to the output just as they are defined in the input table.
Given a table name and the name of a column, the Text Field Analyzer provides a series of tests to distinguish what the correct underlying type should be.
- The first test tests the minimum and maximum values of a column to determine the type of data.
- The next test retrieves a small sample of data for each column and again accesses the type.
- The final test classifies numeric fields into a more specific category. For example, a field considered a FLOAT type after the first two tests might really be a DECIMAL type with two decimal places. In the final test, a date type is validated to make sure all values in that column are truly dates.