With the Aster Loader Tool you can load from multiple data files and insert into multiple tables in a single invocation by passing the --map-file or -m option. We refer to this as loading with a map file. All tables are loaded in a single database transaction.
When loading multiple files with the --map-file option, the Loader Tool checks for invalid UTF-8 characters in the first line or in the second line (if there is a header). If an error is found, file loading is stopped but the Loader Tool does not cancel the transaction. This means that if there are other files to be loaded in the map file, the Loader Tool can continue gracefully attempting to load the other files in the same transaction. The Loader Tool will indicate that the file was skipped.
Note that the Loader Tool only checks the first line of the input files or in the second line (if there is a header). Other lines are checked by the server. If errors are found by the server, they are reported and the transaction is aborted.
If there are invalid UTF-8 characters in the input file, you may have to manually convert the data files into UTF-8 encoding. The following is an example of using iconv to convert the encoding:
iconv --from-code=WINDOWS-1252 --to-code=UTF-8 <input.dat>
--output utf8.dat
When loading a very large amount of data, you may choose to create multiple map files that each load their data files using a different loader. This can help speed up the process of loading a large amount of data. Note that a map file loads files serially.