Release Notes
Syntax and Error Messages
Note: Existing BTEQ jobs that use invalid syntax may need to be corrected in order to continue to function as expected. Contact the Global Support Center (GSC) for assistance.
Beginning with 15.00, BTEQ exhibits much stricter syntax‑checking behavior for BTEQ commands. In addition to producing errors for invalid syntax, unless a BTEQ informational message indicates otherwise, the prior settings state are always preserved.
The changes made establish the infrastructure necessary for providing expanded error handling capabilities in future versions of BTEQ.
In order to avoid compromising existing scripts, batch mode is lenient about some types of syntax errors. However, interactive mode is not lenient. The syntax of existing BTEQ commands is always subject to future expansion. Therefore, each new BTEQ version batch mode will be less lenient. Completely correct syntax should always be used in order to safeguard scripts.
The generation of error messages has been made cleaner. Only one error message is produced for any one particular syntax violation. Superfluous warnings were removed and in many cases the error message text given is more meaningful. In addition to preserving prior state (PSP), BTEQ no longer silently changes the state of one command's setting as a result of using another so that the current settings' state can be completely understood by users at any time.
All BTEQ commands were addressed by the changes and several new features were also able to be introduced in the process. Most notably, multi-step command-step instruction support, a QUIT option for the RUN command and a RESET option for the IMPORT command.
The topics outline the key changes and where there may be impact to existing scripts.
New Features
Impact: With the exception of #3, which was added to address possible impact related to invalid IMPORT command syntax use, all of these features are new optional capabilities that will have no impact to existing scripts.
1 BTEQ input is comprised of instructions. Previously, each instruction could be either a BTEQ command step or an SQL request step. Although a step can span multiple input lines, only use of one step per instruction was supported. For example, the excerpt below is comprised of four one-step instructions:.SET SIDETITLES ON.SET OMIT ON 2.SET SUPPRESS ON 5SELECT * FROM foobar;With 15.00 BTEQ, users can now use multiple BTEQ command steps per instruction. So the above can now be written as:.SET SIDETITLES ON; .SET OMIT ON ; .SET SUPPRESS ON 5SELECT * FROM foobar;The most obvious benefit is that fewer batch mode result blocks will be created making the resulting stdout/SYSPRINT file smaller. The results that are produced may be easier to read. In the case above, there will be two rather than four result blocks. It also means that multiple BTEQ commands can now be ECHOed back to BTEQ in one trip which, depending on use case, could boost performance.There are a few commands which must still be entered as a single-step command instruction or as a last step. For example, LOGON and OS commands can only be a last step within a multi-step instruction.
2 By default, RUN files are treated as an extension of stdin/SYSIN and the steps they contain are sequentially executed. The only documented way to skip to the end of a RUN file has been to use a GOTO. This is not an efficient approach because every instruction must be read and skipped. With 15.00 BTEQ there is now a QUIT option for the RUN command which can be used to immediately revert back to reading from stdin/SYSIN.
3 There is now a way to close out an import file that no longer needs to be left open for use. With 15.00 BTEQ, the IMPORT RESET command can now be used to close an active import file, releasing the file handle. Impact is possible. Scripts that have purposefully been using invalid IMPORT commands must be revised to use the new RESET option instead.
4 As was already valid for mainframe BTEQ, workstation BTEQs will now accept labels which start with a numeric digit.
New and Removed Errors
Impact: Possible. Script revisions may be needed.
1 When BTEQ has successfully parsed expected syntax for a BTEQ command but then finds extra command text, it now produces an error rather than a warning and will preserve the prior state. For example:Before:*** Warning: Extra text found after command. The text 'junk' not processed (ignored).After:*** Error: Invalid command syntax. Extra text found starting at 'junk'.Batch mode will continue to ignore extra text. This is the crux of its leniency and affects other behaviors such as what default values get assumed in absence of any other arguments given for a command.
2 BTEQ is much stricter about having complete punctuation in command syntax. Obvious incompleteness will now cause errors and PSP. For example, an otherwise valid TDP command which is followed by a slash asterisk comment which does not end with asterisk slash (as follows) will cause an extra text error and the TDP command will not be accepted:.SET TDP abc /*junkFor the RETLIMIT command and EXPORT command LIMIT option, commas placed between the ROWS and COLS values will no longer be ignored. The FORMCHAR command will still add xb to the end of a hex sequence if it was not supplied. But it will no longer be the case that a right quote for the string is optional.Batch mode will continue to be lenient about missing end quotes.
3 When BTEQ expects to see specific keywords or values supplied for a command, it generally no longer attempts to assume what the user meant to enter when invalid text is encountered instead. This enables BTEQ's command syntax to be extensible without constraints and ensures that BTEQ properly applies the "Do What I Mean" principle rather than make assumptions in a manner than is non-intuitive for users.EXAMPLESFor an invalid RETRY command:Before:*** Warning: Expected ON or OFF not 'JUNK'. Assuming the ON value.After:*** Error: 'JUNK' is invalid. Use ON or OFF instead.For an invalid “=” command:Before:*** Warning: '=' value is nonnumeric or out of range.REPEAT value is set to 1.After:*** Error: '=' value is nonnumeric or out of range.Invalid syntax for a REPEAT command will no longer cause the initial defaults to be set:Before:*** Error: Nonnumeric after REPEAT.REPEAT value is set to 1.After:*** Error: Numeric value, '*' or RECS expected.For an EXPORT command:Before:*** Warning: BOM|NOBOM is not allowed for DATA or INDICDATA exports.*** To reset export, type .EXPORT RESETAfter:*** Error: BOM|NOBOM is not allowed for DATA or INDICDATA exports.The EXIT command will no longer ignore unrecognized text after the EXIT keyword. The same error message will be given but no exit will take place.The EXPORT command will no longer ignore non-numeric arguments for its LIMIT option. Rather than assume defaults should be reestablished and convey this through warnings, the current state of the ROWS and COLS controls will remain unchanged and an error will be given.For an IMPORT command, invalid keywords will no longer be accepted for data format. In absence of INDICDATA, DATA or VARTEXT (or REPORT for workstation BTEQs), a FILE, DD or DDNAME keyword must be found next in order for DATA to be assumed for mainframe BTEQ or REPORT for workstation BTEQs.For z/OS BTEQ, the warning about a REPORT keyword not being valid to specify for an IMPORT command has been turned into an error.For interactive mode, the RETLIMIT command will no longer ignore non-numeric arguments. Rather than assume defaults should be reestablished and convey this through warnings, the current state of the ROWS and COLS controls will remain unchanged and an error will be given.
Note: To prevent egregious regressions, some of BTEQ's assumptions had to be preserved. For example, assuming REPORT data format for IMPORTs when no keyword at all is given.
4 When BTEQ is given an oversized or over-sectioned quoted string for a command, rather than truncate the value, BTEQ will reject the command and produce an error. ExamplesFor a FOOTING command where the quoted string exceeds 3 segments:Before:*** Error: The line(s) exceed(s) 3 segments. *** truncating the following string*** &date||confidential||&time||&pageAfter:*** Error: Too many dividers found. The quoted string can be divided into a maximum of 10 lines. Each line can have up to 3 sections.If a FOOTING command's quoted string value is simply too long:Before:*** Warning: String truncated to 254 because it was too long.*** Warning: Extra text found after command.The text 'tuvwxyzabcdefghijklmnopqrstuvw...' not processed (ignored).After:*** Error: The quoted string was too large.For the FORMCHAR command, the length of the hex string will be checked before checking for valid hex digits. And an error rather than warnings will be produced when the string is too long:Before:*** Warning: Numeric sequence is not hexadecimal!*** Syntax Error: Command ignored!*** Warning: Extra text found after command.The text 'hijkl"' not processed (ignored).After:*** Error: Hex sequence exceeds 10 digits.Batch mode will continue to be lenient about over-sized and over-sectioned quoted strings.
5 The '=' command will no longer silently discard instruction steps that come after the '=' command step in the same instruction. Instead, the following error will be generated:*** Error: An '=' command must be its instruction's last step.The remaining instruction text has been discarded.The only exception to this is when its just comments that follow the '=' command's terminating semicolon.ExampleValid --> =; -- commentValid --> =; /* comment */Invalid --> =; /* comment */;
6 For the MESSAGEOUT command, when invalid syntax is found when parsing through a 'FILE = filename' command phrase, only one error message will be generated. The "*** btqParseOutValidFileName failed" message will no longer always get generated. This message was meaningless to users since it isn't clear that it refers to an internal BTEQ function.
Default States
Impact: Possible. Script revisions may be needed.
For some settings, BTEQ used to revert back to its initial default state rather than preserve the prior state when invalid command syntax had been encountered. Commands that fall into this category include FORMCHAR, IMPORT and EXPORT. Now BTEQ always attempts to preserve the prior state. Only be when BTEQ is unable to switch to a new requested state for a syntactically valid command will it will revert the associated setting back to its original default state.
For example, for an IMPORT command which is valid syntactically but still fails when Data Connector is unable to open the file, the importing state will be completely deactivated and an informational message will be generated to reflect that the prior state was not preserved:
Before:*** Error: The following occurred during an Access Module open:
Requested file not found.
After:*** Error: The following occurred during an Access Module open:
Requested file not found.
*** Importing deactivated.
New Formatting
Impact: Possible.
1 For the RTITLE command, workstation BTEQ will now behave as mainframe BTEQ with regard to automatic inclusion of keywords for date or page. Neither one of those values will be automatically generated when either &DATE or &PAGE are found in the given RTITLE string. This is because BTEQ determines it needs to avoid causing duplicate information to be placed within the header. Mainframe BTEQ was doing the correct thing previously, but workstation BTEQ was not. Although it is theoretically possible, it is expected that no customer will be affected. Mainframe BTEQ will also continue to suppress date and page generation when &TIME is found. This was done to prevent existing scripts from regressing. It should have been the case initially that mainframe BTEQ didn't do anything but add time, but because it’s possible some customers may see the side effect auto-disappearance of date and page values as a convenience, that behavior was sustained.
2 The leading space that used to be printed out will be stripped out before echoing out an instruction supplied via the command line to stdout.
Other New Behaviors for Adhering to POLA
User Impact: Where files associated with commands such as RUN and IMPORT are left active rather than being closed, there is possible impact.
Some of BTEQ's command parsing behaviors were altered to better adhere to the principle of least astonishment so that state changes can be more intuitively understood by users.
A RUN file that does not exist is still considered to be empty and valid for use. If during parsing of the RUN command it cannot be opened, BTEQ still reverts back to reading from stdin.
If an invalid RUN command exists within a RUN file and it is not the last instruction step, it will need to be corrected.
New and Removed Warnings
Impact: None expected.
1 BTEQ commands should always be prefixed with a period to make sure they are identified as BTEQ commands as opposed to SQL requests. Otherwise, BTEQ might submit the command to the database. Historically, BTEQ has attempted to determine whether it has been given a command step without a period and, if so, produce a warning that it expected the period. Making that determination is problematic when command names are the same as leading keywords for SQL statements, as is the case for SET, HELP and SHOW related instructions. Two things have been done to enable users to help BTEQ make the correct decision. First, for new commands added starting with 15.00 the period is no longer optional. For pre-existing commands the optionality was grandfathered. Second, the wording of the warning was changed to make it clear that not supplying the period has risk.ExampleBefore:*** Warning: Expected a '.' before the commandAfter:*** Warning: Commands not prefixed with a '.' may be submitted as SQL.
2 BTEQ no longer silently makes state changes that would be non-intuitive to users, so some new Warning and Informational messages are produced. For example, the EXPORT command will no longer silently assume a data format when one is not specified. Rather, a warning such the following will be generated that explains the data format from the prior state is being used. XYZ comes out as REPORT, DATA, INDICDATA or DIF:*** Warning: No data format given. Assuming XYZ carries over.This is very different from what is still assumed for an IMPORT. For an IMPORT, when a data format keyword is omitted the initial default is assumed. For an EXPORT, the last value set for exporting is used. For example, the QUIET command will no longer silently reset FORMAT. A new informational message will be given.ExamplesBefore:*** Type QUIET OFF; to resume output.After:*** Type QUIET OFF; to resume output.*** FORMAT has been set to OFF.The RTITLE command will now produce the following warning when the given quoted string is too large to include DATE and PAGE substitutions but is being accepted as valid.*** Warning: String exceeds 233 bytes. Automatic date and page value generation has been omitted.When valid syntax is given for the NOTIFY command but BTEQ is subsequently unable to initialize the method, the state will be changed to OFF and now an error message to explain the setting had to be turned off appears.
3 For some commands, superfluous warnings were being produced for syntax issues which were actually non-issues. Generation of those warnings were removed.ExamplesFor a valid '=' command where no numeric value has been given, the following unnecessary warning will no longer be generated.*** Warning: '=' value is nonnumeric or out of range.REPEAT value is set to 1.For commands FOLDLINE, OMIT, PAGEBREAK, SKIPDOUBLE, SKIPLINE, SUPPRESS and UNDERLINE, the entry of ALL without ON or OFF (for which assuming ON is completely valid) no longer causes the following warning:*** Warning: Expected ON or OFF not 'ALL'. Assuming the ON value.
4 Several warnings messages were rewritten for correctness. For example, for an IMPORT command, the warning message generated to reflect when BTEQ has assumed a particular data format has been altered to use the IMPORT command keyword terminology rather than refer to the underlying response mode name.ExampleBefore:*** Warning: No IMPORT mode was given, assuming field mode.After:*** Warning: No data format specified. Assuming REPORT.
Revised Error Messages
Impact: None expected.
1 Many error messages have been reworded to be more useful. For example, rather than just stating that an illegal value was given, a list of expected values are given.ExampleFor REPORTALIGN:Before:*** Error: Illegal value "junk" specified. Option ignored.After:*** Error: 'junk' is invalid.Use COMPATIBLE, EQUALWIDTH or COLUMNS instead.
2 Some error messages referred to a command as unknown when the command was known but the syntax used was invalid or the command could not be used in the current state. These messages were corrected.ExamplesWhen the '=' command is used without any logged on sessions:Before:*** Error: Unrecognized command =After:*** Error: Must logon to issue '=' command.When the "HELP" command is given without the BTEQ keyword:Before:*** Error: Unknown BTEQ help option given.After:*** Error: Invalid HELP command.For ".SET SESSION xyz" type commands, the general syntax error message has been rewritten to identify the command as SESSION rather than SET.Before:*** Error: Unrecognized SET command ENTRYAfter:*** Error: Invalid SET SESSION command.Unknown keyword 'ENTRY'.
3 Some error messages which were labeled incorrectly as warnings have been reworded. ExampleFor an IF command where nothing follows the THEN keyword...Before:*** Warning: Command or continuation character not found after THEN clause!*** Statement not processed.After:*** Error: No instruction found after the THEN clause.Expected a BTEQ command or SQL request.
4 Some error messages have been revised to correctly identify the source of error as being with unexpected extra text as opposed to expected text to help make it more obvious what the valid syntax is.ExampleFor FOLDLINE, OMIT, PAGEBREAK, SKIPDOUBLE, SKIPLINE, SUPPRESS and UNDERLINE commands, having non-numeric entries other than OFF, ON or ALL occurring after a column number list:Before:*** Error: 'JUNK' is not a valid column number.Column value must be in the 1..100 range.After:*** Error: Invalid command syntax.Extra text found starting at 'junk'.
New and Revised Informational Messages
Impact: None expected.
1 The SHOW ERRORMAP command results will now reflect the severity level that is currently set for the UNKNOWN category of error codes.
2 For the SHOW CONTROLS command, the FOLDLINE, SKIPLINE, SKIPDOUBLE and OMIT results will no longer include a trailing period.ExampleBefore:[SET] FOLDLINE = ON 2-4 .After:[SET] FOLDLINE = ON 2-4
3 For SHOW CONTROLS EXPORT results, the data format keywords have been corrected.ExamplesBefore:EXPORT FIELD FILE=temp , LIMIT * 100 , OPENAfter:EXPORT REPORT FILE=temp , LIMIT * 100 , OPENFor SHOW CONTROLS IMPORT results, when importing is not activated the results will reflect that IMPORT is reset:IMPORT RESET
|
Issue |
Description |
|
CLAC-13644 |
BTEQ now allows Large Data Objects to be exported in deferred mode. This allows for larger single objects to be exported (up to 2GB) and stored in their own file for easier management. This is accomplished by using the new DEFERTRANS option for the EXPORT command. |
|
CLAC-29696 |
On z/OS, BTEQ has validated user input for Chinese, Japanese, and Korean sessions for some time now. In previous releases, z/OS BTEQ simply reported invalid multi-byte sequences with a BTEQ error message, but still processed the request. Starting with release 15.00, z/OS BTEQ will now terminate after reporting an appropriate error message. This is consistent with how BTEQ handles ill-formed input from other multi-byte character sets. This validation only affects multi-byte sequences delimited by Shift-Out/Shift-In characters on z/OS, and is limited to the following character sets: HANGULEBCDIC933_1II KANJIEBCDIC5026_0I KANJIEBCDIC5035_0I KATAKANAEBCDIC SCHEBCDIC935_2IJ TCHEBCDIC937_3IB |
|
CLAC-29742 |
The PAGELENGTH command has been expanded to allow for an asterisk (*), which represents an unlimited page length. When an asterisk is used, each query will produce a single page, sized appropriately for that query's result set. |
|
CLAC-29743 |
The maximum row value for the RETLIMIT command has been increased to 18,446,744,073,709,551,615. Setting the row value to an asterisk (*) or zero still represents no limit. |
|
CLAC-29853 |
A new option ALL was added to the QUIET command. QUIET ON ALL will suppress a greater amount of output than QUIET ON, including elapsed time messages. This new option may be helpful to users processing large amounts of data. |
|
CLAC-30160 |
BTEQ will now use 16 as its default return code for all run-terminating errors that are due to BTEQ-specific errors such as BTEQ being unable to allocate sufficient memory to accomplish a certain task. These types of errors are rarely if ever encountered and it is anticipated that there will be no impact to customer's existing scripts. These changes also do not change any expected behaviors associated with use of IF-THEN branching logic or the ERRORCODE and ERRORLEVEL keywords. |