Revise application of COBOL-WORDS
COBOL scanner logic is revised to apply COBOL-WORDS overrides after token retrieval, improving flexibility.
This commit changes how COBOL-WORDS overrides are applied. Instead of the scanner handling them directly, the generated scanner now retrieves tokens, and the COBOL-WORDS overrides are applied afterward. This revision allows for more flexibility, though not all keywords can be modified successfully with the current implementation.
In Details
The COBOL front-end's lexer implementation is modified so that COBOL-WORDS overrides are applied by the caller of the lexer, rather than within the scanner itself. Specifically, scan.l no longer uses redefined_token directly, and next_token in scan_post.h is updated to apply these overrides when a token is retrieved from the lexer() function. This change separates the concerns of tokenization and keyword manipulation, though it acknowledges limitations with certain keywords.
- COBOL-WORDS
- Likely a mechanism or set of rules in the COBOL compiler that allows for special handling or redefinition of specific keywords or tokens.
- scanner
- A component of a compiler (also known as a lexer) that reads the source code character by character and groups them into meaningful tokens.
- lexer
- Lexical Analyzer. Another term for a scanner, responsible for converting a stream of characters into a stream of tokens.
- token
- A fundamental lexical unit in a programming language, such as a keyword, identifier, operator, or literal.
- parse_ante.h / scan_ante.h / scan_post.h
- Header files likely related to different stages or aspects of the COBOL front-end's parsing and scanning process.