Lex CDF tokens exclusively in COBOL to treat words as data.

COBOL lexer now prioritizes CDF tokens, allowing COBOL words to be treated as data.

By James K. Lowden August 25, 2026 committed

The COBOL lexer has been modified to exclusively prioritize Customer Data Format (CDF) tokens over default COBOL keywords. This change allows COBOL words to be interpreted as data within the CDF context, resolving issues related to parsing and handling specific data formats.

In Details

The lexer (cobol/scan.l) now positions the cdf_state start condition above the default COBOL keyword start condition. This ensures that tokens conforming to the CDF specification are lexed first. Corresponding changes in cobol/cdf.y and cobol/parse.y renumber tokens and add entries from the ISO CDF reserved word list, and cobol/token_names.h is regenerated.

For Context

lexer: The component of a compiler that breaks down source code into a stream of tokens (lexemes).
CDF: Likely refers to 'Common Data Format' or a similar specification relevant to COBOL, indicating a set of rules or definitions for data handling.
COBOL keywords: Words that have a predefined meaning and function within the COBOL programming language, such as MOVE, ADD, IF.
start condition: A mechanism in lexers (like flex or lex) that defines different states for parsing, allowing specific rules to apply based on the current context.
tokens: The basic building blocks of source code, such as keywords, identifiers, operators, and literals, generated by the lexer.

Filed Under: cobollexercompiler

View Commit →