PR 30308 more unbounded recursion in i386 gas simplify
Fixed a stack overflow issue in i386 gas's expression simplification that occurred with specific combinations of symbol types.
Addressed a bug in the gas assembler for i386 where certain complex expressions could lead to unbounded recursion, causing a stack overflow. The issue occurred when an expression involved both X_add_symbol and X_op_symbol. The fix ensures that the assembler correctly handles these cases by marking symbols as resolving during the recursive calls and clearing the mark afterwards, preventing infinite loops.
In Details
The i386 gas assembler's i386_intel_simplify and i386_intel_simplify_symbol functions could enter unbounded recursion when simplifying expressions containing both X_add_symbol and X_op_symbol. This commit introduces symbol_mark_resolving and symbol_clear_resolving calls around the recursive invocations to break the cycle, similar to how nested bracket handling is managed. A specific test case for intel-equ-loop is extended to cover this scenario.
- gas
- The GNU Assembler, part of the GNU Binutils package. It translates assembly language code into machine code.
- X_add_symbol
- An internal representation within the assembler's expression parser indicating an addition operation involving a symbol.
- X_op_symbol
- An internal representation within the assembler's expression parser indicating an operation on a symbol, often used for addressing modes or relocations.
- Expression Simplification
- The process by which the assembler analyzes and simplifies arithmetic expressions involving symbols and constants, often to resolve addresses or calculate relocation values.
- Recursion
- A programming technique where a function calls itself to solve smaller instances of the same problem. Unbounded recursion can lead to stack overflow.
- i386
- A 32-bit extension of the Intel 8086 microprocessor architecture, commonly used in older PCs.