Fix SCEV handling of INT_MIN subtraction to avoid UB
Ensures correct scalar evolution calculation for subtractions involving the most negative integer, preventing undefined behavior.
This commit resolves an issue in Scalar Evolution (SCEV) analysis where subtracting the most negative value of a signed integer type could lead to undefined behavior and incorrect analysis results. Previously, only the negation of INT_MIN was guarded, but this did not correct the resulting SCEV direction when the operation was flipped from subtraction to addition. The fix now promotes the entire operation to unsigned, ensuring correct analysis even in these edge cases. New test cases verify the fix.
In Details
The scev_dfs::add_to_evolution and follow_ssa_edge_binary functions in tree-scalar-evolution.cc have been refactored to remove manual MINUS_EXPR handling. The follow_ssa_edge_expr now handles MINUS_EXPR by promoting the entire operation to unsigned, instead of just the negated operand. This corrects a wrong scev_direction that previously occurred when handling subtractions involving INT_MIN and prevented undefined behavior that could arise from negating INT_MIN directly.
- Scalar Evolution (SCEV)
- A GCC analysis pass that determines how values change across loop iterations. It's used to find opportunities for optimizations like loop unrolling and vectorization by representing values as affine expressions.
- INT_MIN
- The smallest representable value for a signed integer type. For a 32-bit integer, this is typically -2,147,483,648.
- Undefined Behavior (UB)
- Behavior that results from an erroneous program construct or incorrect use of a standard API. The C and C++ standards do not specify what must happen in such cases, and compilers are free to assume UB does not occur, potentially leading to unexpected results or optimizations.
- SSA
- Static Single Assignment form. An intermediate representation where every variable is assigned a value exactly once. This simplifies many data-flow analyses.
- tree-optimization
- GCC's middle-end optimizer that operates on the compiler's internal Abstract Syntax Tree (AST) or GIMPLE representation.