x86: fix EVEX no-broadcast disassembly when W=1
Fixes objdump to correctly identify invalid broadcast encodings for EVEX instructions when the W-bit is set.
This patch corrects an issue in the binutils disassembler where it incorrectly accepts certain EVEX instruction encodings that specify broadcasting when it’s not allowed (specifically for the W=1 case). The fix ensures that these invalid encodings are flagged as errors, matching assembler behavior.
In Details
This patch addresses a bug in the x86 disassembler within binutils. It concerns the EVEX instruction prefix, specifically the handling of the 'no-broadcast' mode when the EVEX.W bit is set to 1. The disassembler was failing to reject encodings that were invalid according to the EVEX specification for W=1, while correctly handling the W=0 case. The fix involves marking the relevant operand mode as non-broadcastable before the EVEX.W-dependent path.
- EVEX
- An advanced instruction prefix used in x86 processors for SIMD operations, supporting features like extended registers, vector lengths, and masking.
- no-broadcast
- A mode in EVEX instructions where the data element is not broadcast from a scalar to a vector register.
- W-bit
- A bit within the EVEX prefix that affects the operand size or other instruction semantics.
- objdump
- A utility that displays information from object files, including disassembly of machine code.