x86: Correct operand type usage in VEX prefix generation
Ensures the correct operand types are used when building VEX prefixes in the x86 assembler.
This commit corrects how operand types are accessed when building VEX prefixes in the x86 assembler. Previously, the code might have incorrectly used the operand_types from the currently installed template (t->operand_types[]) when it should have referred to the types associated with the instruction instance being processed (i.tm_types[]). This change ensures the VEX prefix logic correctly uses the operand types of the intended instruction, preventing potential issues.
In Details
The build_vex_prefix() function in tc-i386.c is updated to use i.tm_types[op] instead of t->operand_types[op] when checking for XMM or YMM word operands. This corrects a potential issue where the wrong instruction template's operand types could be consulted, especially after t[1] might have been installed for use by the instruction. This ensures that the VEX prefix logic correctly references the operand types relevant to the instruction being assembled.
- VEX prefix
- A prefix byte used in x86 instruction encoding to extend the instruction set, particularly for SIMD (Single Instruction, Multiple Data) operations, enabling support for 128-bit, 256-bit, and 512-bit vector registers.
- operand types
- Specifies the type and properties of each operand for a given instruction, crucial for correct instruction decoding and encoding.
- x86
- A family of instruction set architectures (ISAs) originating from the Intel 8086 microprocessor. It is the most common ISA for desktop and server computers.
- assembler
- A program that translates assembly language code into machine code.
- insn_template
- A data structure used in the assembler's opcode table to define the characteristics of an instruction, including its operands and how to encode it.