Anatomy of an Intrinsic: How One ACLE Name Becomes an AArch64 Instruction

The background: the AArch64 intrinsic factory

When a vendor adds an ISA extension, the compiler work is rarely a single pattern in a machine description. Arm’s ACLE (Arm C Language Extensions) specifies hundreds of C-callable functions per extension, each with a mangled name encoding its element types, its predication, and how many vector registers it consumes. GCC’s aarch64 backend does not hand-write a builtin for each one. It generates them from tables.

The tables live in gcc/config/aarch64/aarch64-sve-builtins-sme.def and its siblings. Each line is a macro invocation naming a family, and the macro expansion multiplies that family out across a list of type suffixes and group suffixes. Look at the shape of the existing entries the dossier shows:

DEF_SME_ZA_FUNCTION_GS (svmla_lane, binary_za_slice_lane, za_s_float,
			vg1x24, none)
DEF_SME_ZA_FUNCTION_GS (svmla_lane, binary_za_slice_lane, za_s_b_integer,
			vg4, none)

Five arguments: the base name (svmla_lane), a shape (binary_za_slice_lane), a type array (za_s_float), a group suffix list (vg1x24), and a mode-suffix field. From one such line the builtin machinery emits every legal combination — svmla_lane_za32_f32_vg1x2, svmla_lane_za32_f32_vg1x4, and so on — each with its own overload entry for C++ resolution and its own expansion path.

This matters to a reader outside the subsystem because it explains why an intrinsic patch touches ten files and none of them look like “the code for the instruction”. The instruction is one file. The other nine are the factory that decides which names exist, what argument types they accept, and which of those arguments must be a compile-time constant.

The mechanism: shape, impl class, UNSPEC, pattern

Follow one call from source to assembly. Four distinct pieces of the backend take turns.

First the shape. A shape is a C++ object that knows the argument grammar for a whole family: how many vectors, which ones are immediates, what the return type is, and how to resolve an overloaded call to a specific typed form. Shapes live in aarch64-sve-builtins-shapes.cc; the commit adds ternary_za_uint_dual_single there. The name is readable once you know the convention: a ternary operation writing into ZA, with unsigned-typed handling, taking dual (paired) and single vector operands. Shapes are also where argument-range checking happens — an immediate that must lie in [0, 3] is rejected here, at compile time, with a diagnostic rather than a silently wrong encoding.

Second the impl class, in aarch64-sve-builtins-sme.cc. This is where a family with many type combinations collapses onto a small set of internal operations. svtmopa_lane_za_impl derives from read_write_za<function_base> — a base that tells the rest of the compiler this builtin both reads and writes the ZA state, which is not an ordinary register and needs explicit modelling. Its unspec_for method reads the resolved type suffixes and picks the operation:

if (instance.fpm_mode == FPM_set)
  return UNSPEC_SME_FTMOPA_FP8;
const auto &suffix1 = instance.type_suffix (1);
if (!suffix1.integer_p)
  return UNSPEC_SME_FTMOPA;
const auto &suffix2 = instance.type_suffix (2);
if (suffix1.unsigned_p && suffix2.unsigned_p)
  return UNSPEC_SME_UTMOPA;
else if (!suffix1.unsigned_p && !suffix2.unsigned_p)
  return UNSPEC_SME_STMOPA;
else if (suffix1.unsigned_p && !suffix2.unsigned_p)
  return UNSPEC_SME_USTMOPA;
else
  return UNSPEC_SME_SUTMOPA;

That cascade is the whole signed/unsigned/mixed matrix in eight lines. Thirteen documented intrinsic names funnel into six UNSPECs. An UNSPEC is GCC’s escape hatch in RTL: an opaque operation tagged with an integer, which the optimisers will not try to reason about but which instruction patterns can match on. It is how a backend represents “this machine operation has no portable RTL meaning”.

Third the mode selection, in the same class’s expand:

machine_mode za_mode = e.vector_mode (0);
machine_mode v_mode = e.tuple_mode (1);
if (GET_MODE_UNIT_BITSIZE (za_mode) == GET_MODE_UNIT_BITSIZE (v_mode))
  za_mode = v_mode;
insn_code icode
  = code_for_aarch64_sme_lane (unspec_for (e), za_mode, v_mode);
return e.use_exact_insn (icode);

code_for_aarch64_sme_lane is generated code: for a define_insn named with the @ prefix, GCC emits a lookup function taking the iterator values as arguments. So the (unspec, za_mode, v_mode) triple is the address of a pattern. The GET_MODE_UNIT_BITSIZE normalisation exists because the ZA accumulator’s element width and the input vector’s element width sometimes agree and sometimes do not; when they agree, one mode is enough to name the pattern.

Fourth the patterns in aarch64-sme.md. The commit adds five define_insns, and their names show which mode pairs are legal: @aarch64_sme_lane_<optab><VNx4SI_ONLY:mode><SVE_FULL_BHI:mode> for 32-bit ZA with byte/halfword integer inputs, @aarch64_sme_lane_<optab><VNx4SI_ONLY:mode><SVE_FULL_HF:mode> for half-precision, @aarch64_sme_lane_<optab><SME_TMOPA_FP8:mode><VNx16QI_ONLY:mode> for the FP8 forms, and so on. The <optab> substitution is driven by the new optab entries for the six UNSPECs, which is what makes one pattern template cover signed, unsigned and mixed variants.

The change: a family, a shape, a constraint, two predicates

The commit adds the svtmopa_lane intrinsics for FEAT_SME_TMOP, gated behind +sme-tmop. The .def entries are the clearest statement of what is now callable:

#define REQUIRED_EXTENSIONS streaming_only (AARCH64_FL_SME_TMOP)
DEF_SME_ZA_FUNCTION_GS (svtmopa_lane, ternary_za_uint_dual_single, tmop_base, none, none)
#undef REQUIRED_EXTENSIONS

Read that block carefully, because three separate policies are encoded in it. streaming_only says these functions may only be called from streaming mode — SME’s alternate execution mode in which the vector length and the legal instruction set both change. AARCH64_FL_SME_TMOP is the feature bit that must be enabled. And tmop_base names one of the five new DEF_SVE_TYPES_ARRAYs added in aarch64-acle-builtins.h (TYPES_tmop_base, TYPES_tmop_h_bfloat, TYPES_tmop_h_float, TYPES_tmop_h_mf8, TYPES_tmop_s_mf8), each listing the type-suffix triples that expand into concrete names.

The type arrays are split rather than merged precisely so the extension gating can differ per type. Half-precision float needs a second flag; so does bfloat16; the FP8 forms need a third:

#define REQUIRED_EXTENSIONS streaming_only (AARCH64_FL_SME_TMOP \
					    | AARCH64_FL_SME_B16B16)
DEF_SME_ZA_FUNCTION_GS (svtmopa_lane, ternary_za_uint_dual_single, tmop_h_bfloat, none, none)
#undef REQUIRED_EXTENSIONS

#define REQUIRED_EXTENSIONS streaming_only (AARCH64_FL_SME_TMOP \
					    | AARCH64_FL_SME_F8F16)
DEF_SME_ZA_FUNCTION_GS_FPM (svtmopa_lane, ternary_za_uint_dual_single, tmop_h_mf8, none, none, set)
#undef REQUIRED_EXTENSIONS

Note the different macro on the last line: DEF_SME_ZA_FUNCTION_GS_FPM, with a trailing set. That is the FPM (floating-point mode register) handling — FP8 arithmetic reads format and rounding configuration from a mode register rather than from the opcode, so those intrinsics must be modelled as depending on it. That set is the same signal unspec_for tests as instance.fpm_mode == FPM_set when it returns UNSPEC_SME_FTMOPA_FP8. One declaration flag, read back on the expansion path.

Below the intrinsic layer the commit adds the low-level vocabulary the new patterns need. constraints.md gains Uwo, a register constraint for the Control Vector Register in TMOP operations — a constraint restricts which registers RA may pick for an operand, and a new one exists because this operand’s legal register set is narrower than any existing constraint describes. predicates.md gains aarch64_imm1 and aarch64_imm4, which the commit message describes as tightening “the immediates selecting ZA tiles based on tile width”: ZA is addressed as tiles, and how many tiles exist depends on element width, so a 16-bit-element form accepts a wider tile index range than a 32-bit-element one. Encoding that in two predicates means an out-of-range tile number is a compile error, not a truncated encoding.

aarch64-c.cc gains the __ARM_FEATURE_SME_TMOP predefine, aarch64.h gains TARGET_STREAMING_SME_TMOP, and iterators.md gains the SME_TMOPA_BHSF and SME_TMOPA_FP8 mode iterators, the six UNSPECs, the SME_TMOP_INT / SME_TMOP_INT_CROSS / SME_TMOP_FP / SME_TMOP_FP8 int iterators, and a za_imm_bits mode attribute. Each of those is one row in the factory. Remove any one and the chain breaks at a different place.

The consequences: what to expect outside this corner

If you write AArch64 code, the practical surface is small and sharply bounded. __ARM_FEATURE_SME_TMOP is now the portable way to ask whether these intrinsics exist, and the testsuite addition to pragma_cpp_predefs_6.c explicitly checks that the predefine is off by default and that the feature dependencies hold. Feature-test macros that appear unconditionally are a recurring class of bug; the test exists to stop that.

The streaming_only gating is the constraint most likely to surprise someone reading a patch that uses these. A call to svtmopa_lane_za32[_s8_s8] from a function that is not __arm_streaming is a compile-time error, not a runtime fault. Streaming mode changes the effective vector length and the legal opcode set, so GCC tracks per-function streaming status and validates every SME builtin against it. The same goes for the FP8 forms: they carry an FPM dependency, so the compiler treats the mode register as part of their input, which restricts how freely they can be reordered relative to code that writes it.

If you maintain a different backend, the transferable observation is the layering. Argument grammar and immediate range live in the shape, where errors can be reported against source locations. Type-to-operation mapping lives in the impl class, as ordinary C++ over resolved type suffixes — not as a combinatorial explosion of .md patterns. Only the final encoding lives in the machine description, parameterised by iterators so one template serves signed, unsigned and mixed forms. Thirteen names, six UNSPECs, five define_insns: the fan-in is deliberate, and it is what keeps a several-hundred-intrinsic extension tractable.

One honest limit on this reading: the dossier does not include the bodies of the new define_insns, so nothing here describes the assembly text they emit or the operand order in the encoding. The pattern names tell you which mode combinations are covered; the output templates would tell you the rest.

How to see it yourself

The testsuite is the documentation. gcc/testsuite/lib/target-supports.exp gains a check_effective_target entry for sme-tmop, which is the standard mechanism by which DejaGnu decides whether to run a test at all — it compiles a probe snippet and skips the test if the assembler or compiler rejects it. That is why the thirteen new acle-asm tests can assert exact assembly without breaking on toolchains that lack the extension.

The tests themselves live under gcc.target/aarch64/sme2/acle-asm/, one file per type combination: tmopa_lane_za32_s8_u8.c, tmopa_lane_za16_mf8_mf8.c, and so on. They are built on a new macro, TEST_ZA_TMOP, added to test_sme2_acle.h. The acle-asm convention is to invoke the intrinsic in a minimal wrapper and then scan the assembly output for the expected instruction with dg-final directives — so reading one of these files tells you, per type combination, exactly what the compiler is contracted to emit.

For the error paths, read gcc.target/aarch64/sve/acle/general-c/ternary_za_uint_dual_single_1.c. Files under general-c test diagnostics rather than codegen: wrong argument counts, out-of-range immediates, mismatched types. If you want to know what the new shape and the new aarch64_imm1 / aarch64_imm4 predicates actually reject, that file is the specification.

One small mercy in the commit worth noting for anyone who reads this code later: it also fixes a stale comment on function_resolver::resolve_to in aarch64-acle-builtins.cc, which described “the type suffixes given by TYPE0 and TYPE1” when the function had grown a third. The new svtmopa_lane forms are the ones that need three type suffixes, so the comment and the caller arrived together.