The question
vector.transfer_read means “read N elements starting at index i”. Before emitting code the
compiler must decide whether that can run off the end of the buffer. If it cannot prove safety it
emits a masked load; if it can, a plain one. The canonicalizer’s existing fold only fires when the
index is a constant. After tiling and vectorisation the index is a loop induction variable, or an
affine expression of one, so on real code the fold almost never fires and the transfer keeps
in_bounds = false.
That is not free. On targets without predicated memory operations, LLVM’s
ScalarizeMaskedMemIntrin replaces each masked load with a chain of basic blocks that handle the
lanes one by one. On AArch64 that applies to all NEON-only code.
The answer
-vector-infer-in-bounds, an opt-in pass (llvm-project #215340),
asks ValueBoundsConstraintSet for the largest value the index can take and sets in_bounds when
even that leaves room for a full vector.
On PolyBench jacobi-2d (MEDIUM, N=250, TSTEPS=100), AArch64 NEON, vector width 4: transfers
proved in-bounds 0 → 12, kernel instructions 317 → 85sourcehttps://github.com/llvm/llvm-project/pull/215340checked2026-09-22hardwareAArch64 NEON, Apple siliconcommandbench/run.shkindmeasured, llvm.masked.* calls
14 → 0sourcehttps://github.com/llvm/llvm-project/pull/215340checked2026-09-22hardwareAArch64 NEON, Apple silicon (no SVE); not measured on SVEcommandbench/run.shkindmeasured, wall clock 14.70 → 2.89 mssourcehttps://github.com/llvm/llvm-project/pull/215340checked2026-09-22hardwareAArch64 NEON, Apple silicon (no SVE)commandbench/run.shkindmeasured (median of 21).
How it was measured
Both arms use the same mlir-opt binary and byte-identical input; the only difference is that the
second runs the new pass after -canonicalize. Wall clock is the median of 21 runs. The noise floor
is 0.45 %, from an A-versus-A comparison plus the larger of the two median absolute deviations, and
the driver checks that both arms produce identical output arrays. Measured on a local macOS/arm64
machine (Apple silicon), which has no SVE. No number here was measured on SVE hardware; on SVE the
mask is native and the win, if any, is repeated mask setup rather than avoided scalarisation.
What I got wrong
The first version of the patch put the value-bounds query inside the op’s folder, isInBounds in
VectorOps.cpp. Three maintainers independently asked for it to be an opt-in pass instead, because a
folder carries no options and no pass context, so the query could not be gated behind a flag and
every canonicalisation of every function would have paid for it. An earlier revision offered to add
such a flag; that offer was not implementable and was withdrawn. The restructured pass leaves the
folder untouched and is purely additive.
The bigger question the review raised, whether in_bounds should exist at all or masking should
replace it, I could not answer from opinion, so I wrote it up as an
RFC with per-target measurements. It drew 9sourcehttps://discourse.llvm.org/t/91649.jsonchecked2026-09-22commandcurl -sL https://discourse.llvm.org/t/91649.json | python3 -c "import json,sys;d=json.load(sys.stdin);print(sum(p['username']!='dhairyashilRG' for p in d['post_stream']['posts']))"kindmeasured
replies from the vector dialect maintainers and is still open, as is the pull request.
Reproduce it
git clone https://github.com/dhairyashilRG/mlir-inbounds-harness
cd mlir-inbounds-harness && cat README.md # prerequisites, then:
bench/run.sh # A/B on the PolyBench kernels, with the oracle check
Everything in that repository is meant to be run by someone who is not me. If a number in the patch description cannot be produced by something in it, it should not be in the patch description.
Where it went
Open upstream as #215340 (restructured 2026-08-29, pinged 2026-09-12). Two supporting fixes merged on the way: no fold on scalable dimensions (#213506) and no fold for negative constant indices (#219681); the affine loop bounds the pass relies on come from #214614.