The question
vector.transfer_read means “read N elements starting at index i”. Before emitting code the
compiler must decide whether that can run off the end of the buffer. If it cannot prove safety it
emits a masked load; if it can, a plain one. The canonicalizer’s existing fold only fires when the
index is a constant. After tiling and vectorisation the index is a loop induction variable, or an
affine expression of one, so on real code the fold almost never fires and the transfer keeps
in_bounds = false.
That is not free. On targets without predicated memory operations, LLVM’s
ScalarizeMaskedMemIntrin replaces each masked load with a chain of basic blocks that handle the
lanes one by one. On AArch64 that applies to all NEON-only code.
The answer
-vector-infer-in-bounds, an opt-in pass (llvm-project #215340),
asked ValueBoundsConstraintSet for the largest value the index can take and set in_bounds when
even that left room for a full vector.
On PolyBench jacobi-2d (MEDIUM, N=250, TSTEPS=100), AArch64 NEON, vector width 4: transfers
proved in-bounds 0 → 12, kernel instructions 317 → 85sourcehttps://github.com/llvm/llvm-project/pull/215340checked2026-09-22hardwareAArch64 NEON, Apple siliconcommandbench/run.shkindmeasured, llvm.masked.* calls
14 → 0sourcehttps://github.com/llvm/llvm-project/pull/215340checked2026-09-22hardwareAArch64 NEON, Apple silicon (no SVE); not measured on SVEcommandbench/run.shkindmeasured, wall clock 14.70 → 2.89 mssourcehttps://github.com/llvm/llvm-project/pull/215340checked2026-09-22hardwareAArch64 NEON, Apple silicon (no SVE)commandbench/run.shkindmeasured (median of 21).
How it was measured
Both arms use the same mlir-opt binary and byte-identical input; the only difference is that the
second runs the new pass after -canonicalize. Wall clock is the median of 21 runs. The noise floor
is 0.45 %, from an A-versus-A comparison plus the larger of the two median absolute deviations, and
the driver checks that both arms produce identical output arrays. Measured on a local macOS/arm64
machine (Apple silicon), which has no SVE. No number here was measured on SVE hardware; on SVE the
mask is native and the win, if any, is repeated mask setup rather than avoided scalarisation.
What I got wrong
The first version of the patch put the value-bounds query inside the op’s folder, isInBounds in
VectorOps.cpp. Three maintainers independently asked for it to be an opt-in pass instead, because a
folder carries no options and no pass context, so the query could not be gated behind a flag and
every canonicalisation of every function would have paid for it. An earlier revision offered to add
such a flag; that offer was not implementable and was withdrawn. The restructured pass leaves the
folder untouched and is purely additive.
The bigger question the review raised, whether in_bounds should exist at all or masking should
replace it, I could not answer from opinion, so I wrote it up as an
RFC with per-target measurements. It drew 11sourcehttps://discourse.llvm.org/t/91649.jsonchecked2026-09-25commandcurl -sL https://discourse.llvm.org/t/91649.json | python3 -c "import json,sys;d=json.load(sys.stdin);print(sum(p['username']!='dhairyashilRG' for p in d['post_stream']['posts']))"kindmeasured
replies, including from the vector dialect maintainers, and settled on removing in_bounds: prove
masks all-true instead of inferring the attribute. That made my own pass the wrong step, so I
closed it on 2026-09-25. The measurement above still stands; the route to it changed.
Reproduce it
git clone https://github.com/dhairyashilRG/mlir-inbounds-harness
cd mlir-inbounds-harness && cat README.md # prerequisites, then:
bench/run.sh # A/B on the PolyBench kernels, with the oracle check
Everything in that repository is meant to be run by someone who is not me. If a number in the patch description cannot be produced by something in it, it should not be in the patch description.
Where it went
#215340 was closed on 2026-09-25 and superseded
by #226517, which reuses its scaffolding for an
opt-in -eliminate-vector-masks pass: it proves vector.create_mask all-true with the same bounds
query, so a masked transfer becomes a plain one. That is the order the RFC agreed. The bounds
reasoning it needs merged on the way: fixed-size masks in eliminateVectorMasks
(#221595), loop bounds for affine.for
(#214614), and two fixes to the old fold, for
scalable dimensions (#213506) and negative
constant indices (#219681).