Gigablock series — plan

This document describes two series. The prequel (§13's Phase P) is pageblock evacuation: it stands alone upstream, needs nothing from the rest, and has its own measurable claim. The gigablock series is everything else and builds on it. Wherever both appear below, which one owns a piece is stated.

2026-08-06. Canonical and self-consistent as of this date. Superseded decisions, and the scenario that killed each, are in gigablock-plan-history.md and gigablock-plan-v1-archaeology.md. Neither is safe to cite from: their section numbers overlap this file's and mean different things.

Base for the series is scratch/riel/gigablock at 0f23d56f17fd (linus-upstream, 7.3-rc1 merge window), which has 848acc8ffe1b as an ancestor. Code facts below were derived against e1914add2799. Re-verified on the new base: §12a's seven zone->free_area references in page_alloc.c, still exactly seven in the same seven functions. Everything else still needs re-verifying — mm/page_alloc.c differs by 422 lines from e1914add2799 and mm/vmscan.c by 464. Measurements are from ~/debug/gigablock-phase0/ on devvm34971, node 0 Normal zone.

1. Problem, and the measured baseline

1.0 Why 1 GB pages, and why they cannot be allocated on demand

The kernel allocates two kinds of memory, and only one of them can be moved. Most of a machine's memory is allocated on behalf of userspace — page cache, anonymous memory — and the kernel knows every mapping of it, so it can copy such a page elsewhere and repoint the page tables. That is migration, and it is what compaction and alloc_contig_range() are built on. The kernel also allocates for itself: slab objects, page tables, per-cpu areas, DMA buffers. Those generally cannot move, because some other part of the kernel holds a pointer to the object and there is no way to find and update every such pointer. Everything below follows from that split.

A TLB entry covers one page whatever its size, so page size is TLB reach. One entry maps 4 KB, or 2 MB, or 1 GB. The TLB is a small fixed hardware resource, so a workload whose working set exceeds entries x page_size pays a page walk on the misses, and the walk itself is shorter for a huge page: a 1 GB mapping terminates at the PUD, skipping the PMD and PTE levels a 4 KB mapping has to load. Under virtualization both effects are multiplied, because a guest TLB miss walks two page tables rather than one.

Page table memory follows the same ratio. A 1 GB region mapped at 4 KB needs 262144 PTEs; mapped at PUD level it needs one entry. For a database or a VM with a large, sparsely touched heap that is the difference between page tables measured in gigabytes and page tables that fit in cache.

What a 1 GB page costs to produce is 262144 physically contiguous, PUD-aligned pages, and one page of the second kind anywhere in that range is fatal. Movable content is migrated out of the way, which is work but always possible. A kernel allocation is not: it stays where it is for as long as it lives, and one slab object or page table in a 1 GB range disqualifies the range for that whole time.

There is no free list for 1 GB, and that is a deliberate tradeoff rather than an oversight. The buddy allocator keeps one free list per order and searches upward from the requested order — __rmqueue_smallest() loops current_order from order to NR_PAGE_ORDERS. Every extra order lengthens that loop for every allocation that has to climb it, and enlarges the per-zone free_area[] array. What those extra orders would hold is almost nothing: blocks much above a handful of orders rarely arise by chance on a running system, and even order 5 is uncommon. Paying an iteration on every allocation to check lists that are nearly always empty is a bad trade, so MAX_PAGE_ORDER is 10 and the largest buddy block is 4 MB.

So gigantic pages are not popped off a list; they are assembled by alloc_contig_pages(), which isolates a range and migrates whatever movable content is in it. That is why the requirement is not "free" but free or evacuable — and why one unmovable page defeats it, since evacuation has nothing to offer content that cannot move.

The existing anti-fragmentation machinery does not reach this scale. Migratetype grouping keeps unmovable allocations in unmovable pageblocks, and a pageblock is 2 MB. That is what makes 2 MB THP work. Nothing in it arranges for 512 consecutive pageblocks to stay free of kernel content, which is what a 1 GB range is. Worse, when a kernel allocation cannot find a pageblock of its own type, try_to_claim_block() steals a movable one and leaves behind whatever it could not move, so kernel content spreads rather than concentrating.

So 1 GB pages work today only if they are reserved at boot, through hugepagesz=1G hugepages=N or CMA, which takes the memory out of general use whether or not anything needs it. Allocating them on demand — growing a VM, starting a database, backing a guest after the host has been up for a week — is what does not work, and the measurements below say why.

1.1 The measured baseline

Confine kernel allocations so that PUD-aligned 1 GB ranges stay free or evacuable, and 1 GB hugepages can be allocated reliably.

Node 0 Normal, 822 GB present, 766 PUD ranges, 391808 pageblocks. The three do not reconcile and want re-deriving: 391808 pageblocks is 765 GB and 766 PUD ranges is 766 GB, so the span the gigablock formula divides is about 766 GB, which is what yields 12 GB gigablocks. Where 822 GB comes from is not established here; the derived numbers below use the span.

free                 25.9%
movable (LRU/anon)   61.8%
kernel (upper bound) 12.3%   101 GB, 24.7M pages

PUD ranges holding non-free non-LRU content    766 / 766  (100%)
PUD ranges with kernel pages in MOVABLE pbs    763 / 766  (99.6%)
ideal packing of kernel content                  94 PUDs  (12.3%)
ranges lost to placement alone                  672

Pollution is universal but thin: per range min 0.95%, p10 1.69%, median 10.73%, p90 19.79%. Only 3 ranges are under 1% polluted, so it is not a few ranges absorbing the damage.

Where the kernel pages sit, by pageblock migratetype (pages):

pageblock            free     lru/anon         slab     reserved     flagless
UNMOVABLE         5340945       289171      2309989         1946      4318301
MOVABLE          41342579    124333200      4779242      3281450      1954376
RECLAIMABLE       3921532       355517      8216906          320        59965

Four numbers drive the design:

• 39 GB of kernel pages sit in MOVABLE pageblocks, in 99.6% of ranges. Stopping that leak is the series' job; pageblock-type accounting alone understates the problem.
• 36 GB of free memory is stranded in kernel-typed pageblocks. Space to absorb kernel allocations already exists and is not being used, which is why fresh pageblocks keep being claimed.
• 12.8 GB of the leak is the vmemmap struct page array, allocated from memblock before pageblock types are set and permanently pinned.
• A third of the kernel pages are in RECLAIMABLE pageblocks, shrinkable rather than unmigratable, so the two need separate accounting.

The 12.3% is an upper bound: it counts reclaimable slab, movable_ops pages (zsmalloc/zswap pool, ~529k pages here, migratable), and vmemmap. Refining it needs a page_has_movable_ops() check plus slab-cache attribution in the drgn pass.

1.2 The current mechanism does not contain fragmentation, measured two ways

The machinery this series replaces is migratetype fallback plus should_try_claim_block(), so the case for the series rests on showing it does not work.

Observational. All 766 PUD ranges hold non-free non-LRU content while the kernel content would fit in 94. 672 ranges lost to placement alone.

Causal, in a VM, in 36 seconds. ~/debug/gigablock-phase0/vm-fallback-experiment/, unmodified kernel, ftrace hist triggers only. Normal zone pageblock census before and after a concurrent kernel-plus-movable churn workload:

Unmovable      36 ->  642    (+606)
Reclaimable     4 ->   22    (+18)
Movable     31192 -> 30568   (-624)

Kernel content held at the end: 171 MB unreclaimable slab, 0.5 MB page tables, 41 MB reclaimable slab. Densely packed, 172 MB of unmovable content is 86 pageblocks. It occupied 642 — 7.5x more territory than content. Every traced fallback was a kernel allocation converting a movable pageblock, and 306 of 317 were order 0:

alloc_mt=UNMOVABLE   <- MOVABLE, change_ownership=1 : 309
alloc_mt=RECLAIMABLE <- MOVABLE, change_ownership=1 :   7
alloc_mt=RECLAIMABLE <- UNMOVABLE                   :   2
alloc_mt=MOVABLE     <- anything                    :   0

should_try_claim_block() returns true unconditionally for UNMOVABLE and RECLAIMABLE at any order, so one order-0 kernel allocation that misses its own free list converts a whole 2 MB pageblock. The sentence for the cover letter: the current policy spends territory in 2 MB units to satisfy 4 KB requests, and nothing bounds how much it spends.

And half the conversions cannot currently be seen. The census moved 624 blocks against 318 traced conversions. The residual comes from paths that emit no tracepoint — change_pageblock_range() on the __free_one_page() merge path, and the highatomic reserve and unreserve paths. set_pageblock_migratetype() and set_pfnblock_flags_mask() are both inlined, so they cannot be kprobed either. Closing that gap is Phase -1.

2. Design in one page

• A gigablock is round_up(zone_span / 64, PUD_SIZE), at most 64 per zone, 12 GB on this host. Boundaries are PUD-aligned and gigablocks tile the zone, so every pfn in a gigablock zone is inside exactly one and its free pages live on that gigablock's lists. A gigablock is a free-list container and search unit, nothing more.
• Gigablocks exist in ZONE_NORMAL only, when it spans at least 8 of them, for memory present at boot. On this host that is one zone of five, so zone->gb == NULL is the common case and the path through it stays exactly mainline's, in cost as well as behaviour. ZONE_DMA and ZONE_DMA32 are device-constrained memory that 32-bit devices depend on, and a 1 GB page from there is worth nothing. ZONE_MOVABLE needs no confinement because every page in it is migratable already. ZONE_DEVICE and hot-added memory are out of scope for v1. Below 8 gigablocks a tainted set and a clean set have no room to coexist. Details in §10.3.
• What is tracked per gigablock is how much kernel content it holds, as a count of pageblocks. A gigablock is tainted by its first kernel pageblock, and graded by how full it is.
• What is tracked per pageblock is content class, in three bits: PB_has_unmovable, PB_has_reclaimable, PB_has_movable. All three clear means no allocated content.
• Whether a PUD range can host a 1 GB page is derived, never stored: the range must lie wholly in this zone, contain no CMA pageblock, and have no pageblock with either kernel bit set.
• Every choice is a bitmap query. Kernel allocations search the tainted set, movable searches the clean set first, and the one positional decision — which clean gigablock to spoil — takes the one with the least left to lose. No sweep directions and no cursors, because the bitmap states directly what direction was only a proxy for.
• Headroom watermarks per tainted gigablock decide whether a sleepable kernel allocation must evacuate first and whether background evacuation is kicked. They do not gate movable: movable competes only for content-free pageblocks, and §6.2's barrier decides its access to those.
• Tainted describes a gigablock, content bits describe a pageblock. A gigablock is tainted when nr_kernel_pageblocks > 0; a pageblock holds content classes. The two are never the same statement.
• Evacuation creates space rather than selecting it: it migrates movable content out, and never reclaims. This is the load-bearing mechanism; the bitmaps are the cheap index telling it where to work.

Success metric: the number of PUD ranges passing the derived candidate test, against the 94 that §1 says the kernel content actually needs.

What the prequel does, and does not, contain. The pageblock content bits and the pageblock-scale evacuation helper are useful without any of the above and ship first (§13 Phase P). Their claim is narrower and entirely local: a pageblock stolen for kernel use should become usable by that use, rather than staying half-full of movable pages that push the next kernel allocation into stealing another one. Nothing in the prequel knows what a gigablock is.

3. Data structures

Shapes, sizes, who writes and who reads each field are tabulated once in §3.0; every later mention of a name resolves against that table.

struct zone {
        ...
        struct gigablock_data *gb;      /* NULL when this zone has none */
};

struct gigablock_data {
        u8              nr_gigablocks;          /* <= 64 */
        unsigned long   gb_size_pages;          /* multiple of PUD_PAGES */
        unsigned long   start_pfn;              /* PUD-aligned */

        /* one bit per gigablock, so exactly one word each */
        unsigned long   has_free[MIGRATE_TYPES][NR_PAGE_ORDERS];
        unsigned long   gb_tainted;             /* nr_kernel_pageblocks > 0 */
        unsigned long   gb_fullness[2];         /* §3.4, two buckets */
        unsigned long   gb_above[2][__NR_GB_WATERMARKS]; /* per class, §6.1 */
        unsigned long   gb_evac_failed;         /* TAINTED_FULL (§6.2) */
        unsigned long   gb_evac_retry_at;       /* jiffies, §6.2 retry */

        struct gigablock gb[];                  /* nr_gigablocks entries */
};

struct gigablock {
        struct free_area free_area[NR_PAGE_ORDERS];
        u32             nr_free;                /* free pages */
        u16             nr_kernel_pageblocks;   /* pageblocks the kernel cannot move */
        u32             nr_free_kernel;         /* free pages on kernel lists, §6.1 */
};

Per pageblock, three bits in the existing flat pageblock_flags array. One accessor performs the list operation, both counters and the has_free bit flip, so they cannot diverge.

The four levels, because the search unit and the protected thing are not the same size. A gigablock is 12x coarser than the PUD range a hugepage needs, which is why a gigablock's fullness says nothing about how many of its ranges survive (§4):

zone                       ~766 PUD ranges, 64 gigablocks
 +-- gigablock  12 GB      the search unit: one bit in every mask of §3
      +-- PUD range  1 GB  what a 1 GB hugepage needs: 12 per gigablock
           +-- pageblock 2 MB   512 per PUD range, 6144 per gigablock
                +-- page  4 KB

Total per zone: has_free is 6 x 11 words = 528 bytes, plus gb_tainted, gb_evac_failed, gb_evac_retry_at, two fullness words and eight watermark words of which seven are used, so 632 bytes of index against zone->free_area[]'s 1144 bytes — still under half the size of the structure it indexes.

3.0 Shape, size and who touches what

The structures above are declared once and then referred to by name for the rest of this document: has_free appears in six later sections, gb_tainted in seven. A reader meeting a name in §5.1 cannot tell from the surrounding text whether it is one bit per gigablock or one per pageblock, and the two are 6144x apart. This table is the one place that answers that, and every later mention resolves against it.

Reference host throughout is §1's: a ZONE_NORMAL spanning ~766 GB, 766 PUD ranges and 391,808 pageblocks, tiled by 64 gigablocks of 12 GB. Sizes below use the tiling, so 6144 pageblocks per gigablock and 64 x 12 GB = 768 GB of tiling over a slightly smaller span; the last gigablock is the one that overhangs (§10.2a).

The index — one word per mask, one bit per gigablock. In gigablock_data, reached through zone->gb. Every mask is exactly one unsigned long, because §3.2 caps gigablocks at 64.

The fullness numerator differs by population and is easy to swap: nr_kernel_pageblocks/total for a tainted gigablock, grading concentration; (total - nr_free)/total for a clean one, grading how little is left to protect (§3.4).

| gb_above[2][4] | 64 B, 7 used | gigablock_watermark_update() (§6.1, §6.2) | §4.1, indexed by request class | hot | | gb_evac_failed | 8 B | set by evacuation on failure; cleared by the free accessor and the 10 s retry (§6.2) | §4.1, evacuation trigger | hot read, hot clear, slow set | | gb_evac_retry_at | 8 B | the evacuation trigger (§6.2) | same, lazily | slow | | total | 632 B | | | |

What the allocation fast path actually reads:

elig = gb_tainted & gb_above[class][LOW] & ~gb_evac_failed;  /* 3 words */
cand = elig & has_free[mt][order];                     /* 1 word  */
gb   = rank_by_fullness(cand);                         /* 2 words */

Six unsigned long loads, no per-pageblock access and no counter reads. Only one word of has_free's 528 B is touched per query, the [mt][order] cell, so the five single-word masks should be laid out adjacent and ahead of has_free: the query then costs one cache line plus one line of has_free, where the other order costs two lines for the same six words. The free side adds one conditional clear of gb_evac_failed, almost always already clear.

Per gigablock, same allocation as the index (§10.1a):

Per pageblock is a different array, and this is the distinction the table exists for. The three content bits are not in gigablock_data; they are three spare bits in the pre-existing flat pageblock_flags, one entry per pageblock, 384 KB for this zone. __NR_PAGEBLOCK_BITS is 5 today and NR_PAGEBLOCK_BITS is roundup_pow_of_two(5) = 8, so the design uses all three spare bits (§3.2).

Per-pageblock reads happen in the background contraction pass that re-derives stale-set bits (§3.3), and in the deferred claim cursor of §4a if it is ever built. The allocation-path search of §4.1 touches none of it.

Not structures, though §6 and §7 read like they are: headroom_pages and headroom_blocks are quantities computed per gigablock (§6.1). gb_above[] is the min over both, folded into a mask at update time, which is why it has four levels and not seven.

Assumptions these sizes rest on. A change to one invalidates the numbers above rather than quietly staling them:

• 8 bits per pageblock, giving 384 KB of pageblock_flags and 6 KB of content bits per gigablock. One unsigned long per pageblock instead would make those 3 MB and 48 KB and remove the three-spare-bit ceiling; it would not touch §4.1.
• At most 64 gigablocks per zone, which is what makes every mask one word and what §10.1's sizing exists to hold.
• 6144 pageblocks per gigablock. §4a's deferred cursor is a u16, so gigablocks above 128 GB would need a wider one.
• struct free_area is 104 B, six list_head plus nr_free. The 1144 B and 72 KB figures move if MIGRATE_TYPES changes.

Which patch introduces what (§13), so an implementer knows what must exist before a name is usable:

3.1 Why a pointer rather than fields in struct zone

The layout argument is the strongest and it is not about memory. struct zone has CACHELINE_PADDING(_pad1_) before free_area[] and _pad2_ after lock and trylock_free_pages, deliberately isolating the write-intensive allocator fields. Inlining 544 bytes there moves lock and trylock_free_pages onto different cache lines than today, on every zone of every node including those that will never have a gigablock. That is an unrelated performance change smuggled into the series. A pointer perturbs nothing.

NULL is a better "absent" than a count. zone_has_gigablocks() becomes a NULL test, the boot ordering problem gets a natural "not built yet", and every scope exclusion in §10 collapses to leaving the pointer NULL. The !CONFIG stub is a NULL too.

One allocation, contiguous. Bitmaps and gigablock array come from a single allocation, so the index and the data it indexes share pages and there is one lifetime to reason about. At boot that allocation is memblock_alloc_node(); see §10.1a.

Costs, to measure rather than dismiss: one extra dependent load on the allocator fast path, and two cache lines instead of one on first touch. Mitigate by placing zone->gb inside the existing _pad1_ region next to free_area and lock, so the pointer arrives on a line the allocator has already touched. One load against the ~18 cache lines mainline's ascending free_area scan already touches.

3.2 Maintain a bitmap only if it is read on a hot path and cannot be derived

Every maintained bit is a coherence bug waiting to happen. Each has to earn its place against two questions: is it read often enough that deriving it would cost something, and can it be derived from state already maintained.

What survives. has_free[mt][order] is read on the allocation fast path and cannot be derived without walking the free lists. gb_tainted and the gb_fullness masks are read by every fallback query and although all are functions of nr_kernel_pageblocks, deriving a 64-bit mask would mean reading 64 counters. gb_tainted flips once per gigablock lifetime and a gigablock moves between fullness masks only on a bucket crossing, so both are cheap to keep.

What was dropped, and why, so nobody adds it back.

• A per-PUD "clean" bitmap. Nothing on any hot path consults it; the 1 GB target search and the headline metric are its only consumers, both slow paths, and both can run the derived test of §10.2 at 512 bytes of reads per candidate range.
• A gb_eligible bitmap. §10.2's three-condition test covers every case, and two of the three conditions are structural constants the test can check directly.
• A fourth pageblock bit for "no content". __NR_PAGEBLOCK_BITS is 5 today and NR_PAGEBLOCK_BITS is roundup_pow_of_two(5) = 8, so there are exactly three spare bits. A fourth would round to 16 and double pageblock_flags — 392 KB to 784 KB per zone on this host. It is also unnecessary: "no allocated content" is precisely the state where all three content bits are clear.
• A derived has_free_any[order] to save the ORs in §4's queries. A bit that is the OR over migratetypes can only be cleared once every migratetype's list at that order is empty, which needs a count or a rescan per clear. Two ORs at query time are cheaper.
• A has_content_free_pageblock bitmap. A content-free pageblock is a fully free one, and a fully free pageblock coalesces to pageblock_order — §3.3 already sets PB_has_unmovable on reserved, offline and poisoned pages, so the cases that cannot coalesce are not content-free either. It is therefore the OR of has_free[mt][order] over every migratetype and every order at or above pageblock_order, which is 12 words on this configuration:

  static bool gb_content_free_pageblock(gbd, order_rows)
  {
          unsigned long m = 0;
          for (o = pageblock_order; o < NR_PAGE_ORDERS; o++)
                  for (mt = 0; mt < MIGRATE_TYPES; mt++)
                          m |= gbd->has_free[mt][o];
          return m;
  }
  

Keeping it as a bitmap would trade those reads for a write on every pageblock content transition, on a line every CPU freeing into the zone touches. Reads scale, contended writes do not, and the rows being read are written anyway for the search. The minimal set is also easier to justify upstream. - Group population counts and a zone-level summary word, and any third level above them. Those exist to make a wide level-0 bitmap cheap to scan. With at most 64 gigablocks the level-0 bitmap is one unsigned long, so a search is one load and one find_next_bit(), and every level above it is machinery with nothing to do.

3.3 The three content bits

Introduced by the prequel (§13 P2), where their consumer is the evacuation trigger. The gigablock series adds readers: nr_kernel_pageblocks, the taint threshold, and the derived candidate test of §10.2.

has_unmovable  has_reclaimable  has_movable   meaning
          0            0            0         no allocated content
          0            0            1         movable only: evacuable
          0            1            0         reclaimable only: shrinkable
          0            1            1         reclaimable + movable: both mechanisms
                                              apply, can still reach empty
          1            0            0         unmovable only: permanent
          1            0            1         unmovable + movable: blocks a hugepage,
                                              evacuation cannot fully empty it
          1            1            0         mixed kernel: shrinking cannot fully
                                              empty it
          1            1            1         all three: neither mechanism empties it

Eight states in three bits, which is every distinction §5, §6 and §8 need, at zero additional memory.

The dividing line is has_unmovable. The four states with it clear can still reach "no allocated content" — shrinking empties the reclaimable part, evacuation migrates the movable part — so those pageblocks are recoverable supply for headroom_blocks. The four with it set cannot, whatever runs. That is the distinction §5.1 step 5's asymmetry turns on: putting content into a pageblock that already has has_unmovable set costs nothing, putting unmovable content into one that does not costs its recoverability.

Reserved, offline and poisoned pages set PB_has_unmovable. The bit means "this pageblock holds memory the kernel cannot move", not "an unmovable allocation happened here", and the largest source of unmovable content on a real machine never goes through the page allocator: PageReserved from memblock covers the vmemmap, kernel image, initrd, crashkernel, firmware regions, and the hole pages init_unavailable_range() marks.

Without tainting them, a gigablock holding nothing but vmemmap reports zero kernel pageblocks, is classed clean, and is then protected as a 1 GB candidate it can never satisfy. Set them at init in one sweep over for_each_reserved_mem_range() — a handful of ranges, not a walk over pages — ordered after the gigablock array exists. PageOffline and hwpoison need the same treatment at their own hooks; memory_failure() is the hook for the latter, and a single poisoned page kills a range permanently.

Clearing is conservative and corrected on a slow path. All three bits clear together when a pageblock has no content left. That is exact for the common case but leaves one hole: a pageblock whose kernel content was freed while movable content remains keeps PB_has_unmovable set, so nr_kernel_pageblocks never decrements and the gigablock never returns to clean — precisely the "kernel had a lot of memory and then freed it" case. Per-content-class counts would fix it exactly and cost a 784 KB per-zone array. Instead: the bits are a hot-path hint that may be stale-set and never stale-clear, and the background contraction pass re-derives them for the gigablock it examines. Contraction happens, just not instantly.

3.4 Taint on first contact, graded by fullness

gb_tainted  =  nr_kernel_pageblocks > 0

GB_MOSTLY_DEDICATED   at or above ~25% of that class
GB_MOSTLY_EMPTY       below it

For a tainted gigablock the numerator is nr_kernel_pageblocks, since concentrating kernel content is what the bucket is steering. For a clean one it is total - nr_free, which measures how little is left to protect. This is the gb_lists[category][fullness] structure from the v28 superblock implementation, and both halves of it are load-bearing.

Taint on first contact, because a threshold smears. Under a 25% rule a kernel allocation spilling into a clean gigablock leaves it clean, at 1 pageblock out of 6144. Nothing then attracts the next kernel allocation there, so it opens a new front, and the one after that a third. Kernel content spreads across many gigablocks, every one below the threshold, none tainted, all unable to host a 1 GB page. The threshold meant to bound the damage spreads it instead. Tainting on the first kernel pageblock makes the gigablock a magnet: the second allocation follows the first.

Fullness, because taint alone latches. A bare > 0 bit is a one-way write-off of 12 GB triggered by one stray page. A boot-time burst or a percpu allocation touching every gigablock would taint the whole zone and leave no clean set, so the design would do nothing. The fullness bucket restores the gradation the bit throws away: a gigablock tainted by one allocation is TAINTED, MOSTLY_EMPTY and sorts last among tainted gigablocks, so it is still the one the kernel reaches for last, still worth protecting, and still the prime evacuation candidate — without needing a second classification to say so. The vmemmap case makes the point concretely: the gigablock holding the vmemmap's last 0.84 GB is 7% kernel and holds eleven good PUD ranges, and it lands in TAINTED, MOSTLY_EMPTY, which is exactly where it belongs.

Write-off therefore stops being a predicate. "Beyond saving" is exhaustion, not a bucket: no content-free pageblock and an empty has_free row, which is §9.3's own split of density as a cheap predictor and exhaustion as the hard gate. Everything short of that shades into "recoverable", so the decisions that used to test gb_tainted — protect it, admit movable, target evacuation — test the bucket instead.

Two buckets, because every consumer wants an extreme rather than a rank. Steps 1a, 1b, 7, 8 and the spill all ask for the fullest; evacuation asks for the emptiest; nothing asks to tell 50-74% from 25-49%. Five buckets, which is what v28 used, cost five word ANDs per ranking and four boundary crossings per gigablock as it fills, each crossing a clear and a set under zone->lock. Two cost two ANDs and one crossing, and the magnet argument above survives intact at a 25% boundary.

Evacuation ranks on the counter, not on a mask. Inside MOSTLY_EMPTY the cost of evacuating spans three orders of magnitude, one kernel pageblock against 1474 at 24%. But gigablock_evacuate() is the sleepable path choosing one gigablock, so reading 64 u16 counters is 128 bytes. §3.2's "deriving a 64-bit mask would mean reading 64 counters" is a fast-path argument and does not apply here; ranking on nr_kernel_pageblocks directly is finer than five buckets ever were, with less state.

The two boundaries are independent constants. The numerator is nr_kernel_pageblocks/total for tainted, grading concentration, and (total - nr_free)/total for clean, grading how little is left to protect. Different questions, so the values are not tied to each other. ~25% each is a starting point to measure, not a constant to defend.

Hysteresis is deferrable. A gigablock hovering at a boundary clears and sets two mask words per alloc-free pair. Adding hysteresis needs no new state, since the previous bucket is already recorded by which mask holds the bit, so it is a comparison change in one function and can land whenever the counters show it is worth having.

Open: the bitmap index caps gigablocks at 64, which is what forces 12 GB granularity here. §3.2's one-word-per-mask design needs at most 64 gigablocks per zone, so §10.1 sizes them at round_up(zone_span / 64, PUD_SIZE) — 12 GB, twelve PUD ranges, on a 768 GB machine. v28 had no such cap because it used lists rather than masks, and its gigablock was one PUD range. The bitmap buys O(1) mask queries and costs granularity on exactly the machines the series targets; per-fullness lists buy granularity and cost list operations under zone->lock. Measure before assuming the bitmap wins.

4. Search mechanism

One load and one find_next_bit() per query. With at most 64 gigablocks every per-gigablock bitmap is a single unsigned long, so there is no hierarchy, no population count, no summary level and no cursor. [mt][order] is the right layout because both scans that matter iterate orders for a fixed migratetype: 11 contiguous words, 88 bytes, 2 cache lines covering every order.

Each population searches its own set. Kernel allocations search gb_tainted & ...; movable searches ~gb_tainted & ... first and considers tainted gigablocks only under the §6 watermark rules. That is what direction used to approximate — "prefer the gigablocks that are already lost" — so the bitmap replaces it, and no cursor or sweep order is needed. Contiguity of the clean set is worth nothing here either, because a 1 GB hugepage needs one PUD range and ranges are independent.

The queries are where the policy lives. has_free[mt][order] answers only "which gigablock has a free block of this order and type", which is step 1 below. Every fallback step needs a different candidate set, and handing those steps to mainline's __rmqueue_claim() would be gigablock-blind and would claim in a clean gigablock — the one thing the design exists to prevent. So each step names its set:

fb = fallbacks[start_mt]      /* {RECLAIMABLE, MOVABLE} for UNMOVABLE */

step 1   elig & has_free[start_mt][order], then rank by fullness (§4.1)
           elig = gb_tainted & gb_above[...] & ~gb_evac_failed
step 1'  same, with ~gb_tainted                         then fullest clean
step 2   gb_tainted & gb_content_free_pageblock()
step 3   gb_tainted & (has_free[fb[0]][o] | has_free[fb[1]][o])
step 4   as step 3, largest o that hits
step 5   gb_tainted & (has_free[fb[0]][order] | has_free[fb[1]][order])
step 7   ~gb_tainted & gb_content_free_pageblock()      last resort
spill    ~gb_tainted, fullest bucket first                    which clean one to spoil

Two or three word operations per step, all on paths that are already the slow ones. A fallback query touches fewer cache lines than mainline's existing ascending free_area scan does on its fast path today, so the index is a reduction rather than a cost to apologise for.

Which clean gigablock to spoil is a quality question, not a positional one. Take the fullest bucket — least left to protect, so the fewest usable candidate ranges to lose. A gigablock with several structurally doomed ranges is a better target than a pristine one.

Order-of-orders is unchanged from upstream: ascending for the matching migratetype, descending on the claim path where the largest block is the existing proxy for "the pageblock with the most free pages". Both are orthogonal to the gigablock dimension.

Which pageblock inside a gigablock: today it is whatever the list head offers, and that is not good enough. A claim takes get_page_from_free_area(&gb->free_area[pageblock_order], mt), so the pageblock is whichever was freed most recently — arbitrary with respect to position.

The gigablock is 12x coarser than the thing being protected. A 12 GB gigablock is 6144 pageblocks and 12 PUD ranges, and 1536 kernel pageblocks — the bucket boundary, exactly 3 ranges' worth — say nothing about where those pageblocks sit. Packed into 3 ranges that gigablock still offers 9 candidate ranges; smeared evenly across all 12 it offers none:

1536 kernel pageblocks, 12 PUD ranges in the gigablock

packed    [KKK][KKK][KKK][   ][   ][   ][   ][   ][   ][   ][   ][   ]
           3 ranges spoiled                    9 candidates survive

smeared   [K  ][K  ][K  ][K  ][K  ][K  ][K  ][K  ][K  ][K  ][K  ][K  ]
           every range spoiled                 0 survive

Leaving the choice to LIFO reproduces §1.2's spend-territory-in-2 MB-units pathology one level down.

V1 accepts that, and the arithmetic says it can afford to. 64 gigablocks tile 766 PUD ranges against a target of 94, so 8 clean gigablocks meet the target on their own and up to 56 of 64 may be tainted before within-gigablock position matters at all. At the 8 tainted gigablocks this host expects, the clean set alone yields 672 ranges, 7.1x the requirement, and dense packing inside the tainted 8 adds at most 72 marginal ranges to a surplus of 578. It cannot change the outcome.

Within-gigablock packing only becomes load-bearing once the tainted set passes roughly 56 gigablocks, and at that point concentration has already failed badly enough that packing the wreckage is treating a symptom. So v1 ships without it, and the bet is stated rather than hidden: v1 assumes the tainted set stays small, which is exactly what taint-on-first-contact (§3.4) and fullness steering exist to deliver, and exactly what v1 is already measuring.

The trigger for building it is the success metric of §1 falling toward 94 with clean gigablocks alone, or equivalently the tainted count passing ~48 of 64, leaving margin. Both are in the debugfs dump of Phase 1 patch 8.

4a. Deferred: the per-gigablock claim cursor

Kept because if the trigger above fires, re-deriving this costs more than reading it.

One u16 per gigablock recording the pageblock index of the last claim; a claim takes the first content-free pageblock at or after it, wrapping. Apply it to the steps that pick a content-free pageblock, steps 2 and 7 of §5.1, where every candidate is equivalent and position is the only thing to choose on. Leave step 4 taking the largest fallback block, where the existing largest-first heuristic is doing real work and position is secondary.

This is a cursor, which §3.2 rejected for choosing between gigablocks. The difference is that a bitmap answers the inter-gigablock question directly and there is no bitmap for position within a gigablock, so a cursor is the right tool for the second question and the wrong one for the first.

It is a linear scan, and the amortised claim holds in only one direction. While a gigablock fills, the cursor moves forward and wraps once, so across n claims the scan visits each pageblock about once: O(1) per claim in aggregate, and the next pageblock is usually the one. In a nearly-full fragmented gigablock every claim walks a long run of allocated pageblocks and wraps, and there is no amortisation left. Nearly-full is not an exotic case, since §5.1 prefers tainted gigablocks precisely because they are already spoiled.

What makes it survivable is the shape of the scan rather than its length: a contiguous byte array, one byte per pageblock, 6144 bytes and 96 sequential prefetchable cache lines at worst, against the same length in dependent cache misses for a free-list walk. "All three content bits clear" is a mask test, so one 8-byte load screens 8 pageblocks and the worst case is 768 loads.

Bound the walk anyway. The cursor is a packing heuristic, not a correctness requirement, so nothing breaks if it occasionally picks a worse pageblock. Scan at most N pageblocks from the cursor and fall back to the free-list head, which bounds the work under zone->lock and makes the degradation traceable: count the fallbacks, and a high rate says N is too small before the density number goes bad.

Not taken: a per-gigablock content-free bitmap, 6144 bits = 768 B each and 48 KB per zone, where find_next_bit() makes the scan 96 words. It passes §3.2's test — read on a hot path, and deriving it is the scan being bounded — but 48 KB and a second coherence point against the content bits is a steep price for a heuristic a bounded walk already handles.

4.1 Eligibility is one mask; fullness only ranks the survivors

Written literally, step 1 is a nested loop — fullness buckets by eleven orders, a word AND per cell before the first candidate on a miss. That is the wrong shape, because the two dimensions are not the same kind of thing. Eligibility is correctness: a gigablock below LOW may not take movable, and TAINTED_FULL has nothing to give. Fullness is preference: among gigablocks that may all serve the request, one is a better victim. So test eligibility once, and rank only what survives.

/* one word per level, cumulative: above[HIGH] subset of above[LOW] ... */
unsigned long gb_above[__NR_GB_WATERMARKS];

class = (order >= 4) ? GB_LARGE : GB_SMALL;         /* §6.1 */
elig  = gb_tainted & gb_above[class][LOW] & ~gb_evac_failed;
cand  = elig & has_free[mt][order];
if (cand)
        gb = rank_by_fullness(cand);

Fold each class's capability into its mask at update time. §6.1 grades order 0-3 on headroom_pages + headroom_blocks * (1 << pageblock_order) and order 4 and above on headroom_blocks, so a query that computed either would be doing at every allocation what a bit flip does once per crossing. The class is a property of the request, known without a load, so indexing by it costs nothing: the query is three loads either way.

Cumulative rather than one word per band. The hot path asks "may this gigablock serve me", which is an "at least" question and one load. The background evacuation worker asks "which gigablocks are in the EVACUATE band exactly", which is gb_above[EVAC] & ~gb_above[LOW] — two loads and an andn, on a path that already sleeps. Exclusive per-band words would invert that trade and cost two words per transition instead of one.

Every hard exclusion belongs in the same word, not in a branch. TAINTED_FULL is the one the search would otherwise rediscover per query, since "evacuation failed here and nothing has been freed since" is a property of the gigablock, not of the request. Anything the request decides — ALLOC_CMA, ALLOC_HIGHATOMIC, ALLOC_NOFRAG_TAINTED_OK — stays a per-request mask combined at query time, because a per-request bit maintained per gigablock is a coherence bug with extra steps.

What not to add. A precombined mask per search step: there are nine steps across two migratetypes and the combinations multiply, while each is one or two ANDs of words already in L1 from the same kvmalloc (§3.1). This is §3.2's rule applied to the watermarks — maintain a bit only if it is read on a hot path and cannot be derived cheaply from state already maintained.

Maintenance is the existing accessor's job. The watermark words flip only when a gigablock crosses a level, under zone->lock, in the same helper that already updates nr_free, nr_kernel_pageblocks, the has_free bit and the fullness bucket. A second writer is how these get out of step, and CONFIG_DEBUG_VM should assert gb_above[] against a recomputation the way it already does for has_free.

5. Allocation policy — the complete search order

Every requesting migratetype, every class of gigablock, every kind of pageblock. Oversights here caused bugs in earlier implementations, so nothing may be left to "and then it falls through to the existing code".

5.0 The classes being searched

Gigablocks. Two classes, plus one whole-zone case:

tainted        nr_kernel_pageblocks > 0, graded MOSTLY_EMPTY / MOSTLY_DEDICATED
clean          no kernel pageblocks, graded the same way by total use
no gigablocks  zone->gb is NULL, every page on zone->free_area, behaviour is
               mainline's (§5.9, §10.3)

PUD ranges that can never host a 1 GB page live inside gigablocks like any other range, on gigablock free lists, and are entirely ordinary to the allocator: ranges with a memory hole, spanning two zones, containing CMA, dominated by the vmemmap, or currently isolated. They are a property of a range rather than a third class of gigablock, they matter only to the 1 GB target search, and §10.2 derives them rather than listing them.

Candidate pageblocks, and what taking one costs a kernel allocation. This is the cost model §5.1's order is derived from. Two independent properties decide it: migratetype, which says whether the pageblock may be converted at all, and content, which says what conversion leaves stranded.

pageblock migratetype   content bits         a kernel allocation may       cost
the requested type      any                  allocate from its free list   none
the other kernel type   any                  claim, or steal one block     near zero: kernel
                                                                           content either way
MOVABLE                 all three clear      claim                         best: 512 pages of
                                                                           headroom, nothing
                                                                           stranded
MOVABLE                 movable only, all    reclaim, which empties it,    no copy, no
                        clean page cache     then claim                    destination, no debt
MOVABLE                 movable only, not    claim: converts, strands      evacuation debt
                        all reclaimable      its movable content
MOVABLE                 movable only, not    steal one block, no           creates a mixed
                        all reclaimable      conversion                    pageblock
HIGHATOMIC              any                  nothing                       ALLOC_HIGHATOMIC only
CMA                     any                  nothing                       movable requests only
isolate bit set         any                  nothing                       not allocatable

A kernel allocation may use only the first six rows; HIGHATOMIC, CMA and isolated pageblocks are exclusions rather than options. migratetype_is_mergeable() is the kernel's own name for the line — it is mt < MIGRATE_PCPTYPES, so UNMOVABLE, MOVABLE and RECLAIMABLE may be converted and HIGHATOMIC, CMA and ISOLATE may not.

The "all clean page cache" row cannot be identified from the bits: PB_has_movable does not distinguish clean file from dirty file from anon. The step is therefore speculative in one specific sense — the helper already exists, since the prequel shipped it (§13 P4), and the cost of running it is what cannot be predicted, so run it on a movable-content-only pageblock and let it discover the cost. Do not add a bit for this, or the array doubles (§3.2).

5.1 UNMOVABLE and RECLAIMABLE requests

The numbered steps are a preference order, not a call chain. Steps 6 and above cannot run under zone->lock; the fast path returns no page and __alloc_pages_slowpath() re-enters (§9.3).

fb = fallbacks[start_mt], which is {RECLAIMABLE, MOVABLE} for UNMOVABLE and {UNMOVABLE, MOVABLE} for RECLAIMABLE.

 0  the structurally doomed but usable ranges (§10.2a), if any
      at most two per zone, fixed at init: the partial PUD ranges at the zone
      edges.  They can never host a 1 GB page and they hold real free memory,
      so kernel content there costs nothing and spares a good range.
1a  matching type in a tainted gigablock, fullest bucket first, order ascending
      gb_tainted & gb_fullness[f] & has_free[start_mt][o].  Fullest first
      concentrates kernel content where it already is (§3.4); split a larger
      block here rather than take a smaller one from a clean gigablock.
1b  matching type in a clean gigablock, fullest bucket first, order ascending
      ~gb_tainted & gb_fullness[f] & has_free[start_mt][o].  Spoils a gigablock
      nothing has touched, so it comes after every tainted option, evacuation
      included, and takes the one with least left to protect.
 2  tainted gigablock, content-free pageblock, claim
      gb_tainted & gb_content_free_pageblock()  (§3.2)
 3  tainted gigablock, movable-content-only pageblock: migrate out, then claim
      speculative, see §5.0
 4  tainted gigablock, largest fallback block, claim
      gb_tainted & (has_free[fb[0]][o] | has_free[fb[1]][o]), o descending
 5  tainted gigablock, steal a single fallback block, no conversion
      only while headroom_pages > MIN, and at the requested order only (§6.2).
      Prefer a pageblock with PB_has_unmovable already set: see below
 6  sleepable: shrink or evacuate inside the tainted set (§9.2), retry from 2
 7  clean gigablock, content-free pageblock, claim, fullest first
      ~gb_tainted & gb_content_free_pageblock().  The tainted set grows; trace this.
 8  clean gigablock, largest fallback block, claim, fullest first
      worst case: spoils a gigablock and strands movable content in it
 9  ALLOC_NOFRAG_TAINTED_OK retry, or ALLOC_OOM / ALLOC_RESERVES:
      skip all steering and take what mainline would take

Steps 5 and 6 are ordered by the watermark, not fixed. Step 9 is invariant 1 made concrete. §9.3 gives the rule for how hard to try steps 3 through 6 before reaching step 7.

Fullness ordering dissolves the step-1c question. An earlier draft split the clean class into "already holds kernel content" and "pristine" and could not place the pristine step: measured in gigablocks lost it belonged below steps 7 and 8, measured in pages converted it belonged above them. With a graded ladder there is no binary to place. Every step that touches the clean set walks the same fullness order, so the choice inside each step is "least left to protect first" and the choice between steps stays what it always was, cost per allocation.

Gigablock class outranks order within the matching migratetype. For an order-0 unmovable request with free order-0 pages only in clean gigablocks and a free order-6 block in a tainted one, take the order-6 block and split it. expand() puts one block of each intermediate order back on that gigablock's lists, so the residue serves the next several order-0 requests from the same already-lost gigablock, and the split costs nothing that matters. Taking the order-0 page from the clean gigablock costs up to a whole 1 GB range. So the search runs class on the outer loop and order on the inner, the same shape §5.2 uses for movable with the classes the other way round.

An earlier version of this section claimed step 1 "cannot spoil anything". That was wrong, and it mattered because step 1 is the common path. The claim came from __rmqueue_smallest() allocating within the requested migratetype, so no pageblock changes type. But a clean gigablock can hold an UNMOVABLE-typed pageblock with no content — one converted earlier and since emptied — and allocating there adds kernel content, sets PB_has_unmovable, raises nr_kernel_pageblocks and taints the gigablock (§3.4). Pageblock migratetype and gigablock taint are different things.

Stealing between the two kernel types is not symmetric, so step 5 prefers by direction. Both directions leave nr_kernel_pageblocks unchanged, which is why stealing beats claiming a fresh pageblock at the gigablock level — and with the claim cursor deferred (§4a) it also spoils no new PUD range, where a claim may. But the two directions differ in what they destroy:

• A RECLAIMABLE request stealing from an UNMOVABLE pageblock costs nothing. That pageblock has has_unmovable set and was never going to return to content-free, so adding shrinkable content to it changes nothing.
• An UNMOVABLE request stealing from a RECLAIMABLE pageblock costs that pageblock's recoverability. It moves from 010 to 110 in §3.3's table: every other object in it can still be shrunk and it will never be content-free again. §1 measures 8.2M slab pages in RECLAIMABLE pageblocks, a third of all kernel pages, and that is the supply §9.2 is trying to turn back into headroom_blocks.

So step 5 prefers a victim that already has PB_has_unmovable set, which is one bit test on a pageblock it is already looking at. The damaging direction stays available — it is still better than spoiling a clean gigablock at step 8 — but it is the last of the tainted-set options rather than an equal one.

Step 4 is the same choice with conversion, and there the answer is simply no: claiming a RECLAIMABLE pageblock for UNMOVABLE use loses the whole pageblock's recoverability rather than one block's worth, so step 4 should skip a 010 victim while step 5 may take one.

Step 1a covers the case that used to leak. A gigablock holding one kernel pageblock out of 6144 is tainted (§3.4), so 1a finds it and the next kernel allocation lands there rather than opening a new front. Its MOSTLY_EMPTY bucket sorts it last among tainted gigablocks, so a fuller one absorbs the allocation first if there is one — the magnet without the write-off.

1c is where a gigablock is lost, so it is the only step that needs a reason to run at all. Nothing below it is cheaper: every tainted option, including synchronous evacuation, is preferable to spoiling a range nothing has touched. §9.3 governs how hard to try those before reaching it.

The remaining imprecision is inside a bucket, and it grows with machine size. A gigablock is round_up(zone_span / 64, PUD_SIZE), 12 GB on this host — twelve PUD ranges. A fullness bucket says how much of the group is in use, not which of the twelve ranges holds it, so the search concentrates allocations into the right 12 GB and then picks blindly inside it. On a 32 GB machine a gigablock is exactly one PUD range and the blindness does not exist; on a 768 GB machine every allocation inside a bucket is a twelve-way coin flip. v28 did not have this problem because its gigablock was one PUD range; §3.4 records why the bitmap index reintroduced it.

Nothing else was guarding it either. should_try_claim_block() gates only the fallback paths, so §5.8's "clean, no permission -> no" never sees an allocation that comes off the matching-type free list. For step 1 the search order is the whole protection, which is why 1a and 1b are separate steps rather than one ascending scan.

Interaction with high-order headroom. Splitting the order-6 block consumes that gigablock's largest free run, which is the supply headroom_blocks exists to protect (§6.1). No extra rule is needed: if the split would take headroom_blocks below EVAC, that is the signal to evacuate or to try the next tainted gigablock, not to reach into a clean one. Ascending order within 1a already takes the smallest sufficient block, so the largest run is consumed only when nothing smaller exists.

5.2 MOVABLE requests

0  ZONE_MOVABLE, if populated.  Not a step in this zone's search at all:
     build_zonerefs_node() walks zone_type from MAX_NR_ZONES downward, so a
     GFP_HIGHUSER_MOVABLE allocation tries ZONE_MOVABLE before ZONE_NORMAL.
     Movable pressure reaches the gigablock zone only once ZONE_MOVABLE is
     exhausted, and ZONE_MOVABLE never has gigablocks.
1  CMA, when ALLOC_CMA and NR_FREE_CMA_PAGES > NR_FREE_PAGES / 2
     unchanged from mainline; CMA ranges never sit in a gigablock.
2  matching type in the clean set, order ascending
     ~gb_tainted & has_free[MOVABLE][o]
3  CMA fallback (__rmqueue_cma_fallback), mainline's RMQUEUE_CMA step
4  matching type in a tainted gigablock, emptiest bucket first, order
   ascending
     gb_tainted & gb_fullness[f] & has_free[MOVABLE][o].  Free pages in an
     already-spoiled gigablock cost nothing; emptiest first leaves the ones
     closest to recovery alone.
5  barrier: stop.  Return no page and let __alloc_pages_slowpath() reclaim.
   Kernel headroom is behind this, not the tainted set.
6  ALLOC_NOFRAG_TAINTED_OK retry: barrier released.  Only here may movable take
   kernel headroom, in this order:
     6a  steal a single block from a kernel pageblock at the requested order
         only, no splitting.  Consumes the stranded free space of §1.  Prefer
         the gigablocks with the most kernel content, so the nearly-recoverable
         ones are left alone.
     6b  claim a kernel pageblock.  Converts kernel territory back to movable,
         which shrinks the kernel footprint, but costs a whole pageblock of
         headroom -- hence after 6a.

Claim and steal are ordered opposite to §5.1 for a one-line reason: a kernel allocation wants the pageblock converted, so claim first; a movable allocation does not want to consume a whole pageblock of kernel headroom, so steal first.

The barrier sits between taking free memory and taking kernel headroom, not between clean and tainted. Step 4 is movable using pages that are simply free in a gigablock already spoiled, which costs the design nothing and is memory the machine has. Steps 6a and 6b convert or consume space the kernel is counting on, which is why they need the zone to have already failed: ALLOC_NOFRAG_TAINTED_OK records exactly that, the clean set could not serve the request and reclaim did not help.

When the kernel frees a lot and leaves half-empty tainted gigablocks, movable consumes them at step 4, without needing the barrier released at all, and that is right: the pages are genuinely free, those gigablocks are already spoiled, and refusing free memory in order to reclaim instead is the pathology §7 exists to prevent. Exclusion is graded by headroom, not absolute — nailing the door shut at the taint bit would strand that memory. The refinement is 6a's preference: consume the most spoiled gigablocks first, because a nearly-empty tainted gigablock is the prime candidate for returning to clean, and filling it converts free recovery into recovery that needs migration work.

What a populated ZONE_MOVABLE means for this series. It is the static version of what gigablocks do dynamically: movablecore/kernelcore carve a range kernel allocations may never enter, sized once at boot and never resized. With a large ZONE_MOVABLE the competition §1 measured is reduced and the series buys less; with none — both nodes here have one with zero pages, the fleet configuration — it buys the most. State that in the cover letter: gigablocks give the containment ZONE_MOVABLE gives, without having to size it correctly at boot or stranding the reservation when the workload changes.

5.3 HIGHATOMIC

alloc     __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC), only with
          ALLOC_HIGHATOMIC.  Never a fallback source, never falls back itself.
reserve   reserve_highatomic_pageblock() takes from an already-tainted gigablock:
          the reserve is long-lived and unmigratable, so it must not spoil a
          clean one.
drain     unreserve_highatomic_pageblock() returns the pageblock to its previous
          type.  nr_kernel_pageblocks must be updated on both transitions or
          invariant 7 drifts.
watermark its free pages are already excluded by __zone_watermark_unusable_free()
          and must also be excluded from headroom, for the same reason.

5.4 CMA

Movable requests only, MIGRATE_CMA lists only, never a claim or steal source or target because migratetype_is_mergeable() excludes it and fallbacks[] never names it. A PUD range containing CMA fails the §10.2 candidate test, so CMA never appears in a 1 GB target. It does live inside gigablocks like anything else, so has_free[MIGRATE_CMA][*] is maintained but never searched by §5.1.

5.5 ISOLATE

An isolated range is unallocatable whatever the policy thinks of it, so steering is unaffected. Free lists and counters still track it.

Isolation has two causes with opposite meanings. start_isolate_page_range() is called by offline_pages(), where the range is going away, and by alloc_contig_range(), where isolation is the success path of building a 1 GB page. A fully isolated PUD range is more likely the goal in progress than a disqualification. The persistent state cannot tell them apart — both set the same bit, and only the caller knows via MEMORY_OFFLINE. No consequence for allocation, one for reporting: counting a range mid-alloc_contig_range() as lost undercounts the outcome the series exists to produce. v27's contig_allocated/gigablock_contig_mark() is the record of the success case and is where the metric should read from.

5.6 Which bitmap rows are searched

UNMOVABLE, MOVABLE, RECLAIMABLE   searched by §5.1 and §5.2
HIGHATOMIC                        searched only for ALLOC_HIGHATOMIC
CMA                               maintained, never searched by kernel requests
ISOLATE                           maintained for list consistency, never searched

Stated explicitly because a search consulting the wrong row is exactly the class of oversight that bit earlier implementations.

The unsearched rows are still maintained, and that is the cheap option. The single accessor of §3 is uniform over migratetype: it flips has_free[mt][order] for whatever migratetype it was called with. So maintaining the CMA and ISOLATE rows costs one bit flip like any other, while skipping them would need a conditional on the hot path. The only cost is 176 bytes of the 528-byte array.

5.7 page_group_by_mobility_disabled

When mobility grouping is off, should_try_claim_block() returns true unconditionally and migratetype separation collapses. It is only set on systems too small to host a gigablock, so the two cannot coexist — but assert it rather than rely on it, because the coupling is invisible: VM_WARN_ON(page_group_by_mobility_disabled && zone_has_gigablocks(zone)).

5.8 What should_try_claim_block() becomes

Today it decides on order alone: true at order >= pageblock_order, true at order >= pageblock_order / 2, true for UNMOVABLE or RECLAIMABLE at any order, true when mobility grouping is off, false otherwise. So kernel allocations always claim and movable claims only at order 4 and above.

It gets rebased rather than deleted. For movable the order thresholds are right and stay unchanged. For kernel requests the order terms are already vestigial — every path returns true — so a gigablock-class term replaces them. One new input is needed: whether spoiling a clean gigablock is permitted yet, which is retry state rather than a property of the candidate. Signature becomes should_try_claim_block(order, start_mt, gb, alloc_flags).

UNMOVABLE / RECLAIMABLE request:
  tainted                          yes, any order   already lost, and claiming yields
                                                    512 pages of headroom.  This is
                                                    where claiming should be most
                                                    aggressive, the opposite of a
                                                    threshold.
  tainted, headroom < MIN          yes              still better than spoiling a clean
                                                    gigablock
  clean, no permission             no               the new behaviour and the point of
                                                    the series
  clean, TAINTED_OK or ALLOC_OOM   yes              invariant 1
  zone has no gigablocks           yes              today's behaviour, invariant 2

MOVABLE request:
  any class, order >= pageblock_order        yes    pageblock was entirely free
  tainted, TAINTED_OK, order >=              yes    converts kernel territory back to
  pageblock_order/2                                movable; §5.2 tries steal first
  tainted, no TAINTED_OK                     no     the barrier (§6.2)
  clean, order < pageblock_order / 2         no     unchanged from today
  zone has no gigablocks                     unchanged from today

Two things fall out. A movable request needs no gigablock-class term at all, because movable content never spoils anything — clean means free of kernel content, so a THP landing in a clean gigablock leaves it clean. The class dimension exists for kernel requests only. And for kernel requests the order dimension disappears: the answer is yes in every class except an unpermitted clean gigablock, at every order.

5.9 Zones without gigablocks

The common case, not an edge case. On this host node 0 has DMA at 4095 pages, DMA32 at 4.3 GB, Normal at 821.6 GB, Movable and Device empty, so one zone of five gets gigablocks and every allocation walking the zonelist crosses both kinds. The zone->gb == NULL path is hot and must be free.

pfn_free_area()          returns &zone->free_area[order]; every list operation is
                         mainline's, unchanged
for_each_free_area()     yields exactly one element, so the Phase 2 macro degenerates
                         to the loop it replaced
PB_has_* maintenance     skipped entirely.  The NULL test is already being made, so
                         this costs nothing, and it is what makes "small systems stay
                         bit-identical" true of cost as well as behaviour.
should_try_claim_block() today's answers exactly
watermarks               NR_FREE_KERNEL_TAINTED reads zero, so the arithmetic is
                         unchanged
headroom, evacuation     none.  No watermark states, no kcompactd wakeups.
alloc_contig_range()     works exactly as today: not blocked, just not helped
CONFIG_DEBUG_VM checks   no-ops for this zone, not failures
debugfs                  reports the zone as having no gigablocks rather than omitting
                         it, so selftests skip cleanly instead of failing to parse

Mixed nodes need nothing special — each zone answers for itself. Worth stating only because it means the NULL path is on the hot path for nearly every allocation, so it must be a predicted branch off an already-loaded cache line, not a function call.

6. Watermarks, states and the barrier

6.1 Headroom is two numbers

A page count says nothing about order capability: a gigablock can hold ten thousand free pages in kernel pageblocks, all at order 0, and fail every high-order kernel request.

headroom_pages    free pages in kernel-typed pageblocks, excluding highatomic
                  reserves.  Serves the must-succeed kernel allocations, which
                  are all order 3 or below.
headroom_blocks   content-free pageblocks in the tainted set.  Serves order 4
                  and above, and is what evacuation must produce for an
                  order >= pageblock_order request -- one at a time, through
                  the prequel's pageblock helper (§8).

Both numbers are kernel headroom, and neither gates movable. Movable competes for content-free pageblocks and for nothing else, so §6.2's barrier decides its access to them; there is no movable watermark.

Where they come from. headroom_blocks is exactly derivable from counters already kept, so nothing maintains it:

for (o = pageblock_order; o < NR_PAGE_ORDERS; o++)
        blocks += gb->free_area[o].nr_free << (o - pageblock_order);

headroom_pages is not derivable — struct free_area keeps one nr_free across all migratetypes — so struct gigablock carries u32 nr_free_kernel, free pages on the UNMOVABLE and RECLAIMABLE lists, written by the accessor that already writes nr_free. Same line, same lock, no new contention point.

Each caller is graded on the capability it can actually reach, not on the worse of the two numbers:

kernel order 0-3    headroom_pages + headroom_blocks * (1 << pageblock_order)
                    HIGH, LOW, EVAC, MIN
kernel order >= 4   headroom_blocks
                    HIGH, LOW, EVAC

An order-3-or-below kernel request can claim a content-free pageblock (§5.1 step 2), so content-free pageblocks are capacity for it too. Grading it on the worse of the two numbers would send a sleepable GFP_KERNEL allocation to migrate pages and shrink slab while thousands of pageblocks it could have claimed sat free — the common shape, since kernel pageblocks filling is what tainted means.

The two graded quantities cost 8 mask words against 4, 32 bytes with one word unused, and nothing on the fast path: the caller knows its own order and indexes the mask for its class, so the query is the same three loads (§4.1).

headroom_blocks needs no MIN because its callers are order 4 and above and already have MIGRATE_HIGHATOMIC as their reserve, whose free pages headroom_pages excludes for exactly that reason. At headroom_blocks zero a sleepable high-order kernel allocation spills to a clean gigablock (§5.1 step 8) and spoils it. That is the floor, stated rather than left as an absence.

Derive the thresholds from one ratio per quantity rather than exposing seven knobs.

headroom_blocks is the high-order reserve, built from what already exists. A content-free pageblock inside a tainted gigablock already is one, because claiming it yields an order-9 block, and §5.1 step 2 already prefers exactly those, so maintaining the watermark is the whole reservation. Compare MIGRATE_HIGHATOMIC, which is the same idea for atomic allocations with its free pages excluded from the zone watermark — the difference is that this reserve is usable by ordinary kernel allocations rather than held idle, so an idle system does not pay for it. Start headroom_blocks EVAC at 2 to 4 pageblocks per active gigablock, enough for a vmalloc order-10 attempt, and measure.

6.2 Exclusion, the evacuation trigger, and the barrier

Movable is not gated on kernel headroom; the barrier gates its access to kernel headroom. The line that matters is §5.2's, between taking free memory and taking space the kernel is counting on. Movable taking free pages from movable-typed pageblocks of an already-tainted gigablock (§5.2 step 4) consumes no content-free pageblock and no kernel-typed free page, so it costs the design nothing and is memory the machine has. Refusing it leaves that memory unused and sends the allocation to spoil a clean gigablock instead, which is strictly worse.

What protects the tainted set from movable is ordering, not a watermark. Step 4 takes the emptiest bucket first, so movable lands in the gigablocks least worth saving and leaves the nearly-recoverable ones alone. That is a per-gigablock preference rather than a cliff that treats every tainted gigablock alike.

The barrier sits between §5.2 step 4 and step 6, and it is ALLOC_NOFRAG_TAINTED_OK, not a watermark: only after the clean set has failed and reclaim has not helped may movable steal from a kernel pageblock (6a) or claim one (6b). Movable content is migratable by definition and §10.3 guarantees it most of the zone, so this is the reversible kind of damage (§7).

The cost admitted here is that evacuation later has more pages to move. It is bounded by the same emptiest-first ordering, and moving movable content is what evacuation is for.

Fullness orders, headroom admits. The two now overlap enough to be worth separating explicitly, and it is §3.4's lesson applied again — one number cannot answer two questions. The fullness bucket decides which gigablock the search prefers, and it is a pageblock count because concentration is about position. The headroom watermark decides whether movable may enter the one it picked, and it is a free count because admission is about capacity. A gigablock can be MOSTLY_EMPTY and still closed to movable if its few kernel pageblocks have no free pages left, and MOSTLY_DEDICATED and still open if they do.

§10.3 makes the premise safe: gigablocks exist only in a ZONE_NORMAL spanning at least 8 of them, so the movable workload always has most of the zone to work in.

Hysteresis comes back with readmission, and it is the price of grading the exclusion. Exclude below LOW, readmit only above HIGH. Without the gap, evacuating a pageblock invites movable straight back into it and the gigablock oscillates, paying migration cost forever, so HIGH - LOW must exceed what movable allocation can consume between two watermark checks — on the order of the PCP batch size times the CPU count. An earlier draft deleted this pair by making exclusion absolute at the taint bit. That was an over-correction: it bought a simpler watermark set by throwing away every half-empty tainted gigablock as a home for movable, which is memory the machine has and §5.2 was right to want.

The bands, all read off the two kernel-headroom numbers, HIGH > LOW > EVAC > MIN. Two mechanisms fire in them and they act on different content: evacuation migrates movable content out (§8, it never reclaims), while slab reclaim shrinks kernel content in place (§9.2). A tainted gigablock needs both, since evacuation raises headroom but only the kernel side can lower nr_kernel_pageblocks.

capability vs       band              what fires
-----------------   ----------------  --------------------------------------
(no kernel pbs)     CLEAN             nothing; kernel entering taints it
>= HIGH             TAINTED_OPEN      kernel proceeds
>= LOW              TAINTED_TIGHT     that class prefers another gigablock
>= EVAC             TAINTED_EVACUATE  async: migrate movable, shrink slab
>= MIN              TAINTED_CRITICAL  sleepable kernel does both synchronously
below MIN           ..                non-sleepable kernel only
(any)               TAINTED_FULL      overrides the ladder; retry in 10 s

Movable appears nowhere in that column: it is gated by §5.2's barrier, not by these bands. In full:

CLEAN             no kernel pageblocks.  Movable and kernel both allowed.
TAINTED_OPEN      both numbers >= HIGH.  Kernel proceeds, movable allowed.
TAINTED_TIGHT     a class below its LOW.  Kernel requests of that class prefer
                  elsewhere; movable is unaffected.
TAINTED_EVACUATE  a class below its EVAC.  Async evacuation and slab reclaim run a
                  bounded pass and stop; the next check re-evaluates.
TAINTED_CRITICAL  order 0-3 capability below MIN.  Sleepable kernel allocations
                  evacuate or shrink synchronously; only non-sleepable ones use
                  the rest.
TAINTED_FULL      an evacuation attempt isolated pages and failed to migrate them,
                  and nothing has been freed here since.  Kernel allocation looks
                  elsewhere.  An attempt that found nothing movable to isolate is a
                  different outcome and does not set this; see below.

A bounded pass replaces evacuation hysteresis. A background worker triggered at one level needs either a higher stop level or a work limit, or it stops the moment it makes progress and immediately re-triggers. kcompactd already takes the second option: do a pass, stop, re-check next cycle. That costs one watermark instead of two and needs no gap sized against an allocation rate.

TAINTED_FULL overrides the band rather than sitting below it. Once evacuation has failed there is nothing for either evacuating band to do, so the gigablock skips to spilling regardless of its counters. Otherwise TAINTED_EVACUATE retries a known-failed migration every cycle.

Failure is optimistic: two outcomes, and only one of them latches. Most reasons a migration fails are temporary — a GUP or DMA pin dropped a moment later, writeback completing, migrate_pages() returning -ENOMEM under momentary pressure — and none of them frees a page in this gigablock, so the clear condition above never fires for them. Latching on the first failure would write off 12 GB for the rest of the boot on a transient, and because the bit gates allocation as well as evacuation, the gigablock would be neither evacuated nor used.

So the attempt reports which of two things happened. Nothing movable to isolate means evacuation can never help here and only a free can change that; it does not set TAINTED_FULL. Isolated pages, migration failed is the transient case, and only it does. The attempt already walked the gigablock, so distinguishing them costs nothing.

Retry after 10 seconds, which is a starting constant rather than a tuned one. It covers writeback and transient -ENOMEM; a long-lived DMA pin outlives any period. The quantity worth watching is not the period but how often TAINTED_FULL is entered at all: near zero and the period does not matter, high and the period is the least of the problems. Count entries in the debugfs dump.

The retry is lazy. gb_evac_retry_at holds one jiffies stamp per zone, checked by the evacuation trigger, which already sleeps:

if (gb->gb_evac_failed && time_after(jiffies, gb->gb_evac_retry_at)) {
        gb->gb_evac_failed = 0;
        gb->gb_evac_retry_at = jiffies + GB_EVAC_RETRY;   /* 10 * HZ */
}

No timer and no wakeup on an idle system, and the retry happens when something is asking for memory, which is the only time the answer is worth having. Retry only while the gigablock is still blocking: gb_evac_failed set, no content-free pageblock and an empty has_free row. If something is free there is nothing to evacuate for.

The whole word clears at once, deliberately. An attempt begins with drain_all_pages(zone), an IPI to every CPU with a non-empty list, so one drain and one bounded pass should serve every retry in the cycle. Per-gigablock clocks would multiply the drains by the number of stuck gigablocks: on a 176-CPU host with eight stuck, roughly one IPI-to-all-CPUs per second, forever, to rediscover that they are still pinned.

The bottom band is safe only because spilling exists. Closing a gigablock to sleepable kernel allocations would be an OOM if this were a zone-level reserve. It is not: everyone else moves to another gigablock, and invariant 1 guarantees a spill eventually succeeds. The reserve is for callers with nowhere else to go, on the __GFP_HIGH model.

Headroom is only defined for tainted gigablocks, since a clean one has no kernel-typed pageblocks to count free pages in. A clean gigablock is never a barrier.

The exclusion is a barrier, not a filter. If movable is excluded from a gigablock and the search merely skips it, movable lands in the next tainted gigablock instead — deeper into kernel territory, so the exclusion would be causing the damage it exists to prevent. Movable stops at exclusion and reclaims instead, which needs no new mechanism: the fast path returns no page and __alloc_pages_slowpath() does what it always does. §5.2 step 5 is that stop, and step 6 is the one condition that releases it.

Movable steals a block at exactly the requested order, leaving higher orders intact.__rmqueue_steal() walks current_order ascending and page_del_and_expand() splits, so a movable order-0 steal would split an order-10 block inside a kernel pageblock and destroy that gigablock's high-order capability for a request that could have come from a movable pageblock elsewhere. A movable allocation has a claim on the pages of kernel headroom, not on its contiguity. It still gets the stranded space of §1, which is stranded precisely because it is fragmented. This also keeps the two headroom numbers nearly independent: movable consumption depletes headroom_pages without touching headroom_blocks, while evacuation raises both.

Background evacuation wakes kcompactd with a new reason rather than adding a kthread, and only for the gigablock kernel allocation is currently using.

6.3 Higher-order kernel allocations, checked against the actual GFP flags

vmalloc asks opportunistically and handles failure itself, so the design owes it nothing.vm_area_alloc_pages() starts at large_order = ilog2(nr_remaining) capped at MAX_PAGE_ORDER, so a large vmalloc asks for order 10 first. But it asks with

large_gfp = vmalloc_gfp_adjust(gfp, large_order) & ~__GFP_DIRECT_RECLAIM

vmalloc_gfp_adjust() adds __GFP_NOWARN and clears __GFP_NOFAIL for large orders, and the caller strips __GFP_DIRECT_RECLAIM. Every high-order attempt is non-sleeping, silent and opportunistic: it cannot reclaim, cannot compact, and fails immediately if the free list has nothing, after which the loop walks down to the order-0 bulk allocator. On success it calls split_page() immediately, so it never needed the block to stay high-order.

Failing those attempts is normal and already handled. The design owes vmalloc nothing beyond not making order 0 fail. The cost of the fallback is still real — PTE instead of PMD mappings for the lifetime of the mapping, for BPF JIT text, module text, VMAP_STACK stacks and large hash tables — which is what headroom_blocks exists to reduce, not to guarantee.

The high-order kernel allocations that must succeed are all order 3 or below: kernel stacks at order 2 without VMAP_STACK, dma_alloc_coherent(), kmalloc above 4 KB, ring buffers. They carry __GFP_DIRECT_RECLAIM and sit at or under PAGE_ALLOC_COSTLY_ORDER, so the existing retry, reclaim and compaction path serves them. A GFP_KERNEL order 4 to 9 allocation that must not fail is already considered a bug upstream.

Evacuation for an order >= pageblock_order kernel request is forced by the bits. Emptying a mixed pageblock cannot produce a contiguous run, because its kernel content stays. Only a pageblock whose content is exclusively movable can be emptied into a content-free pageblock, so those are the only useful targets. MAX_PAGE_ORDER is 10, so an order-10 request needs two adjacent such pageblocks.

7. Keeping the watermark honest when free memory is in the wrong place

If migration fails for want of a destination and the fallback is reclaim, reclaim must make progress even though the zone is above its watermark. The hazard is real, but the mechanism is that the watermark lies: __zone_watermark_ok() counts free pages inside tainted gigablocks that a movable allocation may not touch, so the allocator concludes all is well, kswapd is not woken, direct reclaim is skipped, and nothing progresses.

__zone_watermark_unusable_free() already exists for this and already has two instances of the pattern — it subtracts nr_free_highatomic when the caller has no rights to reserves, and NR_FREE_CMA_PAGES when the allocation cannot use CMA. This is a third:

if (!(alloc_flags & ALLOC_NOFRAG_TAINTED_OK) && movable request)
        unusable_free += zone_page_state(z, NR_FREE_KERNEL_TAINTED);

With an honest watermark every existing mechanism engages correctly and needs no special case: kswapd wakes, direct reclaim runs, compaction is considered, both retry paths see a real shortage, and the OOM path is reached only when there is genuinely nothing to do. It also makes the pressure valve automatic — when reclaim and migration both fail to help, the retry relaxes the barrier and lets movable spill into the tainted set, which is the reversible kind of damage. Reuse ALLOC_NOFRAG_TAINTED_OK for both that relaxation and the ALLOC_NOFRAGMENT retry: one flag, two uses, no new retry state machine.

Cost: one zone counter maintained where the per-gigablock free counts already are. It mirrors NR_FREE_CMA_PAGES, which is also the argument that lands it upstream.

8. Evacuation

The helper already exists. __alloc_contig_migrate_range() isolates the range and calls migrate_pages(), with a retry limit, lru_cache_disable() around the loop and putback_movable_pages() on failure. The Phase 3 helper is a refactor of that, not new code, which removes most of the risk from what looked like the largest new piece of machinery.

Movable content is migrated, never reclaimed. __alloc_contig_migrate_range() also calls reclaim_clean_pages_from_list() on what it isolated, and the Phase 3 helper drops that step:

clean file pages    migrate   a copy, but no page is destroyed
dirty file pages    migrate   copying 4 KB beats a writeback
anon                migrate
pinned              give up on this range

Two reasons, and the second is why nothing is lost.

Migration relocates a page; reclaim destroys it. Dropping a clean file page belonging to a cgroup below its memory.min or memory.low is a protection violation, and background headroom maintenance would make that routine rather than rare. Migration moves the page and leaves the charge where it is, so no protection is involved and no policy question arises.

Reclaim still happens when it is needed, through the allocator rather than through us. Migration allocates a destination page for every page it moves, and those allocations take the ordinary path: if memory is short they wake kswapd, enter direct reclaim, and obey every existing watermark and cgroup rule. Reclaiming inline would be doing badly, with none of that policy, what the destination allocations already do correctly. §7's honest watermark is what makes them see the real shortage.

The cost is a copy per clean file page, and a destination allocation that can fail where dropping the page could not. A failed evacuation is the TAINTED_FULL case of §6.2, which retries.

The kernel side of the same pass does shrink, for the same reasons. This restriction is about movable content only. A tainted gigablock needs both halves: migrating movable pages raises headroom but cannot lower nr_kernel_pageblocks, because taint is defined by kernel pageblocks, so evacuation alone can never return a gigablock to CLEAN. Only the kernel side can, which is why the two run together (§9.2).

The same argument that makes migration right for movable content makes shrinking right for kernel content: a shrinker hands objects back to the allocator rather than destroying a user's page, and the memory.min question does not arise. What it recovers is described in §9.2 — mostly pages and slab-internal space rather than whole gigablocks.

Three rules keep the kernel half from becoming the expensive one:

• Bounded like the evacuation half. A fixed request per trigger, then stop and re-check, which is §6.2's bounded-pass discipline rather than a second watermark.
• Node-local. shrink_slab() is not gigablock-aware, so pressure applied while evacuating one gigablock frees objects whose pages are anywhere. Pass the nid. The value is not "help this gigablock", it is reducing the rate at which new pageblocks are tainted zone-wide, and framed as targeted help it will look ineffective in tracing and get removed.
• Behind patch 18's tunable, defaulting conservative. §8's throttle argument applies harder here than to migration: page reclaim's cost is a refault, slab reclaim's is rebuilding metadata from disk, and shredding the dentry cache to preserve 1 GB availability is the wrong trade for most workloads.

The two halves report separately. §6.2's TAINTED_FULL is a statement about the migration half only. A pass that finds nothing movable to isolate but shrinks a useful amount of slab has not failed to evacuate — it had nothing to evacuate — so it must not arm the retry clock, and a pass whose migration failed must arm it even if the shrink went well.

Evacuation has two scales, and the plan needs both. __alloc_contig_migrate_range() works on a range and is what a 1 GB target or a whole-gigablock pass wants. One pageblock at a time is what headroom_blocks is made of, and what §5.1 step 3 reaches for when a movable-content-only pageblock stands between a kernel allocation and a claimable block.

The pageblock-scale helper is the prequel's (§13 Phase P), so the gigablock series inherits it and adds only a second caller. An isolate-and-migrate loop over one pageblock, looping because isolate_migratepages_range() stops at COMPACT_CLUSTER_MAX, queued through a fixed 64-entry preallocated work-item pool. Best effort by construction — a request is dropped when the pool is empty and a block whose pages will not move is left alone — which bounds concurrency rather than only work per pass, so the worst case holds however many gigablocks want service at once. It rechecks the pageblock's migratetype before starting, since anything queued asynchronously can be stale by the time it runs. queue_work() is not safe under zone->lock, so the queue hop is irq_work first.

Two triggers, one helper, one per series. The reactive trigger is the prequel's: a movable pageblock has just been claimed for kernel use, so migrate the movable pages left behind before they keep later kernel allocations out of it. The proactive trigger is the gigablock series': a tainted gigablock is short of headroom_blocks, so produce one (§6.2's TAINTED_EVACUATE). Background use takes MIGRATE_ASYNC; the synchronous TAINTED_CRITICAL path wants a stronger mode.

Keeping the helper's signature free of gigablock types is therefore a requirement rather than good manners: it takes a pfn and a migrate mode, so the prequel can land without anticipating the second caller.

Destinations must be clean, not merely not-the-source. Putting evacuated pages in another tainted gigablock moves the problem: that gigablock's headroom shrinks and it becomes the next target, with ping-pong the worst case. Excluding only the source is not enough, because the nearly-recoverable gigablocks are exactly the ones with the most headroom and therefore the ones a movable allocation is admitted to. So: an evacuation destination may only be a clean gigablock, or a zone with no gigablocks. If none is available, the evacuation does not happen. The justification is definitional rather than tuned — the purpose of evacuation is to reduce movable content in the tainted set, so putting evacuated pages there is counterproductive at any watermark. A gigablock-less zone is a fine destination because it cannot host a hugepage anyway, and alloc_migration_target() is already node-scoped rather than zone-scoped, so DMA32 is reachable without new plumbing.

An evacuation destination is allocated with the barrier enforced. A user allocation may relax it with ALLOC_NOFRAG_TAINTED_OK as a last resort to avoid OOM; a destination never may, because that would let evacuation spill into the set it is trying to empty. Same flag, opposite default, decided by caller.

The tool switches under pressure, which is what makes the restriction safe. Once clean space runs out migration is unavailable, and that is exactly when memory is tight — at which point migration's own destination allocations drive reclaim on the ordinary path, which is where that judgement belongs.

LRU order works against cleaning up mixed gigablocks. Pages that spilled into kernel territory during a spike are the most recently allocated, hence youngest, hence the last thing LRU-ordered reclaim frees. kswapd will not clean them; only directed evacuation will. Undirected reclaim still helps headroom_pages, since any freed page in a kernel pageblock counts. Keep the two metrics separate for this reason, and do not defer directed evacuation because kswapd is busy.

Evacuation has to drain the per-cpu lists first, or it fails on pages that are merely cached.nr_kernel_pageblocks counts content, and a pageblock whose only remaining content sits on some CPU's pcp->lists[] still counts, so a gigablock can look occupied when nothing owns its pages. Isolation and migration cannot reclaim those: the pages are neither on a free list nor on the LRU. drain_all_pages(zone) before an evacuation attempt is what turns them back into free pages. It costs an IPI to every CPU with a non-empty list, which is why this belongs on the sleepable path only, and it is a second reason TAINTED_FULL must be defined by outcome — an attempt that fails because of caching would otherwise be recorded as an unemptiable gigablock. The two outcomes §6.2 distinguishes are reported from here: nothing movable found to isolate, against pages isolated and migration failed.

Background evacuation throttles on the zone watermarks and stops where the zone itself would need to reclaim. Unthrottled it can drive a system into swap to preserve 1 GB availability, which is the wrong trade for almost every workload. Synchronous evacuation for a specific allocation may proceed past that point, because that allocation was going to stall anyway.

9. Invariants, and the tradeoffs behind the first one

1. Every steering rule has a fallthrough to the path mainline would have taken, and it is reachable without any evacuation, migration or reclaim having succeeded. Spilling into a clean gigablock is that fallthrough: bounded attempts, then take what mainline would have taken. So policy is never the reason an allocation fails. Losing a 1 GB range is recoverable; an OOM is not.
2. A pfn with no gigablock behaves exactly as mainline.
3. Kernel content is confined to the tainted set: a clean gigablock holds no kernel pageblocks at all, and the derived candidate test of §10.2 is the authority on which ranges are usable.
4. zone->free_area[order].nr_free equals the sum over gigablocks, and NR_FREE_PAGES stays consistent. CONFIG_DEBUG_VM checks both.
5. A has_free bit is set if and only if the corresponding list is non-empty. CONFIG_DEBUG_VM checks it.
6. Movable allocations enter a tainted gigablock while its kernel headroom is above LOW, or below it only under ALLOC_NOFRAG_TAINTED_OK.
7. A gigablock returns to CLEAN when its last kernel pageblock is freed. Without this the tainted set only ever grows, one transient spike costs a gigablock permanently, and the mechanism decays with uptime until it is useless.

9.1 Invariant 1 constrains the last resort, and everything before it is policy

Read as a goal — "never fail" — it would imply always spilling, which lets the tainted set grow monotonically until every gigablock is in it and the series has cost with no benefit. What it actually constrains is the last resort. Everything ahead of that is policy, and policy may be arbitrarily aggressive precisely because it cannot cause a failure.

9.2 The missing mechanism: reclaim on the kernel side

When the tainted set wants to grow, the options are evacuate, spill, or fail. The third option is to shrink kernel memory, and it is not a small omission: Phase 0 measured SReclaimable at 50 GB of 66 GB total slab on this host, so most kernel memory is shrinkable by machinery that already exists (shrink_slab() and the dentry and inode shrinkers). The kernel shrinks slab under memory pressure and not under fragmentation pressure; adding the second trigger is the missing piece.

What it recovers, from prior experience: pages, not gigablocks. Shrinking mostly returns individual pages, and a large part of what it frees is space inside slab pages that is never returned to the page allocator at all. Whole gigablocks coming back clean has not been observed. So scope §9.2 to what it actually does — feed headroom_pages (§6.1) so the tainted set can absorb more kernel allocation — and do not count it toward contraction back to CLEAN (patch 19), which needs the pages of a whole gigablock to leave, not scattered pages anywhere.

The slab-internal free space is not a loss, it is the magnet working. Space freed inside a slab page stays with the slab allocator and is handed to the next kernel allocation, which lands in a pageblock that is already tainted rather than opening a new front. That is taint-on-first-contact (§3.4) getting help from an allocator that already prefers reuse, and it is a reason to expect the tainted set to stay small — the assumption §4 now rests on.

9.3 Density as a cheap predictor, exhaustion as the hard gate

A fixed rule like "never spill while any tainted gigablock is under half used" has a failure mode: density can be low because movable content is occupying the tainted set, in which case the blocker is not density and the fix is evacuation. So density does not gate spilling. It is an excellent cheap predictor, though, and that matters because the alternative gate is expensive: proving evacuation and shrinking cannot help requires trying them, on an allocation path.

state                                   sleepable                   cannot sleep
density below ~30%                      evacuate, bounded shrink,   spill
                                        then retry
any tainted gigablock not FULL          evacuation has somewhere    spill
                                        to work: try it
all FULL and a shrink pass tried        spill                       spill
everything above failed                 spill unconditionally (invariant 1)

Only a sleepable allocation can choose evacuation over spilling, and that asymmetry is not a softening of the rule, it is the whole rule: migrate_pages() takes page locks, allocates destinations and may wait on writeback. A GFP_NOWAIT or GFP_ATOMIC caller cannot execute a single row of the left column, so for it every row reads "spill". Refusing to spill there would turn a lost gigablock into an atomic allocation failure, which is the worse outcome by a wide margin. §6.2's TAINTED_CRITICAL states the same split for the headroom watermark.

Evacuation is a slowpath retry, not a step in the search. The §5.1 order runs in __rmqueue() under zone->lock, where nothing may sleep, so "evacuate, then retry from 2" cannot happen inline. The fast path returns no page and __alloc_pages_slowpath() does the migration and re-enters — the same shape §6.2 uses for the movable barrier. Reading §5.1's numbered list as a single call chain is the easiest way to mis-implement this section.

A successful evacuation needs a bound too. TAINTED_FULL covers repeated failure, being defined as "last attempt failed and nothing freed here since". Nothing yet limits repeated success: a stream of sleepable kernel allocations can each trigger a migration pass, paying migration bandwidth to keep one gigablock's headroom just above MIN. §6.2's bounded pass is the intended defence — a triggered evacuation does a fixed amount of work and stops rather than tracking a watermark — so measure whether that bound is sufficient before adding a rate limit.

9.4 The tradeoffs, spelled out

• Latency against containment. Every mechanism tried before spilling adds latency to a kernel allocation. Evacuation is milliseconds; a dentry shrink on a large cache is worse. Strict containment means slow kernel allocations under fragmentation pressure. This is the fundamental cost of the approach and belongs in the cover letter.
• Kernel reclaim can be a bad trade. Shrinking dentry and inode caches to recover a gigablock throws away work to gain a hugepage, and whether that is worth it is workload-dependent. Bounded and tunable, defaulting conservative, with an off switch that returns mainline behaviour.
• Contraction matters more than any spill rule. If invariant 7 works the tainted set breathes and unbounded growth does not happen. If it does not, no gate saves the design — it converts growth into allocation stalls. Recovery is the primary defence; the spill ordering is second line.
• The gate can never be a refusal, or we have built an OOM source.
• It is measurement-dependent. If reclaimable slab is smeared one object per pageblock, shrinking frees pages everywhere and whole gigablocks nowhere, and §9.2 is worthless. Measure first.

10. Scope and geometry

10.1 Gigablocks tile the zone

gb_start_pfn   = round_down(zone->zone_start_pfn, PUD_PAGES)
gb_size_pages  = round_up(zone_span / 64, PUD_PAGES)
nr_gigablocks  = DIV_ROUND_UP(zone_end_pfn - gb_start_pfn, gb_size_pages)

Every gigablock exists and owns free lists, including incomplete ones at the zone edges, because their pages need somewhere to live. Some gigablocks contain PUD ranges that can never be hugepage candidates. Those are two different facts and must not be conflated.

Gigablock boundaries are PUD-aligned, which the formulas guarantee. Without that a PUD range could straddle two gigablocks and "which gigablock owns this candidate range" would be ambiguous.

10.1a Boot ordering, taken from the existing implementation

Not a new problem. The v27/v28 superblock series solved it and v1 copies the shape:

Build the arrays in free_area_init_core(), after setup_usemap() and before init_currently_empty_zone(). That is before the zone's free lists exist, so it is before any __free_one_page() can run, and no maintenance site needs a "not built yet" guard.

Allocate with memblock_alloc_node(), not kvmalloc(). The page allocator does not exist yet, which is the whole reason the ordering works. kvmalloc and the RCU teardown of §12a belong to resize_zone_gigablocks(), and v1 has no resize: memory hot-add is out of scope (§10.1) and Phase 1 item 1 says boot-present memory only.

A failed allocation leaves zone->gb NULL, which is already the "this zone has none" state of §3.1, so warn and continue. The zone behaves exactly like one that legitimately has no gigablocks, and every call site no-ops through the same path rather than a second one.

Deferred struct page init frees large ranges where the maintenance is pure overhead. The arrays exist by then, so this is a cost question rather than a correctness one, and §13's Phase 1 patch 3 is where the init sweep decides what to do about it.

10.2 The candidate test for a PUD range

1. the whole range lies within this zone
   excludes the partial ranges at the zone edges and ranges straddling a zone
   boundary.  Structural and cheap.
2. no pageblock in the range is MIGRATE_CMA
   structural, and not expressible in the content bits: CMA content is movable,
   but a CMA pageblock never changes migratetype, so the range can never be
   handed to hugetlb.
3. no pageblock in the range has PB_has_unmovable or PB_has_reclaimable
   the content test.  Memory holes land here rather than in a case of their own,
   because init_unavailable_range() marks hole pages PageReserved and §3.3 makes
   PageReserved taint.

Derived on demand by the 1 GB allocator and by the debugfs metric: 512 bytes of reads per candidate range, and 48 KB for the whole zone.

10.2a The doomed-but-usable ranges at the zone edges

Neither zone_start_pfn nor zone_end_pfn is PUD-aligned in general, so the first gigablock's lowest range extends below the zone and the last gigablock's highest range extends above it. Both fail condition 1, both can never host a 1 GB page, and both hold real usable memory — up to PUD_PAGES - 1 pages each, just under 2 GB per zone. A range straddling a zone boundary fails the test in both zones, so each zone may fill its half with kernel content at no cost to either.

PUD grid   |....1G....|....1G....|....1G....|....1G....|....1G....|
zone            [====================================]
             ^^^                                  ^^^^
             doomed: range straddles the zone start / end, fails
             condition 1 in both zones, yet holds usable memory

So they are a two-entry list, fixed at init, and §5.1 consumes them first. Kernel content there costs nothing and spares a good range.

The effect scales badly downward. On this host Normal starts at pfn 1048576, which is PUD-aligned, so only the end range is doomed: 1 of 766. On a 32 GB machine round_up(32 GB / 64, PUD_SIZE) is 1 GB, so gigablocks are PUD ranges, ZONE_NORMAL from 4 to 32 GB gives 28 of them, and both ends give one each: 2 of 28, against a fixed cost of one more range for the vmemmap. 28 ranges minus one vmemmap, minus two edges, minus three or four for kernel content leaves roughly 22.

10.3 What gets no gigablocks

Whole zones only. ZONE_NORMAL is the only zone type that gets them, and only when it spans at least 8 gigablocks:

• ZONE_DMA and ZONE_DMA32 are device-constrained rather than general memory. A naive round_up(span / 64, PUD_SIZE) would give DMA32 four 1 GB gigablocks on this host, and steering inside a 4.3 GB zone that 32-bit devices depend on risks starving them to chase a hugepage nobody wants from there. Excluding by zone type is simpler than tuning a size threshold to exclude them.
• Below about 8 gigablocks there is no room for a tainted set and a clean set to coexist, so the machinery adds only cost.

ZONE_MOVABLE gets none because every page there is migratable by construction — that is why hot-remove uses it — so any 1 GB range is freeable on demand through alloc_contig_range() and steering protects nothing. ZONE_DEVICE gets none. Hot-added memory gets none in v1.

Every one of those is the same zone->gb == NULL fallback, so all are free to implement, and together they delete the gigablock array resize entirely. v27 needs three different reader-safety contracts around resize_zone_gigablocks() — rcu_read_lock() plus synchronize_rcu() for non-sleeping readers, get_online_mems() for page_reporting_cycle() which drops zone->lock to call into a driver, and nothing at all for readers holding zone->lock throughout — purely because the old array is kvfree()d. No resize, no contract to get wrong, and page_reporting stops needing to know what a gigablock is.

10.4 Known bounds, stated rather than fought

• Order-0 allocations mostly never reach the buddy allocator; PCP absorbs them, and PCP frees and reallocations do not consult gigablock policy. This bounds achievable purity at low order.
• A single long-lived unmovable page costs one 1 GB range until it is freed. That is the point of confinement.
• Long-term GUP pins make a movable page unmigratable with nothing to distinguish it at allocation time (§14.1).
• The vmemmap is 12.8 GB per node of permanently pinned memory, and on this host it sits at the top of the zone because memblock.bottom_up is false by default and early allocations therefore cluster at the top of available memory. movable_node flips that.

11. Salvage from v27

Reuse close to verbatim:

• the PB_has_* bits and their maintenance sites
• pfn_gb_free_area() — this is Phase 2's pfn_free_area(), already written and proven behaviour-identical for add, del and move
• the zone->free_area[].nr_free exact shadow
• contig_allocated / gigablock_contig_mark() for the CMA and contig-range cases, and as the metric's record of a successful 1 GB allocation
• the fast_isolate_freepages() cross-gigablock freelist fix-up, as the worked example in Phase 2's changelog
• evacuate_pageblock() and its async plumbing, which is the prequel's P3 and P4 rather than part of the gigablock series. 224 lines already written and debugged: the isolate-and-migrate loop over one pageblock, the fixed 64-entry work-item pool, the staleness recheck, and the irq_work hop that makes queue_work() reachable from under zone->lock. Drop its reclaim_clean_pages_from_list() call per §8

Delete: gb_lists[category][fullness], the PASS_1/2/2B/2C/2D category search machinery, gb_hint and gb_warm_hints, GB_AGGRESSIVE_THRESHOLD, the per-gigablock free_area fullness ranking, and the resize path.

The series applies to 848acc8ffe1b, not to the v27 branch. Say so in the cover letter with this list, so "small diff" is not read against v27's 4666 lines.

12. Open questions

Items 1, 2, 5, 7 and 8 are settled and kept so they are not re-raised. Live going into Phase 1: 3 and 9 need v1's data, 6 needs a benchmark that does not exist on either side, and 4 is deferred to a follow-up series.

1. Deferred out of v1, not open: within-gigablock packing (§4a). Build it if the tainted count passes ~48 of 64 or clean-only candidate ranges approach 94. The quantities to measure then are candidate ranges per tainted gigablock — 9 of 12 is the ceiling, 0 is what LIFO gives — and the bounded-walk fallback rate.
2. Answered: kernel-side shrinking recovers pages, not gigablocks. Prior experience is that shrinking mostly returns individual pages, and much of what it frees is space inside slab pages that is never returned to the allocator at all. Whole gigablocks coming back clean has not been observed. §9.2 is therefore a headroom mechanism, not a contraction one; see there.
3. The PUD fast path's lock hold, unmeasured: removing every free block of a content-free 1 GB range from the lists under zone->lock. pageblock_order is 9 and MAX_PAGE_ORDER is 10, so a fully coalesced free 1 GB range is 256 order-10 blocks, not 512 order-9, and the per-pageblock bit updates are 512.
4. Compaction scope, split in two. The correctness half is not optional and is not deferred: isolate_freepages() scans pfns and survives, but __isolate_free_page() must route through the gigablock's free_area, which Phase 2 does. The policy half is deferred to a follow-up series: compaction migrating pages into a clean gigablock does not taint it, since taint is nr_kernel_pageblocks > 0 and movable content is evacuable, so making destination selection gigablock-aware is an optimisation rather than a correctness fix, and which direction it should prefer is exactly what v1's data will say.
5. Answered: evacuation migrates and never reclaims (§8), so no page is destroyed and no cgroup protection is involved. Where reclaim is genuinely needed it is driven by migration's own destination allocations, on the ordinary allocator path, which respects memory.min and memory.low like any other allocation.
6. No hugepage-success benchmark exists on either side.

7. Answered: solved in the v27/v28 implementation, and copied from it. See §10.1a.

8. Answered: gb_above is [2][4]. Two request classes (§6.1), four levels HIGH, LOW, EVAC, MIN. The order 4 and above class leaves MIN unused, since it has no floor below spilling; 64 bytes with one word wasted is cheaper than two arrays of different length.

9. The two fullness boundary values, which need v1's data. The thresholds are two independent constants, not one shared knob: for tainted gigablocks the numerator is nr_kernel_pageblocks/total and grades concentration for the magnet of §3.4, for clean ones it is (total - nr_free)/total and grades how little is left to protect when choosing what to spoil. Different questions, so no reason the values should match. ~25% each is the starting point; the distribution of both ratios across the tainted and clean sets is what says whether either is right.

Two things need no change and should be said up front, because both are obvious reviewer objections: __zone_watermark_ok() reads the exact nr_free shadow so watermark checking is untouched, and fill_contig_page_info() with its users fragmentation_index(), extfrag_for_order() and unusable_free_index() read only nr_free.

12a. Where the code lands

v28 is the warning: it added 5034 lines to mm/page_alloc.c and created no new file, which is why its steering, accounting and search policy could not be reviewed or tested apart from the buddy allocator. The split below exists to keep that from happening again, and the target is that mm/gigablock.c answers which gigablock while page_alloc.c keeps doing buddy mechanics and gains only call sites.

mm/gigablock.c (new, the bulk of the series)

gigablock_alloc_search()      §5.1, §5.2 step order for both populations
gigablock_eligible_mask()     §4.1 mask composition
gigablock_rank_fullness()     §4.1 ranking among survivors
gb_get_fullness()             §3.4 bucket, and the mask move on a crossing
gigablock_account_pageblock() §3.3 content bits, nr_kernel_pageblocks, taint
gigablock_watermark_update()  §6.1, §6.2 band transitions and gb_above[]
gigablock_evacuate()          §8, the sleepable path and TAINTED_FULL
resize_zone_gigablocks()      §10.1 sizing, kvmalloc, RCU teardown -- not in v1, see §10.1a
gigablock_debugfs()           §15 introspection

mm/internal.h — struct gigablock_data, struct gigablock, enum gb_fullness, enum gb_watermark, zone_has_gigablocks(), and the !CONFIG_GIGABLOCK stubs that make every call site above compile to nothing. Nothing here is exported outside mm/.

include/linux/mmzone.h — struct zone gains one pointer, zone->gb, placed inside the existing _pad1_ region (§3.1). That is the only change to a public header.

mm/page_alloc.c — the seven lines that decide which list, plus call sites

zone->free_area appears exactly seven times in page_alloc.c on the current base, and three of those are inside the free-list accessors:

__add_to_free_list              area = &zone->free_area[order]
move_to_free_list               area = &zone->free_area[order]
__del_page_from_free_list       zone->free_area[order].nr_free--
__rmqueue_smallest              area = &zone->free_area[current_order]
__rmqueue_claim                 area = &zone->free_area[current_order]
__rmqueue_steal                 area = &zone->free_area[current_order]
unreserve_highatomic_pageblock  area = &zone->free_area[order]

Replacing each with gb_free_area(zone, page, order) is the whole per-gigablock-free-list change, and because the accessors are already the chokepoint, their 24 call sites need no edit at all. The remaining page_alloc.c work is call sites rather than logic: __rmqueue() consults gigablock_alloc_search() before falling back to today's behaviour, should_try_claim_block() gains a gigablock term, __free_one_page() and the highatomic reserve paths call the accounting helper, and __zone_watermark_ok() learns the headroom subtraction of §7.

Files that gain a call, not a policy — mm/mm_init.c (build the arrays after the zone is sized), mm/compaction.c (target selection asks gigablock.c which gigablock, §8), mm/page_reporting.c (§10.1's zone->lock drop), mm/memory_hotplug.c (resize), mm/vmstat.c, mm/show_mem.c, mm/debug.c (counters and dumps).

What the split buys, and the one place it may not hold. Buddy mechanics stay testable without gigablocks, the search policy becomes unit-testable through one entry point, and !CONFIG_GIGABLOCK compiles the whole thing out through internal.h stubs. The risk is gb_free_area() on the fast path: a function call where there used to be an array index, in a file that is not page_alloc.c, is where LTO and inlining stop being an abstract concern. If it does not inline, that helper is the one thing that moves back into internal.h as a static inline — and it is small enough to. Measure before assuming either way.

13. Patch series

Phase P — the prequel: pageblock evacuation

Ships as its own series, before and independent of everything below. It needs no gigablock, no per-zone structure and no policy, and it is justified entirely by what happens today when try_to_claim_block() steals a movable pageblock and cannot move everything out: the leftovers keep the stealing type from filling the block, so the next allocation of that type steals another one.

P1. Extract count_alike_pages() from try_to_claim_block(), and the whole-block path with it (§16.3a). Prep for P5, which changes the same function. P2. The three PB_has_* content bits and their maintenance at the migratetype-change, whole-pageblock free and allocate sites, plus the init sweep over for_each_reserved_mem_range() and the PageOffline and hwpoison hooks (§3.3). Three spare bits, no array growth. P3. Deferred-work infrastructure: the fixed 64-entry preallocated work-item pool and the irq_work-to-workqueue hop, because queue_work() is not safe under zone->lock. P4. evacuate_pageblock(): isolate and migrate over one pageblock, looping because isolate_migratepages_range() stops at COMPACT_CLUSTER_MAX, with a staleness recheck of the migratetype before starting. Migrate only, never reclaim (§8). P5. Queue an evacuation when a movable pageblock is claimed for kernel use, skipping blocks whose PB_has_movable is clear. This is what makes P2's bits load-bearing in the series that adds them. P6. Counters: steals, evacuations attempted, succeeded, and dropped for want of a pool entry. Without these the series cannot be argued for or against.

How it sits with defrag_mode. Johannes Weiner's defrag_mode sets ALLOC_NOFRAGMENT on every allocation, reclaims at max(order, pageblock_order), and drops ALLOC_NOFRAGMENT only after reclaim and compaction have both failed. Its strategy is to prevent the mixing it can; the prequel recovers the mixing it cannot prevent, and the two meet exactly at that last-resort retry, which is the moment a steal is permitted. They compose rather than compete:

• kswapd cannot do this job even at pageblock_order, because the leftovers in a freshly stolen block are the youngest pages and LRU-ordered reclaim reaches them last. Only a directed pass targets them.
• Under defrag_mode the evacuation's own destination allocations inherit ALLOC_NOFRAGMENT, so they cannot fall back and pollute a second pageblock. The cost is that they fail more often, which best-effort already tolerates. Measure with defrag_mode both on and off.
• The work is sequential, not duplicated: compaction failed to produce a whole block, one was stolen, and the prequel makes the stolen one usable.

The objection to pre-empt is "make defrag_mode stricter instead". It cannot be: an unmovable allocation cannot be refused forever, so the forced path is unavoidable, and today its leftovers are pure loss.

Phase -1 — tracing for the pageblock conversion paths that emit nothing

Half of all pageblock conversions are invisible (§1.2), which blocks validating a series whose premise is controlling which pageblocks change type. Prerequisite instrumentation, independently useful upstream, and it has a hard pass/fail: rerun the VM experiment and the 624-versus-318 gap should close.

set_pageblock_migratetype() and set_pfnblock_flags_mask() are inlined, so a kprobe is not an option and the tracepoint has to go in the source. Check whether every conversion funnels through set_pageblock_migratetype() or whether change_pageblock_range() and the highatomic paths need their own.

Phase 0 — measurement

Done, in ~/debug/gigablock-phase0/. Outstanding refinement: a page_has_movable_ops() check plus slab-cache attribution in the drgn pass, to turn the 12.3% upper bound into a real number, and open question 2. Also worth having generally: for the caches that pollute, track how many pageblocks a slab cache touches rather than how many bytes it holds. A 147 MB cache spread over 36k pages can spoil more pageblocks than a 2 GB densely packed one.

Where the cleanup patches sit

Not in a block of their own. Each cleanup lands immediately before the patch that changes the same function, lettered Na so the numbering of the functional patches stays put. A reviewer reading patch Na sees a function split along theme boundaries with no behaviour change, and patch N then changes one named helper instead of reaching into a 166-line body. Reviewing the pair together is also the only way to check that the split actually helped, which a cleanup block at the front of the series makes impossible.

Each is still justifiable without mentioning the series (§16.1) — adjacency is about review order, not about weakening that test. Detail in §16.3.

Phase 1 — inert infrastructure, nothing reads it

1. struct gigablock_data and the per-zone pointer: round_up(zone_span / 64, PUD_PAGES), at most 64, PUD-aligned, boot-present memory only, no resize, ZONE_NORMAL only and only at 8 gigablocks or more.
2. pfn_to_gigablock(), zone_has_gigablocks(), accessors. NULL and no-op when absent. 3a. Extract the done_merging tail of __free_one_page() into its own helper (§16.3a). The merge loop stays as it is; buddy_pfn escapes it.
3. (In the prequel as P2, listed here because the gigablock series depends on it.) The three PB_has_* bits plus maintenance at the migratetype-change and whole-pageblock free and allocate sites, plus the init sweep over for_each_reserved_mem_range() and the PageOffline and hwpoison hooks.
4. Per-gigablock nr_free and nr_kernel_pageblocks, and the gb_tainted threshold.
5. has_free[mt][order], maintained from list_empty() and the pageblock bits. Written, not yet read.
6. The two-entry doomed-but-usable range list, computed at init.
7. Free-list iterator helper for the four external readers — page_reporting, hibernate mark_free_pages(), show_mem, /proc/pagetypeinfo — still iterating zone->free_area. Pure refactor, reviewable by those maintainers without knowing what a gigablock is.
8. debugfs state dump. Lands here rather than in Phase 4 because Phases 2 and 3 cannot be developed or debugged without seeing the tainted set, the candidate count and the per-gigablock state. Also carries the counters the open questions of §12 are waiting on: the distribution of both fullness ratios, the TAINTED_FULL entry count, and candidate ranges from the clean set alone.

Phase 2 — move the lists, in two steps

A page has one buddy_list node, so the lists cannot exist in both places at once and the move cannot be split by caller. It can be split by indirection.

9a. Rename free_area_empty() to free_list_empty(). It reads as a question about the whole free_area but tests one migratetype's list, and patch 9 makes the distinction load-bearing. 9b. Split fast_isolate_freepages(), 166 lines, along the six themes of §16.3: fast_isolate_setup(), fast_isolate_scan_order() for the b+c+d group that cannot be separated, fast_isolate_fallback(). 9c. Extract compact_suitable_page_free() from __compact_finished(), the one theme of four this series replaces. Both functions touch the free lists directly — three references and two respectively — so they are modified by patch 9 as well as by patch 16, and the cleanup belongs before the first of the two. 9. Convert every free-list search and update site to for_each_free_area(zone, order, mt, area) and pfn_free_area(zone, pfn, order), whose first implementations yield exactly &zone->free_area[order]. Mechanical, bit-identical, no gigablock knowledge. The changelog names the sites that need pfn_free_area() to re-resolve a specific found page's owner rather than merely iterate — chiefly fast_isolate_freepages(), whose move_freelist_head() reorder must act on the list the candidate actually lives on, not the last one iterated. 10. Gigablocks own free_area[]: pfn_free_area() returns the gigablock's, for_each_free_area() iterates through has_free, and one accessor does the list operation, both counters and the bit flip. Small diff, all of the risk, trivially bisectable. This is where behaviour changes, and the change is "allocate from whichever gigablock the bitmap finds". No taint preference exists yet. 11. CONFIG_DEBUG_VM cross-checks for invariants 4 and 5.

Phase 3 — policy and evacuation

12a. Split __zone_watermark_ok() into watermark_min_for_flags() and zone_has_suitable_free_page() (§16.3a). 12. NR_FREE_KERNEL_TAINTED, the __zone_watermark_unusable_free() subtraction, and ALLOC_NOFRAG_TAINTED_OK. First, so the watermark is honest from the first patch that can create a shortage. 13a. Lift the CMA-balance preamble out of __rmqueue() as rmqueue_cma_balanced() (§16.3a). The switch (*mode) escalation ladder stays whole. 13. Kernel candidate order (§5.1), including the doomed-range step and the content-free-pageblock preference. 14a. (In the prequel as P1.) Extract count_alike_pages() from try_to_claim_block(), and the whole-block path with it (§16.3a). 14. Movable candidate order (§5.2), including steal-at-requested-order-only, and the should_try_claim_block() rebase of §5.8. 15a. Give __alloc_contig_migrate_range() a range and a target policy instead of a compact_control it builds itself. No behaviour change; it has one caller today only because it is not callable. 15. Build the range-scale directed evacuation helper on it, for whole gigablocks and 1 GB targets. The pageblock-scale helper already exists by now — it is the prequel's P4 — so this patch adds the coarse scale only, and patch 17 adds the proactive trigger that asks for a pageblock when a tainted gigablock is short of headroom_blocks. 16. compact_suitable_page_free() becomes a one-line substitution. Teaching compaction destination selection about gigablocks is deferred to a follow-up series (§12 item 4): it is an optimisation, not a correctness fix, since migrating movable pages into a clean gigablock does not taint it. 17. Headroom watermarks and states: synchronous evacuation for sleepable allocations, kcompactd wakeup for the background path, throttled on the zone watermarks. Includes TAINTED_FULL's two outcomes and its 10 s lazy retry (§6.2). The retry ships with the bit rather than waiting for patch 19, so the bit is never load-bearing without an exit. 18. Kernel-side shrinking (§9.2) and the spill ordering of §9.3, behind a tunable defaulting conservative. 19. Contraction back to CLEAN, including the background re-derive of §3.3. 20a. Extract the conversion and accounting from unreserve_highatomic_pageblock(), 71 lines: three nested loops with an early return in the middle, victim selection tangled with conversion. 20. Highatomic reserves from an already-tainted gigablock.

Phase 4 — exploit

1. PUD-order fast path: satisfy a 1 GB request from a content-free PUD range with no migration. Measure the lock hold.
2. Tracepoints and the debugfs metric: candidate ranges, tainted-set size, mixed-gigablock count, and taint transitions. Three numbers, not one — candidates is what the series delivers, tainted-set size is what it costs, and the mixed count is outstanding evacuation debt.
3. Documentation.
4. Tests (§15), as their own member of the series.

Phases 1 and 2 are the arguable part; a reviewer who accepts patch 10 has accepted the design. Phase 3 is where the behaviour lives and Phase 4 is optional, so both can land incrementally or be dropped without leaving the tree worse than before patch 1.

14. Future work

14.1 Long-term GUP pins, and the lifetime principle behind them

A FOLL_LONGTERM pin makes a movable page unmigratable for the pin's lifetime, so a pinned page in an otherwise clean gigablock quietly makes that PUD range non-evacuable, with nothing at allocation time to distinguish it. RDMA, vfio, io_uring fixed buffers and dma-buf are the volume cases.

Segregate pinned content, and the reason is lifetime rather than movability. A pin ends when the workload restarts and its memory is freed, so whole gigablocks become available again. A random kernel allocation mixed into the same gigablock may live until reboot, and mixing the two destroys the recoverability of the shorter-lived one.

That generalises: the axis worth segregating on is expected lifetime, and movability is only a proxy for it that breaks exactly here. In bitmap terms this is one set per lifetime class rather than bands in an address ordering, which is also why the two-boundary problem an earlier positional version of this had disappears.

The mechanism to extend already exists. folio_is_longterm_pinnable() already returns false for MIGRATE_CMA and MIGRATE_ISOLATE, and check_and_migrate_movable_folios() migrates such pages before allowing the pin, because unmigratability is unacceptable there. Clean gigablocks are the same kind of region.

Pin granularity decides the cost, and the two interesting sizes behave oppositely. A 1 GB hugetlb page pinned long-term costs nothing to place: it already owns its PUD range exclusively, so the pin adds no constraint the allocation did not already impose. Nothing to migrate, no set to join, just an accounting mark, and it frees wholesale on exit. That makes RDMA workloads that consume 1 GB pages the best case for this series rather than the hard case, which is worth saying because the reviewer instinct is the opposite.

2 MB pins are the hard case: big enough to matter, small enough to scatter. One pinned THP makes one pageblock unmigratable and a PUD range is 512 pageblocks, so a single pinned THP costs a whole candidate. 100 GB pinned as 1 GB pages consumes exactly 100 ranges; the same bytes pinned as THPs is 51200 unmigratable islands that can spoil many times that number. So the pinned set must be filled at PUD granularity: complete a partially filled pinned range before starting a new one.

Measure first. /proc/vmstat has nr_foll_pin_acquired and nr_foll_pin_released, equal at 614962909 on this host, so nothing is pinned at the moment and these workloads may not need any of this.

14.2 Deferred elsewhere

• Per-section gigablock lifetime, if 1 GB pages on hot-added memory ever matter. A PUD range is 8 sections, so hanging the struct off the memory section ties lifetime to section online state with still no array to reallocate.
• Per-PUD-range taint bits, if the §3.4 threshold proves too coarse — specifically if measurement shows tainted gigablocks losing clean ranges faster than the design needs them.
• Evacuation cost-awareness: a per-gigablock skip bit and last-attempt record, dropped from v1 because correctness comes first. Add if futile attempts turn out to be frequent.
• pbsteal, making movable steal before it claims, is a dead end unless mTHP adoption makes orders 4 through 8 a large share of movable allocation. Two runs found mechanism A firing zero times, and the reason is structural: should_try_claim_block() refuses movable claims below pageblock_order / 2, and a claim at or above pageblock_order takes an already-free pageblock and leaves nothing behind. Branch scratch/riel/pbsteal is kept as the record.

15. Tests

Guidance: tests ship as their own member of the series (5.Posting.rst via ~/kernel-style/patch-series.md §3); every patch builds and runs on its own; KUnit tests for mm live in mm/tests/ with a mm/Kconfig entry following lazy_mmu_mode_kunit.c; behavioural tests live in tools/testing/selftests/mm/ using kselftest.h, wired into the Makefile, run_vmtests.sh and the config fragment.

15.1 The decision that makes allocator tests non-flaky

Allocator tests earn their reputation by trying to induce a state through a workload and then asserting on it. Fragmentation is stochastic, so those tests get marked flaky and then disabled. Do it the other way round: expose each operation under test as a debugfs trigger, drive it directly, and assert on the reported state. "Evacuate gigablock N and tell me what happened" is deterministic; "allocate until a gigablock happens to need evacuating" is not.

Assert directional and threshold properties with generous margins, never exact numbers, and always print the measured values so a failure is diagnosable rather than just red.

15.2 Every invariant gets a test

I1  no allocation fails due to policy
    debugfs fault injection: force every evacuation attempt to fail, then run the
    existing mm selftests and a memory hog.  Nothing may OOM or fail that would
    have succeeded on a base kernel.  Highest-value test in the set, because I1 is
    the invariant whose violation is unrecoverable.
I2  a pfn with no gigablock behaves as mainline
    boot a VM too small to host a gigablock and run the full mm selftest set;
    results must match the base kernel exactly.  Plus a KUnit case that
    pfn_to_gigablock() returns NULL for excluded zones.
I3  kernel content stays in the tainted set
    slab-heavy workload, then read debugfs: candidate ranges outside the tainted
    set must be within a stated tolerance of the pre-workload count.
I4  counter consistency
I5  bitmap-versus-list consistency
    CONFIG_DEBUG_VM checks are the assertions; the test's job is to exercise the
    allocator hard while they are on.
I6  movable never enters a gigablock below LOW without ALLOC_NOFRAG_TAINTED_OK
    a stat counter for barrier stops plus a tracepoint.  Under movable pressure
    with a saturated tainted set, barrier stops must be non-zero and movable
    allocations below LOW without the flag must be zero.
I7  the tainted set contracts
    build a large dentry tree to grow slab, record the tainted-set size, drop
    caches, assert it returns to within a tolerance of its starting value.
    Deterministic, and it catches the failure mode where the mechanism decays with
    uptime.

15.3 KUnit: the parts that are pure logic

mm/tests/gigablock_kunit.c, CONFIG_MM_GIGABLOCK_KUNIT_TEST. Runs anywhere, in CI, with no memory requirements:

• sizing across zone spans from 1 GB to 64 TB, including the degenerate single-gigablock case, zones too small to host one, and the PUD-alignment of gb_start_pfn and gb_size_pages
• the candidate test: partial ranges at both edges, ranges spanning two zones, ranges containing CMA, ranges dominated by PageReserved
• the has_free search in both index directions, including empty, full and single-bit bitmaps
• the taint threshold and the watermark state derivation, including every boundary and the hysteresis gap
• the content-bit clearing rule: a pageblock that loses all content clears all three; one that keeps a single kernel page does not

15.4 Selftests

Each must ksft_test_result_skip() rather than fail when its preconditions are absent — not enough memory, no debugfs, no hugetlb, no gigablocks on this zone. run_vmtests.sh gains a size gate.

• gigablock_taint.c — I3 and I7, workload plus debugfs read
• gigablock_pressure.c — I1, I6 and the barrier, under fault injection
• gigablock_1g.c — allocate 1 GB hugetlb pages before and after a fragmenting workload and assert the count does not fall. Model on compaction_test.c, which measures, fragments, measures and compares against a fraction of RAM rather than an absolute count.
• gigablock_thp.c — THP availability must not regress, reported per eighth of the zone, since an improvement confined to the already-clean part of the zone would hide a regression elsewhere.

A test module covers what userspace cannot produce: GFP_ATOMIC bursts from non-sleepable context, and GFP_KERNEL order-0 storms faster than evacuation can keep up. Same Kconfig as the KUnit tests, driven by debugfs, in the pattern of lib/test_*.

15.5 Tests that must exist beyond the invariants

• Both gigablock size extremes. A 32 GB machine gets 1 GB gigablocks and a 768 GB machine gets 12 GB; the edge effect differs by a factor of 27 and both ends want a test.
• Fork storm and netdev surge. Many GFP_KERNEL order-0 allocations arriving faster than evacuation can free space, to size the watermarks and confirm the latency bound.
• Movable pressure with a full zone. Free memory present only inside the tainted set, to confirm invariant 1 and §7's watermark accounting rather than an OOM.
• Memory offline while gigablocks exist, and a hot-add run confirming the added memory gets none.

15.5a Proving the fast path did not regress

The plan predicts three times that the cost is negligible — §3.1's "a pointer perturbs nothing", §4's "the index is a reduction rather than a cost to apologise for", §4.1's masks on the allocation path — and never says how any of it gets checked. On an allocator series that is the first thing a maintainer asks for, and each of those is a claim to falsify rather than a conclusion to state.

Three configurations, because they separate three different costs.

CONFIG_GIGABLOCK=n                  the code does not exist: the baseline
=y, zone_has_gigablocks() false     a zone under 8 gigablocks (§10.3), so every
                                    hook is compiled in and every one returns
                                    early: measures the cost of the branches alone
=y, gigablocks active               measures the policy

The middle configuration is the one that has to come out free, and it is the one that will run on the majority of machines. If it does not, the problem is a hook placement rather than the policy, which is a different and much cheaper fix — so measure it before the third.

What to run. Existing tools first: will-it-scale's page_fault1 and page_fault3 for concurrent allocation, hackbench and kernbench for mixed kernel allocation under load. Then a targeted microbenchmark, which does not exist yet and has to be written: a debugfs trigger doing a tight alloc_pages()/__free_pages() loop at orders 0, 3 and 9 for each migratetype, reported in cycles per operation, so a change inside __rmqueue() shows up without a workload's noise on top of it.

Verify the inlining rather than assuming it. §12a's one real risk is gb_free_area() failing to inline across the page_alloc.c/gigablock.c boundary. That is checkable directly and cheaply: objdump -d the three free-list accessors and look for a call. Do it in the patch that introduces the helper, not after the series is written, because the fix is to move the helper into internal.h and that changes where later patches put things.

State the result either way in the cover letter. A series that reorganises the allocator's hottest structure and reports no numbers invites the reviewer to assume the worst, and a small honest cost with a named cause is far easier to accept than silence.

15.6 Will upstream accept knobs that exist for tests

Yes, and nothing needs inventing — four established patterns cover everything above.

• A debugfs trigger whose consumer is a selftest. <debugfs>/split_huge_pages in mm/huge_memory.c is write-only, CONFIG_DEBUG_FS-gated, and its documented consumer is tools/testing/selftests/mm/split_huge_page_test.c. /proc/sys/vm/compact_memory is the same idea in the most analogous subsystem.
• Fault injection. Do not invent one: mm/fail_page_alloc.c with CONFIG_FAIL_PAGE_ALLOC under FAULT_INJECTION, setup_fault_attr(), a boot parameter and debugfs attributes.
• A test-only device behind its own CONFIG. CONFIG_GUP_TEST provides /sys/kernel/debug/gup_test and its Kconfig help text points straight at the selftest.
• Fragmentation observability in debugfs. mm/vmstat.c already exports the extfrag and unusable indices under CONFIG_DEBUG_FS via DEFINE_SEQ_ATTRIBUTE(extfrag). A gigablock state dump sits beside them.

The distinction that decides acceptance is whether a knob is only scaffolding. Candidate-range count, tainted-set size, per-gigablock headroom and the doomed-range list are genuine diagnostics — the first thing anyone debugging a failed 1 GB allocation wants — so they belong with the extfrag files on that basis. The evacuation trigger and the failure injection are scaffolding, so they go behind default-off CONFIGs in the GUP_TEST and FAIL_PAGE_ALLOC pattern.

15.7 What does not ship

The Phase 0 A/B measurement harness needs a 64 GB VM, a fixed-work workload and three runs per configuration. That is a measurement, not a pass or fail: it stays in ~/debug/gigablock-phase0/ and its numbers go in the cover letter. Boot-matrix cases belong in CI rather than run_vmtests.sh.

16. Tech debt in the code the series touches

16.1 The test a prep patch has to pass

A preparatory cleanup must be justifiable without mentioning this series. Maintainers reject "refactor so my work fits" and accept "this function is 166 lines and does three things". If a cleanup can only be motivated by what comes after it, it belongs inside the patch that needs it.

Placement is adjacent, not up front. Each of these lands as patch Na immediately before the functional patch N that changes the same function (§13). That keeps the pair reviewable together, which is the only way to see whether the split earned its place, and it means a cleanup dropped in review takes one functional patch with it rather than stranding the whole series behind a refactor block.

The related rule is that a series owns the functions it grows (~/kernel-style/patch-series.md §1), and scripts/series-function-growth.py reports the offenders. Its function_lengths() handles multi-line declarations correctly; do not write a second parser, because a naive regex silently misses exactly the functions this is for.

16.2 Lengths of the functions the series touches

Measured on 848acc8ffe1b:

166  mm/compaction.c        fast_isolate_freepages
116  mm/compaction.c        isolate_freepages_block
113  mm/compaction.c        isolate_freepages
109  mm/compaction.c        __compact_finished
109  mm/page_reporting.c    page_reporting_cycle
 87  mm/page_alloc.c        __free_one_page
 77  mm/page_alloc.c        __zone_watermark_ok
 71  mm/page_alloc.c        unreserve_highatomic_pageblock
 65  mm/page_alloc.c        try_to_claim_block
 63  mm/page_alloc.c        __rmqueue
 55  mm/page_alloc.c        __alloc_contig_migrate_range
 49  mm/page_alloc.c        __rmqueue_claim
 45  mm/page_alloc.c        prep_move_freepages_block
 43  kernel/power/snapshot.c mark_free_pages

16.3 The prep patches, split by theme

1. fast_isolate_freepages(), 166 lines. Six themes, and the interesting part is which must not be separated:

a  decide where and how hard to search   the cc->order <= 0 bail, the scan_start
                                         choice, distance/low_pfn/min_pfn, the
                                         search_order clamp
b  find the best candidate in one order  the reverse list walk, tracking highest,
                                         accepting pfn >= low_pfn, remembering the
                                         best >= min_pfn, honouring limit
c  reorder the list for next time        move_freelist_head(freelist, freepage)
d  take the page                         __isolate_free_page, counters, cc->freepages[]
e  what to do when nothing was found     the if (!page) block
f  record progress                       compact_cached_free_pfn, total_free_scanned,
                                         fast_isolate_around

fast_isolate_setup() for a, fast_isolate_scan_order() for b+c+d together, fast_isolate_fallback() for e, leaving f in a ~30-line main function. b, c and d stay together because c's correctness depends on freepage referring to the list b actually walked, and d must run under the same zone->lock acquisition. Separating c from b is precisely the mechanical split that would make review harder, which is why this function is the worked example for CS-13.

2. __compact_finished(), 109 lines. Four unrelated decisions: scanners-met, proactive compaction, defrag_mode, and the direct-compactor "is a suitable page free" loop. Extract the last as compact_suitable_page_free(), because it is the only theme the series replaces, making the later change a one-line substitution instead of surgery inside a four-way branch.

3. __alloc_contig_migrate_range(), 55 lines. One theme in the wrong home. Split the isolate, reclaim and migrate loop from the compact_control setup so it takes a range and a target policy. It has one caller only because it is not callable.

4. unreserve_highatomic_pageblock(), 71 lines. Victim selection across the zonelist, then conversion and accounting. Extract the conversion; three nested loops with an early return in the middle.

5. Rename free_area_empty() to free_list_empty(). It reads as a question about the whole free_area but tests one migratetype's list, and the series makes the distinction load-bearing.

Their series positions are 9b, 9c, 15a, 20a and 9a respectively — each one patch ahead of its consumer.

A function modified twice gets one cleanup, before the first. fast_isolate_freepages() and __compact_finished() both touch the free lists, so patch 9 rewrites them and patch 16 changes their behaviour. Splitting them at 9b and 9c rather than 16a and 16b keeps patch 9's mechanical diff readable and leaves patch 16 the one-line substitution it was supposed to be.

16.3a The page_alloc.c functions, one at a time

Read on 848acc8ffe1b, counting braces to braces. Two of the five come out under the cap, two come out over it on purpose, and one should not be touched.

__zone_watermark_ok(), 77 lines, 25 of them comment. Split, and it is the clearest win of the five. Three themes with no shared mutable state:

a  lower the bar for reserve callers   ALLOC_RESERVES, ALLOC_MIN_RESERVE,
                                       ALLOC_NON_BLOCK, ALLOC_OOM: pure
                                       arithmetic on a local `min`
b  the order-0 answer                  two lines
c  is any suitable page free            the descending order loop over
                                       free_area[] and free_area_empty()

watermark_min_for_flags(mark, alloc_flags) returns a long and touches nothing else. zone_has_suitable_free_page(z, order, alloc_flags) is the loop. That leaves about twelve lines in which the order of the two checks is visible at a glance, and it puts each of the series' two changes inside its own helper: §7's headroom subtraction belongs to a, and c is what becomes gigablock-aware. Lands as 12a.

try_to_claim_block(), 65 lines, 23 of them comment. Split one theme out, and it is the one carrying the subtlety. Computing alike_pages is twenty lines of reasoning about what a non-movable page in a movable pageblock can be assumed to be, and it is a pure function of four scalars: count_alike_pages(start_type, block_type, free_pages, movable_pages). Extracting it puts the count next to the threshold test that consumes it, instead of a page apart. The current_order >= pageblock_order whole-block path is a second candidate — seven lines, self-contained, early return — and taking both leaves roughly forty lines. §5.8's gigablock term goes into the threshold decision, which is exactly what becomes readable. Lands as 14a.

__free_one_page(), 87 lines, 16 of them comment. Split the tail only, and accept staying over the cap. The obvious extraction is the merge loop, and it is the wrong one: buddy_pfn is assigned inside the loop and read after it by buddy_merge_likely(), and page, pfn and order all mutate across iterations. A helper would need four out-parameters or a state struct to say what the loop says now, and compaction_capture()'s early return carries a compensating account_freepages() that a careless extraction drops. Take the done_merging tail instead — set the order, choose head or tail, add to the list, notify reporting — which is self-contained, about fifteen lines, and is where the series' free-path accounting hook goes. That leaves roughly sixty-five lines, still over the cap, and the changelog should say the merge loop was left deliberately rather than let a reviewer assume it was missed. Lands as 3a.

__rmqueue(), 63 lines, 18 of them comment. Extract the preamble; leave the switch. The switch (*mode) is four cases chained by fallthrough, and the escalation order — matching list, CMA, claim, steal — is the entire content of the function. Hiding a case behind a helper would hide the fallthrough, which is the one thing the reader is there for. The CMA-balance preamble ahead of it is a different decision that happens to be adjacent: rmqueue_cma_balanced(zone, alloc_flags) as a predicate leaves about forty-eight lines and makes the escalation ladder the first thing in the body, which is where the gigablock search inserts. Lands as 13a.

__rmqueue_claim(), 49 lines, 14 of them comment. Do not split it. One descending loop implementing one policy, largest fallback block first, with a three-way result and an early break; the body is under twenty lines of code. Any extraction here is a helper called once taking six parameters, which trades a readable loop for an unreadable signature. The series changes which free_area the loop reads, not what the loop does, so there is nothing for a prep patch to prepare. Leave it at 49 and say so.

Two of five stay over 40 after cleanup, on the grounds above rather than by omission. kernel-style.md §3 treats the cap as a prompt to examine rather than a limit to satisfy, and a declined split should be a recorded decision.

Every prep patch, and where it lands.

3a   __free_one_page()               tail only, stays over cap    §16.3a
9a   free_area_empty() rename
9b   fast_isolate_freepages()        six themes                   §16.3
9c   __compact_finished()            one theme of four            §16.3
12a  __zone_watermark_ok()           two helpers, biggest win     §16.3a
13a  __rmqueue()                     preamble only, switch stays  §16.3a
14a  try_to_claim_block()            count_alike_pages()          §16.3a
15a  __alloc_contig_migrate_range()  make it callable             §16.3
20a  unreserve_highatomic_pageblock() conversion out              §16.3
     __rmqueue_claim()               declined, 49 lines           §16.3a

Left for a separate cleanup series. isolate_freepages(), isolate_freepages_block() and page_reporting_cycle() are all over the cap, but the series either does not modify them or touches a line or two. Splitting them is somebody's cleanup series, not a prerequisite for this one, and bundling them would make the series look like a drive-by refactor of mm/compaction.c.

16.4 Debt to name but not fix

struct free_area accounting is asymmetric. free_list[] is per migratetype but nr_free is a single total across all of them, so "is there a free block of this order and type" is a list_empty() while "how many blocks at this order" is a count that cannot answer it. That asymmetry is precisely why the design needs a bitmap rather than reusing counters. Making it coherent means either per-migratetype counters on the hot path or dropping nr_free, both behaviour-affecting refactors of the allocator's hottest structure. State it in the cover letter as the reason the bitmap exists.

Comments around the claim path may be stale. The fallback code was reworked between e1914add2799 and 848acc8ffe1b (-2 became FALLBACK_NOCLAIM) and should_try_claim_block()'s comment block predates that. Read it on the new base before quoting it.