PorkiCoder Research · Tab titles

Four beams, no new weights

The decisive gain came from search, not new weights: compare complete, safe pairs of words already on the page.

15 August 2026 · Clean development n=1,000 · Blind Gemini 3.5 Flash-Lite, two decimals · Sealed 1,000 unused · Continues The 35M that caught Hybrid A

This is a new result, not a revised table

The last paper said the 35M was still short of FLAN plus centroid and warned that beam search was too expensive to ship. The quality conclusion changed; the latency objection narrowed in a controlled Ada diagnostic, but remains open in the application. The winning change was comparing complete two-word titles instead of committing to the first locally likely token. That reverses the quality result, changes the research pin, and gives us a mechanism worth testing elsewhere.

“FLAN” here means the exact title-tuned FLAN-T5-small descendant used throughout this campaign, not untouched stock FLAN and not a claim that a weekend run reproduced Google’s instruction training.

The paired result

Absolute means move when Flash-Lite sees a different set of anonymous candidates. Compare systems only inside one row below. The first row is a focused three-way packet; the last two comparisons share a separate six-way packet. Both use the same reusable 1,000-row clean-development set; the sealed 1,000 remains unopened. Means are on a 0–10 scale, and W/T/L is always 35M versus FLAN.

The fresh six-way packet also included the pre-correction beam output, corrected greedy, and bounded variants. The occurrence-level safety correction cost 0.004 points in that same packet; the win was not caused by the bug it removed.

The one change

Greedy and beam-4 see the same legal titles. Greedy chooses the best next token, then lives with that choice. Beam-4 preserves four partial paths long enough to compare whole pairs. In two fresh packets, beam-4 beat corrected greedy by 0.481 (447/325/228) and 0.453 (472/321/207). Beam-2 helped but trailed beam-4 by 0.188; beam-8 was effectively tied at -0.015 and cost more. The default length penalty of 1.0 won its sweep.

We are writing an app that sends email invitations. The template looks bad in Outlook 2007 because Microsoft uses Word as the rendering engine.

A developer moving from SQL Server to a LAMP hobby project asks how to start using stored procedures with MySQL and PHP5.

A developer wants a tested PHP library for safer picture uploads rather than designing one from scratch.

“On the page” had to mean what the encoder saw

The first constrained decoder extracted words by decoding a truncated token sequence. Removing punctuation and unknown pieces could merge fragments into words that never appeared, such as a regex becoming dd. We moved lexical extraction back to the untouched source text and used fast-tokenizer offsets only to enforce the truncation boundary.

A second review found a subtler hole. A word could tokenize cleanly in isolation while its actual source occurrence overlapped a tokenizer piece missing from the reduced vocabulary. Eligibility now requires at least one complete visible occurrence whose overlapping source token IDs are all mapped and non-UNK; the offered target spelling must also be fully representable. This changed five of 1,000 raw titles.

We cleaned the training data. Training still lost.

The obvious next move was to continue B-9500 on exactly the two title-derived page words it should own, with the centroid attached only after inference. The rule was one immutable JSONL miner, one authenticated shared token pack, then cache-only trainers. That workflow exposed two data bugs before it exposed the larger modeling result.

Cleaner targets improved literal copying and sometimes the greedy result. They did not preserve the ranking over complete pairs. The best v5 continuation was 0.647 behind B-9500 under beam-4. V6 at step 1,000 was 0.534 behind; at step 3,000 it was 0.571 behind even though its greedy decode edged B-9500 by 0.028 in that packet.

We then removed the auxiliary losses and made the update tiny. After only 500 CE-only steps at 5e-6, the beam result was already 0.601 behind the pin. At 2e-5 it was 0.758 behind. The pattern suggests that useful ranking was present in B-9500’s sequence distribution; optimizing short labels made that distribution easier to copy from greedily and worse to search.

The frozen centroid won its own ablation

We also tried more conservative centroid_v2 selection: a capped-picker bundle (minimum document frequency, capped IDF, and digit/hash filters), mapped-word-only filtering, and both together. Each tested cleanup reduced the product score. The promoted system therefore uses the original frozen centroid_v2 and locked Hybrid A ordering. No centroid text was attached during training.

The 32-character contract is a separate switch

Raw beam titles were already safe (maximum 31 characters), but locked Hybrid A can append a long centroid word. It exceeded 32 characters on 49 of 1,000 beam rows; FLAN’s locked glue did so on 73. A separate bounded combiner skips whole words that would overflow instead of slicing through one. That strict variant produced zero over-length titles and still beat bounded FLAN by 0.601.

Size stayed fixed; search overhead was modest in the Ada diagnostic

The namer remains 35,045,888 parameters, versus roughly 77M for FLAN-T5-small. No second model, reranker, or learned head was added. On identical RTX 6000 Ada boxes, nine fresh 100-row batch-one processes per decoder averaged 18.72±0.11 seconds for greedy and 20.33±0.21 for beam-4 (mean ± sample SD, n=9), an 8.6% increase. One 1,000-row batch-64 run took 18.19 and 19.82 seconds, a 9.0% increase.

Those timings include model load and candidate construction. They are controlled throughput diagnostics, not per-request production latency. Apple MPS showed a larger batch overhead. The beam path still needs measurement in the actual worker and on target hardware.

The revised pin

FLAN is no longer the quality gate on this clean-development packet. It remains an important capacity benchmark and a useful fallback until the new decoder is timed and shadowed in the application. The model checkpoint did not change; the product change is the constrained decoder and, when required, the bounded combiner.

What we are not claiming

• This is repeated clean-development evidence, not a sealed-test result. The untouched 1,000 remains the final gate.
• Packet means are not interchangeable. The paired delta and W/T/L inside each packet are the evidence.
• The benchmark is our title-tuned FLAN descendant, not untouched google/flan-t5-small.
• The decoder can only choose page words. The frozen centroid still fills the remaining slot(s) and sometimes supplies the mistake.
• Ada and MPS process timings are not production request latency, energy, or memory measurements.
• The 32-character-safe result uses a separately scored bounded combiner; we did not silently mutate the locked baseline.

Reproduce the raw beam decode

python scripts/generate_constrained_titles.py \
  --model output/gsg_scaleup_20260814/keepers/B/checkpoints/step_9500 \
  --tasks data/derived/campaign/so_clean_holdout_v1/tasks_dev.jsonl \
  --tokenizer assets/tokenizer \
  --vocab-map data/derived/campaign/technical_250m_vocab.json \
  --constraint-mode source-words --strip-wrapper \
  --max-content-words 2 --max-source-length 160 \
  --max-new-tokens 8 --num-beams 4 --length-penalty 1 \
  --output constrained_b9500_words2_tok8_beam4_ctxmapped.jsonl

Generic-title banning stays off; it hurt the product packet. The default source-word cap is 96.

Reproducibility hashes

• B-9500 model.safetensors: 0f45a1fef0b549bac512243df0aadad64aedec2a6ade7c17af730247b0b3d52a
• B-9500 config.json: bd146261ee5eaab464002123f5680cc17a0547daee9397fc2d54ec8e8870b8d9
• Frozen centroid_v2 JSONL: a28aa740572bc369150006bb9e3a850e967555289046c480d64216f9390a160c
• Corrected raw beam JSONL: f736d9c1e232d99fc56d21bc208004c998c576e0aea085d2fd7742619dd90477
• Locked-glue JSONL: 5f95a123491fd1c8f9216ed149db57ac62f9bd8638ec0b564e2aa9b07cd471ca
• Bounded-32 glue JSONL: 9c9b1bff3a5241e5442f0e0b07103d7472c67f6c5fc6f3b5168726ac0f069928

Implementation and audit record: commit 931e57b. Full campaign findings, packet paths, cache hashes, and teardown record live in docs/beat_flan_quality_push_20260814.md.