The Cheapest Way to Make Open Weight AI Models Better

We raised a 4B model’s accuracy from 32% to 72% without training or fine-tuning.

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction. We run on a “pay what you can” model—so if you believe in the mission, there’s likely a plan that fits (over here).

Every subscription helps me stay independent, avoid clickbait, and focus on depth over noise, and I deeply appreciate everyone who chooses to support our cult.

PS – Supporting this work doesn’t have to come out of your pocket. If you read this as part of your professional development, you can use this email template to request reimbursement for your subscription.

Every month, the Chocolate Milk Cult reaches over a million Builders, Investors, Policy Makers, Leaders, and more. If you’d like to meet other members of our community, please fill out this contact form here (I will never sell your data nor will I make intros w/o your explicit permission)- https://forms.gle/Pi1pGLuS1FmzXoLr6

---

(Help me out by sharing this article on LinkedIn here)

Thousands of engineering teams are burning margin, forcing heavy frontier models to handle basic production tasks because smaller, cheaper open-weight alternatives can’t deliver reliable outputs. And on the surface, they’re completely right.

A model like Qwen3–4B — one of the best small open-weight architectures on the market — gets basic arithmetic wrong almost all the time, landing a correct answer just 32% of the time. With odds like that, you might as well hit the roulette and gamble like a proper degenerate.

But if you dig into the failure data, an interesting stat emerges — it actually computes the correct answer inside its internal states an impressive 80% of the time. Somehow, the model gets to the correct answer and then veers off track. This tells us that the issue isn’t in the base intelligence (the model has the knowledge somewhere), but in navigation (discovering the knowledge reliably and then stopping). In fact, when we tracked whether models finished their responses naturally versus hitting the token limit, every single naturally completed answer was correct (across 500 generations). Wrong answers are destinations that our model failed to reach, not destinations that our model lacked in its map (full EOS analysis in the appendix).

This means that models get trapped in an optimization loop — formatting, restructuring, and elaborating indefinitely — until they brick the run before ever stating the final result. What we need is a system that reliably find and unlocks the knowledge already hidden in the language model.

Last time I wrote about Latent Space Reasoning, the system we broke down required a pretty sophisticated setup requiring trained judges, specialized aggregation, and a universal embedding projected space. Since then, our team at Irys has been exploring simpler ways to unlock the hidden knowledge in models that bring the power of latent space reasoning to less technical teams. And our results have been very compelling — 

• Injecting just two random vectors into the model’s embedding space before it starts generating forces it to break the lock-in, spiking that 32% success rate straight to 51.6%. Re-read that number — we almost doubled accuracy by adding 2 random tokens.

• Bump that to ten random vectors with a basic plurality vote, and the success rate hits 72%. (Use plurality, not majority voting — majority actually performs worse than baseline when individual seed accuracy is below 50%. Details in the appendix.)

• At ten vectors, every single task in the benchmark gets solved by at least one seed — 100% oracle coverage extracted purely from random noise.

• On a Redis debugging task, the baseline model spit out 14 incoherent words. With two random tokens, every single seed produced a complete 650-word diagnostic plan.

• On 12 legal reasoning tasks, the top seed beat the baseline on 11 of them.

• Smaller models with perturbation beat or matched larger models of the same family more than half the time, proving that this technique unlocks a new axis of capabilities w/o needing increased hardware or training investment.

These results allow us to get better performance from smaller models than from bigger models AND for lower cost (small model + multiple runs is much cheaper than big models).

^^Experiment details on GitHub. Once again, our system unlocked performance that parameter scaling did not.

Given how impactful our findings are for democratizing intelligence for everyone, we’ve open-sourced what we could on our latent space reasoning repo — github.com/dl1683/Latent-Space-Reasoning. While the effects of our research are statistically significant (McNemar p < 0.001 across two separate models), we recognize how small our sample size is. This is where we’d love to collaborate with the rest of our open community to scale up the experiments and look into different outcomes.

Executive Highlights (tl;dr of the article)

The finding: A Qwen3–4B model computes correct arithmetic answers 80% of the time but only states them 32% of the time. The model isn’t failing to reason — it’s failing to finish. It gets trapped formatting answers it already computed.

The fix: Injecting 2 random vectors into the model’s embedding space before generation breaks the lock-in. No training, no fine-tuning, one line of code. Accuracy jumps from 32% to 51.6% on a single seed, 72% with plurality voting across 10 seeds, and 100% oracle coverage — every task solved by at least one seed.

It generalizes. On a Redis debugging task, baseline output was 14 incoherent words. With 2 random tokens, every seed produced a 650-word diagnostic plan. On 12 legal reasoning tasks, the best seed beat baseline on 11. On incident response, evolved vectors surfaced honeypot deployment, MITRE ATT&CK analysis, and HSM credential rotation from a 4B model that baseline could only get “rotate credentials, check logs” out of.

Direction doesn’t matter. Random noise and carefully optimized projections produce identical results (Mann-Whitney p = 1.000). The perturbation provides energy, not information. Leading candidate explanation: stochastic resonance. Three independent groups (Kim et al., Shi et al., Pfau et al.) reached the same conclusion through different methods.

Small models + perturbation beat bigger models. Scaling from 4B to 14B bought 4 percentage points on our arithmetic benchmark. Changing how we interrogated the 4B model bought 40. The 4B runs on a $429 GPU at 200 tokens/sec. The 14B needs a $1,999 GPU at half the speed. At cloud rates, 10 perturbation seeds cost $0.009/query versus $0.45 for GPT-5.6 Sol thinking tokens — 56x cheaper.

Practical guardrails. Use 8-bit quantization, not 4-bit (4-bit nearly eliminates the effect). Use plurality voting, not majority (majority performs worse than baseline). If your model already scores 75%+ on a task, perturbation can hurt mean accuracy — it helps stuck models, not cruising ones.

What doesn’t work. Our original verbosity claim was a code bug. The scorer is barely trained (broken on 9/12 legal tasks). DeepSeek showed a negative mean effect at high baseline. The data breach task exposed hallucination when the model lacked the knowledge entirely. Sample sizes are modest: 25 arithmetic, 5 planning, 12 legal tasks.

What’s next. Temperature comparison (the test that determines if this is a new technique or a mechanistic insight), gated attention probe, position-shift ablation, scorer improvement. Full pipeline preregistered and open-source.

The thesis: The industry’s default answer to “my model isn’t good enough” is a bigger model. Our results say that for a meaningful class of tasks, the capability was already in the small model — the bottleneck was the inference path. Lifting the ceiling gets the headlines. Raising the floor is where the new value gets created.

Everything is open-source: github.com/dl1683/Latent-Space-Reasoning

I put a lot of work into writing this newsletter. To do so, I rely on you for support. If a few more people choose to become paid subscribers, the Chocolate Milk Cult can continue to provide high-quality and accessible education and opportunities to anyone who needs it. If you think this mission is worth contributing to, please consider a premium subscription. You can do so for less than the cost of a Netflix Subscription (pay what you want here).

Want access to a repository containing all of our research? 300+ files containing our notes of various experiments, discussions with cutting-edge teams, and insights into where the industry is headed next. Get a Founding Member Subscription to AI Made Simple. Want to talk to me for details/get my insights into the tech ecosystem? Reach out to me through any of my socials over here or reply to this email.

How Standard Formatting Might Be Making Small Models Fail

The standard narrative (small open-weight model hallucinations/mistake are a result of their lower intelligence) is true quite often, but as we’ve derstand why, let’s quickly remind ourselves of what models are doing when they do inference.

Language models generate text autoregressively — one token at a time. Because each token becomes part of the hard context window for every subsequent decision, the first token constrains the second, and the first two dictate the third. In our research across multiple models and sizes across multiple domains, we found that the first 20 generated tokens tend to disproportionately impact the AI context generation (even in very long context generations; research shared in our comprehensive proprietary AI research map available to all founding members of the Chocolate Milk Cult over here). This leads to the “autoregressive lock-in” phenomenon we have discussed extensively in the past.

Image Source

In practice, this will manifest in several ways:

1. Models will spend tokens on the formatting and run out of their token budget, even when they know the answer.

2. Models might follow a reasoning path, realize it’s no good (either themselves or by your injection), but the false path will STILL influence the outputs.

3. The models you pay for by token are occasionally setting their token budgets on fire to format answers they’ve already computed. We can call this structural failure formal presentation mode. Left to its own devices, the model burns through its footprint producing immaculate headers, enumerated steps, and clean LaTeX expressions. In other words, it commits to a visual template and mindlessly fills it in — performing the aesthetic of math rather than actually doing the math (anyone getting shades of pulling up a terminal in front non coders to seem “hackery”).

Knowing that, we decided to compute the geometry of the model’s search space (nerd stuff, not the most relevant here; check next section, GitHub experiments or gimme a shout if you want deets) and then used that geometry to disrupt that default trajectory by prepending a couple of random, noisy tokens to the embedding space. This pushed the models into exploratory computation mode — a messy, stream-of-consciousness scratchpad.

Look at the text execution on the arithmetic problem (45 + 23) * 17 - 89:

Without prefix (Wrong — bricks on the formatting template):

Let me solve this step by step.
Step 1: Calculate 45 + 23 = 68
Step 2: Multiply by 17: 68 x 17 = ...

With prefix (Correct — hits the exact payload in half the footprint):

ok so first 45+23 thats 68, then times 17... 68*10=680, 68*7=476, so 680+476=1156, minus 89, 1156-89=1067

The random prefix doesn’t inject any new knowledge into the models (it can’t). Instead, it seems to bypass a lot of the hacking behaviors injected from the alignment process and forces the system to compute directly.

Ever since Wei et al. dropped their foundational Chain-of-Thought paper, the consensus has been that intermediate steps boost accuracy. But the ecosystem completely missed the real bottleneck: chain of thought worked best when the ideas occur in overlapping knowledge regions—

This proved that intelligence was a “search through the latent space” issue (in hindsight, this paper had much bigger implications than any of us realized). It also gives us an interesting framing for the ubiquity of the formal presentation mode: the alignment hacking present in modern post-training pushes form over substance. The random token perturbation allows us to attack this gap by undercutting the formatted alignment spaces and leads to a system that wastes 800 tokens on beautifully formatted, incomplete garbage (base LLM), losing to a model that spends 400 tokens on a messy, complete solution (our perturbed variant).

Take one second to appreciate how insane this is. The most common engineering band-aids either operate way too late after the trajectory is already committed (like temperature sampling or best-of-N), require heavy gradient updates (fine-tuning), or try to extract diversity during token selection rather than the computation step (self-consistency). We avoid all of that by directly attacking the root of this issue, and are able to get significantly better results out of this with a solution that’s practically free.

In simple words, change where the model starts computing and how it searches a predefined space, and you can dramatically update its results. So how exactly does that work? We will look at that next.

What If You Shift Where the Model Starts Computing?

In the vein of data augmentation techniques like RandAugment, we take two vectors of random numbers, scale them to match the model’s native embedding magnitude (an RMS of ~0.022 for Qwen3–4B), and prepend them as positions 0 and 1 before your prompt tokens. The intuition here was similar to what made RA so effective — the randomness should inject a level of diversity into your search space, ensuring better performance.

Our thrifty twosome now sits at the start of every model call, shifting the computational trajectory through every subsequent layer from the very first real token onward. Apparently, this randomness is all you need to start unlocking model performance. We tested this approach against a simple, uncalibrated version of the latent space reasoning pipeline we had shared earlier:

• Optimized projections vs. Random noise: Mann-Whitney p = 1.000. The distributions are identical.

• Euclidean vs. Hyperbolic mutations: p = 1.000. Same result.

In other words, this setup provided the same level of geometric exploration as a more complicated exploration framework. This leads to our next interesting question — why would random tokens make LLMs better?

Why Pure Noise Unlocks AI Model Reasoning

Truth be told, no fucking idea. That’s one of the reasons I’m open-sourcing this, so that one of you can figure this out while I take all the credit.

Until that happens, I’ve had some researchers from Irys and the larger Chocolate Milk Cult looking into this. Our leading candidate explanation is stochastic resonance — the physical phenomenon where adding noise to a non-linear system amplifies weak signals that would otherwise fall below an activation threshold. This is an idea we explored over here when looking into why Brains are so much more efficient than LLMs by using Sub-Landauer patterns (patterns that encode some useful information, but are not given “full energy” so they only activate when a bunch of Sub-Landauer patterns trigger together to collectively overcome the energy threshold) — 

Mapping this to NNs, standard transformers are like a non-linear threshold system: the correct reasoning path exists in the model’s parameters but sits just below the greedy selection boundary. Random embedding noise provides the activation energy needed to push the computation over that threshold. Because the noise provides energy rather than information, optimizing its direction adds zero value (which is why directional perturbations without tweaking magnitudes didn’t present much utility).

We are currently running experiments to isolate stochastic resonance from competing accounts (attention redistribution, positional offsets from shifted RoPE IDs, and attention-sink disruption). If you want in, just pull up and start running some experiments yourself.

In the meantime, we have some reason to be confident in our assessment of the situation since independent teams are reaching the exact same conclusion across different setups:

• Kim et al. (“Random Soft Prompts”): Showed random, unoptimized embedding vectors match the accuracy of heavily trained soft prompts on math tasks.

• Shi et al. (“Meaningless Tokens”): Found inserting sequence strings of meaningless discrete tokens consistently boosts reasoning via activation redistribution.

• Pfau et al. (“Let Me Speak Freely”): Demonstrated that simple repeated filler tokens (“…”) allow hidden computation in the residual stream.

All 3 seem to be pointing at the same thesis — the perturbation matters much more than specific tuning. At least until we have a more rigorous map of the latent space itself.

Why Perturbation Changes the Unit Economics of LLM Reasoning

Why go through this effort? For the vibes? Streetcred? To find some way to spend my Codex tokens (unfortunately, that’s not true for me since I don’t get paid to tokenmax)?

Nah, it’s coz I’m a gareeb, and I need to find ways to make AI cheaper. A lot cheaper so that Irys company lunches can go from two packs of Oreo shared b/w all of us, to at least 4 Chipotle bowls for the whole team. So watch your dearest Dev Dev fight his way to put more meat in his mouth.

Turns out that even accounting for the 10 extra inference passes of the smaller model (+ the oracle), latent-space reasoning extracts far more capability per dollar without ever forcing you onto a bigger model. This changes your business in two meaningful ways — 

1. You can route more work to smaller models safely and get the same threshold of performance. This opens up a whole new class of latency-sensitive work that couldn’t be done before, while also driving down the costs of the existing agentic loops where you might be worried about costs.

2. The magnitude of the performance shifts means that you can deploy the same level of intelligence in a much cheaper class of hardware.

Both will massively speed up the Jevon’s Paradoxification of AI, ultimately massively upping the value generated by AI.

Following are some rough markers of short-medium ROI I put together across various scenarios (image is a summary, we’ll break them down)—

Raw Performance

On our arithmetic benchmark:

• Qwen3–4B scored 32% under ordinary greedy inference.

• Prepending a two-token perturbation raised that to 51.6%.

• Ten perturbed trajectories with plurality voting hit 72%.

• Qwen3–14B — 3.5 times the parameters — scored 36% (the small delta b/w 4B base and 14B isn’t surprising here since the arithmetic isn’t going to be novel for even the 4B model).

In other words, scaling the model bought four percentage points. Changing how we interrogated the small model bought forty. The 14B model isn’t even competing here. This becomes much more impactful when you consider the macroeconomics and the memory supercycle (companies like Micron and Nvidia are committing highway robbery b/c of how expensive High Bandwidth Memory has become in GPUs) and how LSR (even single pass with perturbation) gets us higher gains than scaling, w/o increasing the hardware requirements (the implications of this will be made clear in the next sections).

The Hardware Class Gap

Model size dictates the minimum hardware required to run the system. From Qwen’s official AWQ benchmarks:

• Qwen3–4B: 2.9 GB VRAM, 199.7 tokens/sec

• Qwen3–14B: 10.0 GB VRAM, 96.5 tokens/sec

• Qwen3–32B: 19.1 GB VRAM, 47.7 tokens/sec

The 4B model sits in a completely different deployment class, meaning that we get cheaper cards, more room for context, and double the throughput of 14B. How much cheaper? Take a look at the difference:

• RTX 5060 Ti (16 GB): $429 at launch

• RTX 5090 (32 GB): $1,999 at launch — before the rest of the rig

On Runpod’s current serverless rates, a 16 GB worker runs $0.58/hour versus $1.58/hour for the 32 GB tier. That is a 2.7x hourly premium for a larger model that generates at half the speed.

Factoring in current cloud rates, power draw, and the May 2026 US residential electricity baseline (18.44 cents per kWh):

• $1,000 local machine (16 GB): pays for itself after ~1,830 GPU-hours

• $3,250 local machine (32 GB): pays for itself after ~2,205 GPU-hours

At 240 GPU-hours per month, you are looking at your investments paying back more than a month earlier — 

Worth stressing here is that the lower upfront hardware costs enable market creation since the costs lower the barrier for participation.

Unit Economics vs. Frontier Models

Let’s now look at the calculations from a cloud costing perspective. Using standard market baselines (Qwen3–4B at 4-bit, 2,048 token sequence, continuous prefix noise at positions 0 and 1):

• 10 Perturbation Seeds ($0.008 per query): 10 parallel generation passes = 20,480 tokens at $0.40/M. The 2 prefix vectors add 0.097% overhead — pure rounding error.

• 10 Seeds + Scorer MLP ($0.009 per query): A 300K-parameter MLP evaluates candidate trajectories in one pass on short sequence embeddings, adding ~$0.001.

• GPT-5.6 Sol Thinking Tokens ($0.45 per query): OpenAI’s flagship bills thinking tokens at the output rate ($30.00/M). Mid-range queries burn ~15,000 thinking tokens before answering ($30.00 × 0.015).

Computing the difference, we get Sol being 56x more expensive than our system.

Take a second to consider what that means at scale. At 10,000 queries per day, that is $90/day versus $4,500/day. Over a month, you are comparing $2,700 to $135,000. At 50,000 daily queries, the perturbation approach stays under $15,000/month while Sol crosses $675,000.

Now, this is not meant to say that our system will replace Sol/big models everywhere. Ultimately, no matter how much we might wish, latent space reasoning can’t make existing models more knowledgeable. So in many tasks, a heavier model will simply outperform a lighter model, no question. Similarly, if your model already scores well on a task (75%+), perturbation can actually hurt mean accuracy since it disrupts trajectories that were already efficient. The oracle still improves performance, but the average gets pulled down by bad perturbations.

However, this also leaves a large gulf of techniques where the simpler models have the base knowledge, but they get stuck in unhelpful attention sinks they can’t avoid. And your constraints limit you from hitting bigger models. In that case, we believe that Latent Space Reasoning with Perturbations can help your model punch above it’s weightclasses.

Conclusion: The Overlooked AI Revolution

The market is currently pricing in only one economic revolution: pushing the capability ceiling. Frontier labs are burning billions to maximize peak capacity. As economic pressures mount, this will lead to higher prices for consumers as the labs have to justify their massive capex buildouts. We’re already seeing this manifest with the higher token prices in recent models (both directly in PPM and indirectly with models consuming more tokens for their tasks).

Take a second to consider that there is a second revolution: raising the floor. The smartphone didn’t win by being a faster laptop. It won by delivering a fraction of a laptop’s compute at a hundredth of the cost, completely changing the deployment envelope. That cost collapse birthed entirely new economies — ride-hailing, mobile logistics, the app ecosystem — because adequate compute became cheap enough to integrate a billion new participants.

Delivering adequate intelligence at a radically lower cost changes who gets to participate in the system. It makes problems economically tractable that simply never pencil out at frontier pricing. You cannot integrate the broader economy when operators are bleeding out on $135,000/month inference bills.

APPENDIX: Full Experimental Detail

A1. Arithmetic: Dose-Response, Controls, and Voting

Setup. Prefix-length sweep from 0 to 8 tokens. Qwen3–4B at 4-bit quantization. 25 multi-step arithmetic tasks (nested expressions, 3–6 sequential operations).

Dose-response. Baseline (0 tokens): 32.0%. 1 token: 42.7% (+10.7pp, n=3 seeds). 2 tokens: 51.6% (+19.6pp, n=10) — — the peak. 3 tokens: 44.0% (+12.0pp, n=10). 8 tokens: 44.4% (+12.4pp, n=10). McNemar p = 0.000015 at 2 tokens. Performance peaks at 2 and drops at 3 and 8 because of token budget competition: correct runs average 718 tokens, wrong runs hit the 1,024 ceiling and get truncated. At 8 prefix tokens, the model branches into multiple partial strategies and exhausts its budget before concluding any of them.

Controls. Zero embeddings (adding positions only): 36.0%, +4pp. Mean embedding (identical values at both positions): 36.0%, +4pp. Random noise (diverse values): 51.6%, +19.6pp. No chain-of-thought: +0pp. The effect requires both diverse prefix values and step-by-step reasoning. Remove either and it vanishes.

Voting at n=10 seeds. Plurality (most common answer): 72% on 4B, 56% on 8B. Oracle (best of 10): 100% on 4B, 80% on 8B. Majority (>50% agreement): 40% on 4B, 12% on 8B — worse than the 16% 8B baseline. Majority fails because when individual accuracy is below 50%, the threshold demands agreement that doesn’t exist among mostly-wrong seeds. Correct answers cluster (different trajectories converge on the same right answer). Wrong answers scatter (each truncation produces a different spurious value).

Caveat. The n=3 scout runs overestimated effects: Qwen3–4B dropped from 60% (n=3) to 51.6% (n=10). 25 tasks establish significance but not precise effect sizes.

A2. EOS Completion

Setup. 250 perturbation responses on Qwen3–4B, 250 on Qwen3–8B. Tracked whether each response finished with a natural end-of-sequence token or hit the 1,024 max token limit.

4B results. Finished + correct: 94 (37.6%). Finished + wrong: 0 (0.0%). Truncated + correct: 35 (14.0%). Truncated + wrong: 121 (48.4%).

8B results. Finished + correct: 32 (12.8%). Finished + wrong: 0 (0.0%). Truncated + correct: 40 (16.0%). Truncated + wrong: 178 (71.2%).

P(correct | finished naturally) = 1.000 on both models. Zero exceptions across 500 responses. Same pattern holds for baseline greedy decoding on this benchmark. EOS responses average 718 tokens. Truncated responses hit 1,024. Perturbation increases the EOS rate: 37.6% vs 24% at baseline. Tested on 25 arithmetic tasks only.

A3. Planning Tasks

Setup. 5 tasks: fraud detection system design, incident response, healthcare data platform, Redis cache debugging, Oracle-to-PostgreSQL migration. 3-way comparison: greedy baseline, random perturbation (5 seeds), evolved latent vectors (5 seeds). Qwen3–4B at 4-bit, 2,048 max tokens, greedy decoding. LLM judge scored on coherence, correctness, completeness, specificity, actionability.

Attention sink rescue. On Redis debugging, baseline collapsed to 14 incoherent words — — stopped generating immediately in a degenerate attention pattern. Every perturbation seed produced 650–710 word complete diagnostic plans. Likely mechanism: attention sinks (Xiao et al.) cause earliest token positions to accumulate disproportionate attention; under greedy decoding, degenerate early-position patterns propagate through every subsequent token. Random noise at those positions disrupts the pattern before it forms.

Evolutionary search. A 300K-parameter MLP scorer evaluates candidate latent vectors and guides an evolutionary population. On incident response, baseline produced “rotate credentials, check logs.” Evolved vectors produced honeypot deployment, MITRE ATT&CK lateral movement tracking, tiered HSM credential rotation, and immutable container rebuilds from verified base images.

LLM judge tally. Perturbation won 3/5, evolution won 2/5, baseline won 0/5. Sample size: 5 tasks.

A4. Legal Reasoning

Setup. 12 tasks across 5 categories: FTC unfairness, GDPR classification, disparate impact, SaaS contract review, acquisition due diligence, data breach triage, IP risk portfolio, negotiation leverage, regulatory response, contractor misclassification, corporate veil piercing, whistleblower retaliation. Same 3-way comparison. Qwen3–4B at 4-bit, 2,048 max tokens. Blind evaluation — condition labels stripped, order randomized, scored on Legal Accuracy, Analytical Depth, Practical Utility, Structural Quality, Completeness (1–10 each).

Oracle results. Best of 5 seeds beat baseline on 11/12 tasks (92%). Average lift: +1.6 points on a 10-point scale. Largest lifts: negotiation leverage 2.0 → 5.4 (+3.4), contractor misclassification 2.2 → 5.6 (+3.4), IP risk 3.6 → 6.4 (+2.8), whistleblower 3.2 → 5.6 (+2.4), FTC unfairness 5.2 → 7.2 (+2.0).

Qualitative examples. Contractor misclassification: baseline gave surface analysis. Best perturbation seed applied state-specific tests (California ABC test, New York economic reality test, Texas common law) with per-jurisdiction liability estimates. Corporate veil piercing: one seed identified “SubCorp appears to be a real operating business, not just a sham shell” — a Delaware Chancery Court doctrine element other outputs missed.

Boundary condition. GDPR controller/processor classification (the 1 loss): exceeded the model’s knowledge. Perturbation seeds consumed full 2,048-token budgets in thinking loops without substantive analysis.

Mean vs oracle gap. Most random seeds produce lateral moves or slight regressions. The gap between 92% oracle and mixed mean results is the scorer problem. Sample size: 12 tasks.

A5. Cross-Model and Quantization

Cross-model results on the 25-task arithmetic benchmark:

Qwen3–4B at 4-bit: 32.0% → 51.6% (+19.6pp), 100% oracle. 80% answer-anywhere at baseline, 32% stated. Perturbation barely changes answer-anywhere (80% → 82%) but jumps stated accuracy. Convergence aid — helps finish, not compute.

Qwen3–8B at 8-bit: 16.0% → 28.8% (+12.8pp, p = 0.000177), 80% oracle. 32% answer-anywhere at baseline, raised to 50%. Exploration aid — finds answers the model couldn’t compute before.

DeepSeek-R1-Distill-Qwen-1.5B at 4-bit: 76.0% → 74.4% (-1.6pp), 100% oracle. Near ceiling already. Perturbation disrupts efficient trajectories more often than it finds better ones.

phi-2 unquantized: 12.0% → 18.7% (+6.7pp), 28% oracle. Low ceiling limits gains.

Quantization. Qwen3–8B at 4-bit: +1.3pp (null result). Same model at 8-bit: +12.8pp (p = 0.000177). Working hypothesis: 4-bit rounds to 16 distinct weight values, too few for perturbation to create meaningfully different trajectories. 8-bit (256 values) preserves enough diversity. Based on one model comparison — may not generalize.

A6. Limitations, Failures, and Next Experiments

Verbosity bug. First analysis claimed perturbation increases verbosity. Artifact of a field-ordering bug (commit 6c69284). Perturbation actually produces 3% fewer words on Qwen3–4B.

Data breach hallucination. All 11 outputs (baseline, perturbation, evolution) fabricated legal content — invented statutes, fictional agencies, incorrect deadlines. Task exceeded the model’s knowledge.

Broken scorer. 300K-parameter MLP was non-deterministic on 9/12 legal tasks. Fix applied (deterministic projection with fixed seed), needs clean re-run. The 48pp gap between 100% oracle and 51.6% mean accuracy on arithmetic is the scorer’s failure to select the correct seed.

Sample sizes. 25 arithmetic, 5 planning, 12 legal. Statistically significant (McNemar p < 0.001 on two models) but effect sizes will shift with scale.

Next experiments (preregistered Phase A pipeline):

Temperature comparison. Head-to-head: prefix perturbation vs temperature sampling vs prompt rephrasing, same tasks/models/compute budgets. If temperature at n=10 matches perturbation’s oracle and mean, the contribution is mechanistic insight rather than a new technique. If embedding-level perturbation accesses trajectory classes token-level sampling can’t, the technique claim holds.

Gated attention probe. Qwen3.5–4B uses hybrid gated DeltaNet + gated attention layers that eliminate attention sinks via post-softmax sigmoid gating. If perturbation disappears on gated architectures, the effect depends on attention sinks specifically. If it survives, the mechanism operates below the attention layer.

Position-shift ablation. Soft prefixes shift RoPE position IDs for downstream tokens. Decomposes the total effect into: position offset, embedding diversity, and their interaction.

Scorer improvement. Domain-specific training, more sophisticated architectures, learned aggregation. The unresolved problem is selecting the correct candidate from among seeds.

Thank you for being here, and I hope you have a wonderful day,

Dev <3

If you liked this article and wish to share it, please refer to the following guidelines.

That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, my links will be at the end of this email/post. And if you found value in this write-up, I would appreciate you sharing it with more people. It is word-of-mouth referrals like yours that help me grow. The best way to share testimonials is to share articles and tag me in your post so I can see/share it.

Reach out to me

Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.

Small Snippets about Tech, AI and Machine Learning over here

AI Newsletter- https://artificialintelligencemadesimple.substack.com/

My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/

My (imaginary) sister’s favorite MLOps Podcast-

Check out my other articles on Medium. :

https://machine-learning-made-simple.medium.com/

My YouTube: https://www.youtube.com/@ChocolateMilkCultLeader/

Reach out to me on LinkedIn. Let’s connect: https://www.linkedin.com/in/devansh-devansh-516004168/

My Instagram: https://www.instagram.com/iseethings404/

My Twitter: https://twitter.com/Machine01776819