Playback speed

Share post at current time

Share from 0:00

0:00

How to Make Legal AI Truthworthy

How you can use Irys's AI platform to get much better results in your legal work

Jul 13, 2026

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction. We run on a “pay what you can” model—so if you believe in the mission, there’s likely a plan that fits (over here).

Every subscription helps me stay independent, avoid clickbait, and focus on depth over noise, and I deeply appreciate everyone who chooses to support our cult.

Help me buy chocolate milk

PS – Supporting this work doesn’t have to come out of your pocket. If you read this as part of your professional development, you can use this email template to request reimbursement for your subscription.

Every month, the Chocolate Milk Cult reaches over a million Builders, Investors, Policy Makers, Leaders, and more. If you’d like to meet other members of our community, please fill out this contact form here (I will never sell your data nor will I make intros w/o your explicit permission)- https://forms.gle/Pi1pGLuS1FmzXoLr6

Thanks to everyone for showing up the live-stream. Mark your calendars for 8 PM EST, Sundays, to make sure you can come in live and ask questions.

^^Bring your moms and grandmoms into the Chocolate Milk Cult.

Below is a walkthrough of Irys, the legal AI platform that we’ve been building. Sabih (co-founder, CEO, former BigLaw litigator) and Christian (head of product, also former lawyer) walked through a live litigation matter on the platform, showing the research pipeline, the drafting assistant, the matter-level knowledge system, and the personalization stack. The core thread running through the whole stream was trust — why legal AI adoption keeps stalling despite improving accuracy, what it actually takes to build AI that lawyers will put their name behind, and how the architecture we chose for Irys (stateful swarms, reasoning transparency, persistent memory) addresses the specific failure modes that have made every incumbent feel unreliable to practicing attorneys.

If you’re building for any regulated industry where professionals carry personal liability for their outputs, the problems and design decisions discussed here apply directly.

Use Irys to get all your legal work done for free here (irys.ai)

Community Spotlight: Kingdom

Kingdom (the manga) has been an absolute obsession of mine recently. It has a really cool plot, tons of fun characters, and a really good theme underpinning the whole story. I’d highly recommend it if you’re looking for a story to binge through (unfortunately I’ve caught up now, but the feeling of reading peak while having hundreds of chapters left was such a great feeling).

If you’re doing interesting work and would like to be featured in the spotlight section, just drop your introduction in the comments/by reaching out to me. There are no rules- you could talk about a paper you’ve written, an interesting project you’ve worked on, some personal challenge you’re working on, ask me to promote your company/product, or anything else you consider important. The goal is to get to know you better, and possibly connect you with interesting people in our chocolate milk cult. No costs/obligations are attached.

Companion Guide to the Livestream

This guide expands the core ideas and structures them for deeper reflection. Watch the full stream for tone, nuance, and side-commentary.

1. The Trust Gap That Ate Legal AI’s Promise

The Event — Sabih Siddiqi, co-founder and CEO of Iqidis and a former litigator with close to eight years at a major New York firm, opened with a claim that reframes how you should evaluate any AI system targeting regulated industries: the adoption problem in legal AI is not an accuracy problem. It’s a trust problem. And those are different things.

Why this matters — If you’ve been watching the legal AI market from the outside, you’d be forgiven for thinking that the race is about which system hallucinates least. Every vendor has some version of “we reduced hallucinations by X%.” That framing misses the actual bottleneck. Sabih had direct experience deploying incumbents at a firm with the budget to trial everyone — Harveys, Paxtons, the full roster. His assessment was blunt: none of them were worth anything. Not because the outputs were always wrong, but because a lawyer can never know when they’re wrong, which makes the outputs useless for the thing lawyers actually do — put their name on documents with career consequences. The failure mode here is subtle and it matters for anyone building for high-stakes professionals. A system that’s right 95% of the time but can’t prove when it’s right forces the user to re-verify everything manually. That re-verification eats the entire productivity gain the AI was supposed to provide. You end up rereading every document yourself, checking every citation against the source material, and confirming that nothing critical got dropped.

Sabih put it in terms of two axes: trust and work product. Most legal AI has tried to compete on work product alone — better drafts, faster research — while the trust axis remained unsolved. Lawyers have been sanctioned in court for relying on AI-generated citations that turned out to be fabricated. That’s not a hypothetical failure mode, it’s a documented pattern, and it creates an adoption ceiling that no amount of benchmark improvement can break through on its own. The question Irys set out to answer was whether you could move the trust axis independently of the accuracy axis, and that distinction drove most of the architectural decisions the team walked through during the stream.

2. Post-Hoc Checking vs. Showing Your Work

The Event — Dev broke down the current best-in-class approach to legal AI trustworthiness: retrieve your chunks, generate an answer from those chunks, then do a post-hoc verification pass where you check whether every cited case actually maps to a legal database. If it does, ship it. If it doesn’t, flag it. This is the approach that the most careful incumbents have adopted, and it represents real progress over the systems that simply hallucinated citations with no checking at all.

Why this reframes everything — Post-hoc checking treats trust as a filtering problem — generate first, verify after. The assumption is that the generation step might produce garbage, so you build a sieve at the end to catch the worst of it. It works, to a degree. But it has a structural limitation that no amount of engineering on the sieve can fix: the user never sees how the system arrived at its answer. They see the output and the citations, but the reasoning chain between input and output stays opaque. Irys takes a different approach. The system exposes its reasoning process at every step — which sources it’s consulting, how it’s interpreting the legal posture, what defensive strategy it’s identifying, which case law it’s pulling and why. A lawyer can intervene at any point in that chain, not just at the output.

This matters because legal reasoning isn’t just about getting to the right answer, it’s about getting there through the right process. Two lawyers can arrive at the same conclusion through completely different analytical paths, and in law, the path matters — it determines whether the conclusion survives scrutiny, whether it covers the right bases, and whether it holds up under adversarial pressure. The agentic harness Irys built doesn’t just coordinate retrieval and generation. It maintains a visible reasoning trace that functions like a transparent brief-writing process rather than a black box that occasionally produces the right document. Dev made the point that this transparency is also what enables the grounding to work — when you can see the system consulting specific case law from Court Listener and other providers in real time, the citations aren’t decorative footnotes stapled onto a generated text. They’re load-bearing elements that the system’s reasoning actually flows through.

3. The Context Rot Curve

The Event — Dev introduced what might be the single most useful mental model from the stream: an inverted V-shape that describes how agentic system performance relates to context volume. As you add context, the system gets better — more documents, more precedent, richer understanding of the matter. But that improvement hits a peak and then collapses, because the system becomes context-wrought and has to compact. The second you compact, key instructions get lost.

Why this reframes everything — If you’re building anything that requires sustained reasoning over large document sets, this is the curve you’re fighting. Most agentic architectures today run into this wall somewhere between the fifth and fifteenth interaction in a complex task. The context window fills up, the system either truncates or summarizes to make room, and that summarization is lossy in ways that are hard to predict and hard to detect. The instructions that get dropped aren’t random — they tend to be the constraints and nuances that were important precisely because they were edge cases, which means they occupy less space in any summary representation.

Sabih described this from the practitioner side. Before leaving his firm, he was closing out a large arbitration using incumbent legal AI tools and hit context limits after roughly ten prompts. Even when he tried to push through, the system’s memory of the matter had rotted — it was losing the thread of the case, forgetting earlier context, producing outputs that no longer reflected the full picture. For a lawyer against the clock, that’s worse than having no AI at all, because you’ve invested time building up a context that just evaporated.

The Irys architecture addresses this through what the team calls stateful swarms — the system persists its learnings and understanding of a matter rather than holding everything in a single ephemeral context window. Sabih showed matter folders containing a full litigation’s worth of documents, with an insights tab that maintained entity maps, key dates, court filings, and evidentiary connections across the entire corpus. The pitch is that the more you interact with the system, the better it continues to become, rather than hitting a ceiling and degrading. Dev mentioned the team has processed up to a billion tokens for a single matter.

4. Partner-Level AI vs. Associate-Level AI

The Event — Sabih drew a line between two tiers of legal AI capability that resonated beyond law into any domain where AI serves professionals. Associate-level AI follows instructions and produces competent first drafts. Partner-level AI understands context, anticipates needs, and catches things the user didn’t explicitly ask about.

Why this matters — The distinction maps to a real structural gap in how most AI systems are designed. A junior associate given a task will execute that task. A senior partner given the same task will first evaluate whether the task is the right one to be doing, check it against the broader context of the matter, and flag things the assigning attorney might have missed. Sabih gave a concrete example: Irys knowing your client’s birthday from the uploaded documents and flagging it when you’re about to send an email. That sounds trivial, but it encodes a fundamentally different relationship with context — the system isn’t just retrieving information in response to queries, it’s proactively surfacing relevant facts based on its understanding of what matters in the current workflow.

The more technically interesting version of this showed up in the demo. Sabih asked Irys to draft a responsive pleading with what he described as a deliberately bad prompt — just “draft responsive pleading” with no further specification. The system recognized the litigation posture from the uploaded documents, identified that the case was in the Southern District of New York, determined that the matter was at the answer stage rather than the motion stage, and produced a responsive pleading formatted accordingly. When they ran the same query through incumbent systems, those systems sometimes produced a motion instead — a fundamentally different document type that reflects a misunderstanding of where the case sits in its lifecycle. The difference between getting the document type right and getting it wrong isn’t a quality gradient. It’s a categorical error that reveals whether the system actually understands the legal context or is just pattern-matching on the prompt.

5. Targeted Surgery vs. Full Rewrites

The Event — Dev flagged a cost and trust problem that anyone building document-centric AI should think hard about: most systems, when asked to modify a specific section of a long document, regenerate the entire document with the changes applied. This is expensive, slow, and — critically for trust — introduces the possibility that unchanged sections got silently modified in the rewrite.

Why this reframes everything — The drafting assistant Sabih demonstrated takes a different approach. Instead of regenerating from scratch, it operates on the specific highlighted text. He selected a passage, asked the system to simplify it, and got back a redlined version showing exactly what changed with tracked changes that could be accepted or rejected individually. That workflow mirrors how lawyers actually work — they don’t rewrite entire briefs when they need to adjust a paragraph. They edit in place, with changes tracked. This is architecturally harder than full regeneration. It requires the system to maintain a stable representation of the document, identify the boundaries of the requested change, produce a targeted edit, and present the diff in a format the user can review. But it’s also what makes the system trustworthy for professional use, because the user can verify at a glance that only the intended section changed. Full regeneration, even when it produces a better overall document, erodes trust by making it impossible to know what else moved.

Dev’s broader point was that this granularity is also what makes the system cheaper to run — you’re not burning tokens reprocessing an entire document every time someone wants to adjust a clause.

6. Personalization Runs Three Layers Deep

The Event — Sabih walked through Irys’s personalization architecture, which operates at three concentric levels: the legal profession as a whole, the individual user, and the firm or organization.

Why this matters — Most AI personalization is a single-layer feature — you set some preferences, the system tries to match them. Irys treats it as a layered system where each level compounds on the ones below it. The base layer is the legal profession itself. Before a user ever logs in, the system already understands legal formatting conventions, document types, jurisdictional requirements, and procedural norms. The team described running thousands of inferences daily to refine this layer based on how lawyers actually use the platform.

The second layer is the individual. Users can set their risk tolerance, formality level, detail preferences, and drafting style. Or they can skip all of that and just upload a few of their best documents — the system builds a writing profile from the examples. Sabih said clients have told him the output “sounds scary similar” to their own writing. The system also handles multilingual input, with users switching between Spanish, English, and Arabic mid-sentence and the system maintaining coherence across the transitions.

The third layer is the firm. Workspaces pool institutional knowledge — document libraries, matter folders, organizational norms — and make it available across the team. This is where the product starts to feel less like a chat tool and more like a firm-level knowledge infrastructure that individual lawyers interface with. The compounding nature of these three layers is what Sabih pointed to when he said users spend hours per day in the platform and don’t want to leave. The system becomes more useful as it accumulates context at every level, which creates genuine switching costs that have nothing to do with lock-in tactics and everything to do with compounding institutional memory.

7. The Organic Expansion Signal

The Event — Sabih revealed something the team hadn’t planned to discuss: non-legal teams at client organizations have been pulling themselves onto the Irys platform without being sold to. One large organization that originally onboarded its legal department saw finance and accounting teams start using the platform independently, expanding from 37 seats to 77.

Why this matters for builders — Organic cross-departmental expansion is one of the strongest signals a platform can generate because it means the product is solving a problem that users recognize in their own workflows without being told the product can solve it. Iqidis positions as a legal AI company, but the underlying capabilities — long-context reasoning, document-grounded outputs, persistent memory, matter-level knowledge management — don’t have anything intrinsically legal about them. They’re general infrastructure for high-context, high-regulation professional work.

Christian made the point explicitly during the stream: this isn’t legal-specific. The architecture addresses problems that exist wherever professionals work with large document sets under accuracy constraints. Dev has talked publicly about investor pressure to position Iqidis as general-purpose reasoning infrastructure rather than a vertical legal play, and this organic expansion pattern is evidence for that thesis showing up in production rather than in a pitch deck. Whether Iqidis leans into that expansion or stays disciplined on legal as the beachhead market is a strategic question the team is still navigating, but the signal itself matters for anyone thinking about where vertical AI products hit their ceiling and what it looks like when the underlying technology outgrows its initial positioning.

Subscribe to support AI Made Simple and help us deliver more quality information to you-

Flexible pricing available—pay what matches your budget here.

Thank you for being here, and I hope you have a wonderful day.

Dev <3

If you liked this article and wish to share it, please refer to the following guidelines.

That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, my links will be at the end of this email/post. And if you found value in this write-up, I would appreciate you sharing it with more people. It is word-of-mouth referrals like yours that help me grow. The best way to share testimonials is to share articles and tag me in your post so I can see/share it.