Long read, August 9, 2026
the difference between retrieval and reasoning in practice
The problem
We keep coming back to the difference between retrieval and reasoning in practice, because the cost of a miss here compounds across a portfolio.
Volume changes the shape of the problem. At ten documents you can hold the exceptions in your head. At four hundred you need the exceptions written down and checkable.
What we do about it
We score against a human baseline drawn from executed matters rather than a synthetic set, because synthetic contracts do not contain the drafting scars that cause real misses.
In the current build this runs as part of the standard pass, so it applies to every document in the set rather than only the ones someone thought to check.
Where it breaks
Scanned originals with poor image quality remain the weakest input. So do agreements that were assembled from three precedents and never reconciled, which is common in long-lived supplier relationships.
We will revisit this once we have a larger sample. The current numbers are directional rather than settled.
Working notes from the Lawrs team. General information about legal technology and practice, not legal advice.