Benchmark: clause classification across 40,000 executed agreements

The problem

The short version: clause classification across 40,000 executed agreements is easier to get wrong than most teams assume, and the failure is usually silent.

How we measured it

Our approach is to index the document twice: once by structure, once by defined term. Findings then carry a pointer back to both, so a reviewer can confirm a conclusion without re-reading the agreement.

In the current build this runs as part of the standard pass, so it applies to every document in the set rather than only the ones someone thought to check.

Where it breaks

Scanned originals with poor image quality remain the weakest input. So do agreements that were assembled from three precedents and never reconciled, which is common in long-lived supplier relationships.