Patent Quality Can't Be Measured From Metadata

By Contist AI

Among 299 patents that received a final validity decision in U.S. litigation between 1989 and 1996, roughly 46% were held invalid.^[1]1John R. Allison & Mark A. Lemley, Empirical Evidence on the Validity of Litigated Patents, 26 AIPLA Q.J. 185 (1998). The study analyzed 299 patents that received a final validity decision in litigation between 1989 and 1996; roughly 46% were held invalid. A quarter-century of PTAB practice has hardened rather than softened that picture. In FY25, of the roughly 26% of PTAB petitions that reached a final written decision, about two-thirds held all challenged claims unpatentable, and on a per-claim basis the invalidation rate in IPRs reaching a final written decision has been running near 78%.^[2]2USPTO Patent Trial and Appeal Board, Trial Statistics — FY25 Year-End Outcome Roundup (Sept. 30, 2025). Available at uspto.gov/patents/ptab/statistics. Multi-year commentary reports per-claim invalidation rates in IPRs reaching final written decision rising from roughly 70% (FY19) to roughly 78% (FY24). See IPWatchdog, "The PTAB's 70% All-Claims Invalidation Rate Continues to Be a Source of Concern" (Jan. 12, 2025). Passing examination and surviving adversarial scrutiny are not the same test, and the gap between them is not narrow.

Portfolio tools score patent "quality" anyway. They score it using whatever a database can compute cheaply from bibliographic records — claim count, family size, forward citations, prosecution speed. Every one of those signals is upstream of the question quality is supposed to predict, which is whether the claims survive when someone with money at stake tries to invalidate them. Every one is derivable without opening a single file wrapper. That is why they are used. It is also why they do not answer the question.

Real quality assessment requires claim-level, prosecution-aware signals — the shape of how the claims moved through examination, what was argued to overcome which rejection, how deeply the specification supports the issued scope, how correlated the prior-art exposure is across the portfolio. Those signals only exist in the prosecution record. They do not exist in the bibliographic one.

"Quality" usually means whatever is easiest to compute

Ask a portfolio dashboard for a quality score and, under the hood, you will almost always find some weighted combination of four inputs:

• Claim count. A single field on the front page. Correlates loosely with drafting effort. Says nothing about scope, support, or survivability.
• Family size. The number of related applications filed across jurisdictions. Long used as a proxy for perceived value on the theory that maintenance and translation costs across multiple offices reveal how much the applicant thinks the invention is worth.^[3]3Dietmar Harhoff, F. M. Scherer & Katrin Vopel, Citations, Family Size, Opposition and the Value of Patent Rights, 32 Rsch. Pol'y 1343 (2003). Family size — the number of related applications filed across jurisdictions — has been widely used as a proxy for patent value on the theory that maintenance and prosecution costs across multiple offices reveal how much the applicant thinks the invention is worth.
• Forward citations. How many subsequent patents cite this one — the workhorse metric in innovation economics since Trajtenberg introduced citation-weighted patent counts as a value indicator in 1990.^[4]4Manuel Trajtenberg, A Penny for Your Quotes: Patent Citations and the Value of Innovations, 21 RAND J. Econ. 172 (1990). The origin of citation-weighted patent counts as a value indicator, using CT-scanner data to link forward citations to willingness-to-pay estimates.
• Prosecution speed and office-action count. Fewer rejections, faster allowance — read as a signal that the claims were clean on filing.

These metrics have earned their place because they were tractable when they were invented. Bibliographic data is what portfolio databases had. Now that structured inference over the entire file wrapper is technically feasible, continuing to score quality from bibliographic proxies is a design choice, not a technical necessity.

The problem is not that these signals are noise. The problem is that they measure something adjacent to what quality is supposed to predict. A patent can have a lot of forward citations because the field is crowded. It can have a large family because the applicant's filing budget was healthy in 2018. It can have a short prosecution because the art unit is generous and the examiner had a light day. None of that constrains how the claims behave when they meet a serious §103 challenge in an IPR or a Phillips-standard construction dispute in district court.

Citations are the weakest of the four, and treated as the strongest

Forward citations carry more weight in portfolio scoring than any other single signal. They also carry the most baggage.

Trajtenberg's 1990 CT-scanner paper made the original case: patents cited more often were, on average, associated with higher-value innovations. That finding is real, and it launched most of the innovation-economics literature that followed. But two later findings undermine the "more citations = higher quality" mapping that portfolio tools import wholesale.

First: most citations on the average U.S. patent are inserted by the examiner, not the applicant. Alcácer and Gittelman showed in 2006 that examiner-added citations account for roughly two-thirds of the citations on the average U.S. patent, and that about 40% of patents have citations that are entirely examiner-added.^[5]5Juan Alcácer & Michelle Gittelman, Patent Citations as a Measure of Knowledge Flows: The Influence of Examiner Citations, 88 Rev. Econ. & Stat. 774 (2006). Approximately two-thirds of citations on the average U.S. patent are inserted by the examiner rather than the applicant, and about 40% of patents have citations that are entirely examiner-added. Examiner citations are added during examination to identify prior art the examiner found relevant — they are, in effect, a signal about the examiner's search, not about how much subsequent inventive work built on the citing patent. Citation counts on a mature patent are therefore a blended signal of two very different things: applicant-community influence and examiner-search behavior. The blend is different in every art unit.

Second: the citation-to-value relationship is not monotonic. Abrams, Akcigit, and Popadak, using tens of thousands of NPE-owned patents with observable licensing revenues, document an inverted-U between forward citations and realized patent value.^[6]6David S. Abrams, Ufuk Akcigit & Jillian Popadak, Patent Value and Citations: Creative Destruction or Strategic Disruption?, NBER Working Paper No. 19647 (2013, rev.). Using NPE-owned patents with observable licensing revenues, the paper documents an inverted-U relationship between forward citations and realized patent value. The most-cited patents are not, on average, the most valuable ones; mid-citation-count patents disproportionately include strategic filings whose citation rates reflect the density of the litigation-adjacent art around them rather than technical importance. That finding is directly at odds with the linear "more citations = more valuable = higher quality" step that citation-based portfolio scores implicitly take.

Neither result invalidates citations as an input. Both make it hard to defend a single-number quality score with citations doing most of the work.

What quality is supposed to predict

Strip away the bibliographic proxies and the question underneath is the one Allison and Lemley asked in 1998: when this patent is tested, does it hold? Their answer, from 299 patents with final validity decisions between 1989 and 1996, was that about 46% did not.^[1]1John R. Allison & Mark A. Lemley, Empirical Evidence on the Validity of Litigated Patents, 26 AIPLA Q.J. 185 (1998). The study analyzed 299 patents that received a final validity decision in litigation between 1989 and 1996; roughly 46% were held invalid. PTAB practice has sharpened the picture.

In FY25, of the 1,470 PTAB petitions that reached an outcome, roughly 26% reached a final written decision. Of those, only about 4% of total petition outcomes were "all claims patentable"; the modal outcome was all challenged claims held unpatentable, and per-claim invalidation among decided IPRs has been running near 78%.^[7]7USPTO PTAB Trial Statistics FY25 report, supra note 2. Approximately 26% of petitions reaching an outcome between Oct. 1, 2024 and Sept. 30, 2025 reached a final written decision; among those, roughly two-thirds held all challenged claims unpatentable and only about 4% of total petition outcomes were fully "all claims patentable." Institution denials — many under Fintiv or §325(d) — account for another 44% of outcomes, but those cases turn on discretionary and record-based factors that are themselves prosecution-history-driven.

There is a selection effect here — patents that reach litigation or a final written decision are not a random sample of issued patents. But that cuts the other way from portfolio scoring. Challengers pick the assertions that look weakest to attack. If a portfolio-quality score is supposed to help an assertee, an acquirer, or a defendant predict where the portfolio is fragile, the relevant survival rate is precisely the rate among adversarially-selected patents.

The USPTO itself has tried to close some of this gap on the examination side. The Enhanced Patent Quality Initiative launched in 2015 introduced the Master Review Form to score office actions for both correctness and clarity across every stage of prosecution.^[8]8USPTO, Enhanced Patent Quality Initiative (launched Feb. 5, 2015). Introduced the Master Review Form, a standardized instrument for scoring office actions on both correctness and clarity across every stage of prosecution. See uspto.gov/patents/quality-metrics. That effort is worth doing on its own terms. But it measures examination-time quality — was the rejection well-formed, was the reasoning clear — and stops before the question of whether the issued claims later survive a §103 or §112 challenge on someone else's evidentiary record. Examination-time quality and post-grant survival are correlated. They are not the same thing.

Passing examination is a low bar. The bibliographic proxies sit one level upstream of that low bar. The signal an IP team, an acquirer, or a litigator actually needs is one level downstream.

What a real quality system looks at

If quality means "how likely are these claims to survive when tested," then the signals worth measuring are the ones that describe how the claims got where they are — not what the front page of the issued patent looks like. Four are load-bearing, and all four require the file wrapper.

1Scope trajectory across the prosecution

Every narrowing amendment is a data point about how much room the applicant had to give up before the examiner would allow. A claim that went out the door substantially as filed — one round of §102/§103, a clean argument, allowance — is a fundamentally different asset from a claim that survived six office actions of narrowing amendments to overcome prior art the examiner kept adding. The former carries broad scope with a thin prosecution history to constrain construction. The latter carries narrower scope and a thick record of surrendered equivalents under Festo. Both facts are downstream signals about how a Phillips-standard construction is likely to run at the PTAB or on remand. Neither is visible from claim count, family size, or prosecution-speed metrics. Scope trajectory is measurable directly from the amendment sequence in the file wrapper; nowhere else.

2Argued around versus amended around

For every §103 rejection, the applicant chose one of two paths: argue that the reference does not disclose the element, or amend the claim to distinguish it. Those two responses have very different downstream consequences. Arguments made to distinguish prior art create prosecution disclaimer, binding the patentee to a narrower construction at trial and at the PTAB under Phillips. Amendments made for reasons related to patentability create Festo estoppel on the amended element, subject to the three rebuttal grounds Festo specifies. Both narrow the claim; they narrow it in different ways, along different vectors, and against different challenges. A quality score that treats "the claim was amended" and "the claim was argued" as the same event throws away the signal that most predicts how the claim behaves under adversarial challenge.

3Specification support depth for the issued scope

§112 is where a large share of PTAB and district-court invalidity findings actually land — written description, enablement, and, post-Nautilus, definiteness. The question is not "was there a §112 rejection during prosecution" — plenty of strong patents cleared §112 with an unmemorable amendment. The question is whether the specification supports the scope of the issued claims: the density of embodiments, the treatment of the amended terms in the description, whether narrowing amendments walked the claim toward or away from the disclosure's center of gravity. That analysis requires reading the specification alongside the final claim set with the amendment history in mind. It is not derivable from any bibliographic field.

4Portfolio-level correlated risk across shared prior art

At the portfolio level, a per-patent quality score misses the risk that matters most: correlated invalidation. When multiple patents in a family — or across families — share the same primary references, an IPR that establishes those references' teaching against one claim propagates risk across every claim in the portfolio that reads on the same disclosure. That exposure is only visible if the prior-art graph is built at the claim-element level across the whole portfolio, not per-patent from citation counts. A portfolio scoring 90/100 on citation-weighted metrics can still concentrate all of its assertion value in claims that share a single §325(d)-eligible reference. That is a portfolio one well-briefed petition away from a bad quarter.

Contist computes these four signals directly from the prosecution record and composes them into a per-matter quality score with per-feature reasoning attached, so an attorney can see which prosecution facts moved the number. Weighting is anchored to real outcome data — PTAB Final Written Decision labels for post-grant survival, and litigation-outcome data for assertability.

A note on where we are

Contist is built around a specific version of this thesis: patent-portfolio decisions — assertion, acquisition, defense, cross-license — are worth more when they are made on claim-level, prosecution-aware signals than on bibliographic proxies dressed as quality scores. The architecture follows from where the signal actually lives. The product follows from the architecture.

Contist is built for in-house IP teams, patent litigators, PTAB counsel, and IP-aware investors making these portfolio decisions today. If any of the argument above resonates — or if you think we are wrong about something specific — we would like to hear from you.

Contist AI is building a patent prosecution platform focused on structured prosecution history, claim-level infringement analysis, and portfolio analytics. The founder previously worked on document intelligence.

Reach Contist at sales@contist.ai · Follow on LinkedIn.