How Candidate Retrieval and SOP-Guided Verification Make PI Checks More Explainable

Instead of scanning every slide with a large model, separate low-cost candidate retrieval from SOP-guided verification and product-consistency checks.

Finding a prescribing information page in a presentation is not the same as searching for the phrase "prescribing information."

In a long medical slide deck, the same phrase may appear in a table of contents, training instruction, workflow diagram, or disclaimer. A QR code may point to approved PI—or to a survey, corporate account, event page, or unrelated resource. A dense text page may be genuine PI, a clinical guideline, or a general legal notice.

This creates a technical problem with two very different parts:

1. Retrieval: Which pages are plausible PI candidates?
2. Adjudication: Which candidate is actually the applicable PI page for this material?

Treating both tasks as one large-model prompt is possible in a demo. It is less attractive as an enterprise architecture.

Why keyword-only matching breaks down

Keyword matching is fast and useful, but it does not understand page role.

A rule such as title contains "prescribing information" will retrieve true PI pages, but it may also retrieve:

• a contents page listing a PI section;
• an MLR training slide explaining PI requirements;
• a workflow note telling the author to attach PI later;
• a disclaimer page that uses similar legal vocabulary;
• a screenshot of an approved material rather than the material's own PI.

The same problem appears with QR codes. A QR code is a visual feature, not proof of destination or approval status.

If keyword or QR presence becomes the final decision rule, the system produces avoidable false positives and gives reviewers little reason to trust the output.

Why scanning every page with a large model is also weak

The opposite design sends every slide, image, table, chart, and text block to a large multimodal model and asks it to determine whether the page is PI.

That approach removes some rule engineering, but introduces other operational costs:

• more tokens or image-processing workload per document;
• higher latency for long decks;
• lower and less predictable throughput under concurrency;
• greater sensitivity to prompt and model-version changes;
• less transparent failure analysis when a page is missed or misclassified.

The problem is not that large models cannot help. It is that they should spend their reasoning budget on ambiguous, high-value candidates—not on repeatedly proving that a title slide or thank-you page is not PI.

A two-stage architecture: retrieve, then verify

A more practical architecture separates low-cost search from deeper judgment.

In the first stage, a multi-feature retrieval engine scores every page using signals that are cheap to compute and easy to inspect. The engine does not decide whether a page is PI. It creates a high-recall candidate set.

Possible signal families include:

• Lexical signals: PI-related titles, section names, standard wording, indications, contraindications, warnings, adverse reactions, or dosage language.
• Position signals: location near the end of a deck, proximity to references or closing sections, and relationship to section boundaries.
• Layout signals: dense legal-text blocks, structured headings, multi-column information, small-print patterns, or approved template features.
• Visual signals: one or more QR codes, label images, product marks, or a screenshot-like PI layout.
• Object signals: text boxes, tables, images, grouped shapes, speaker notes, or embedded objects that carry evidence.
• Document-context signals: product mentions across the deck, material type, audience, market, channel, and neighboring slide roles.

The weights and thresholds should be configurable. A QR code may be a strong signal in one company's approved workflow and a weak signal in another. A final-slide position may increase candidate priority, but should never be treated as proof.

Pages above a retrieval threshold—or the top-ranked pages when no page crosses the threshold—move to the next stage.

Stage two: SOP-guided verification

The second stage asks a model to verify a small set of candidates using explicit company rules.

This is different from asking, "Does this look like PI?" The verification prompt or policy layer should define positive conditions, required combinations, and blocking conditions.

Positive evidence may include:

• an approved instruction that directs the reader to complete product information;
• a verifiable PI destination associated with the product;
• product name and dosage-form evidence;
• recognizable prescribing-information sections;
• text structure consistent with the applicable approved format.

Blocking evidence may include:

• a brand-only advertising page without prescribing-information structure;
• a generic disclaimer without product-specific PI elements;
• a treatment pathway or guideline page without applicable product information;
• a survey, social account, or event QR code;
• a screenshot or training example that is not the material's own PI.

The model's job is narrow: evaluate the candidate against the SOP, cite matched conditions, identify blockers, and explain uncertainty.

Structured output makes the result reviewable

The verification result should be structured rather than hidden in free-form model prose.

For example:

{
  "candidate_page": 42,
  "classification": "likely_pi_page",
  "matched_rules": ["approved_access_route", "product_name_present", "pi_structure_present"],
  "blocking_rules": [],
  "product_detected": "Product B",
  "confidence_band": "high",
  "evidence": [
    { "object": "qr_code", "location": "bottom_right" },
    { "text": "See complete prescribing information", "location": "footer" }
  ],
  "next_action": "compare_with_core_product_and_route_for_human_review"
}

This output is useful because a reviewer can inspect the page, rule matches, evidence objects, detected product, and proposed next step. A technical evaluator can also trace why a classification changed after a rule update.

Confidence should not substitute for evidence. A high score without inspectable reasons is still a black box.

Detection is not enough: verify product consistency

Even a correctly identified PI page can be wrong for the material.

The next layer extracts the generic name, brand name, dosage form, or product identifier from the candidate page and compares it with product evidence across the full deck.

The core product may be inferred from repeated title references, claim language, product logos, indication context, or structured metadata. Because decks can mention comparators or multiple products, the system should preserve uncertainty and show the evidence used to identify the primary product.

The consistency check can then produce one of several review states:

• likely match;
• possible mismatch;
• multiple products detected;
• product identity unresolved;
• approved exception or market-specific rule requires human confirmation.

This step moves the system from "a PI page exists" to "the PI appears connected to the right product."

Material type and channel rules complete the loop

The last technical layer connects the page result to the material's intended use.

A configurable policy profile can evaluate signals such as:

• promotional versus scientific or educational content;
• healthcare-professional, patient, or public audience;
• internal versus external distribution;
• market and language;
• approved text version versus approved QR mechanism;
• company-specific exceptions and escalation routes.

The output should be a review recommendation and evidence package, not an autonomous regulatory conclusion.

What to measure in evaluation

Enterprise evaluation should measure each stage separately. A single "accuracy" number hides where the system succeeds or fails.

Useful measures include:

• candidate retrieval recall: did the true PI page enter the candidate set?
• candidate-set size: how many pages require deeper verification?
• verification precision: how often are non-PI candidates rejected correctly?
• document-level mismatch detection: are wrong-product attachments surfaced?
• false-negative rate for missing PI scenarios;
• latency and compute cost by deck size;
• throughput under expected concurrency;
• evidence completeness: can a reviewer understand and reproduce the flag?
• rule-change stability: can the team see what changed when an SOP is updated?
• human override patterns: where do reviewers disagree with the system, and why?

Human-in-the-loop by design

The architecture is strongest when human review is not treated as an exception.

Reviewers should be able to inspect the candidate page, see matched and blocked rules, open the QR destination where policy permits, compare product evidence, override the recommendation, and record a reason. Those decisions can then support error analysis and rule maintenance.

The system helps narrow the search space and standardize routine checks. Qualified reviewers remain responsible for material classification, jurisdictional interpretation, exception handling, and the final decision before formal MLR approval.

Spend model reasoning on judgment, not search

The central design principle is simple:

Use deterministic and document-native signals to find a small number of plausible pages. Use model reasoning to verify ambiguous candidates against explicit SOP rules. Then connect the result to product identity and material context.

This staged approach improves more than efficiency. It makes failures easier to diagnose, rules easier to update, and outputs from an AI-assisted review workflow easier for reviewers to trust.

That is the difference between a model that scans a presentation and a review system designed for enterprise MLR workflows.