A general contractor sends your insured a 60-page subcontract. Page 41, buried in Exhibit C, says the sub must carry $2M per occurrence general liability, additional insured on a primary and non-contributory basis for ongoing and completed operations, a waiver of subrogation in favor of the owner and the lender, and 30 days' notice of cancellation. Nobody in the agency reads page 41. Six weeks later a certificate request arrives from the GC's tracking portal, your CSR issues an ACORD 25 off the policy as written, and the GC rejects it. Now you are remarketing an endorsement under deadline pressure while the sub cannot get on site.
The failure is one of sequence. The agency finds out what the contract demands at the moment it has the least time to do anything about it. An agent can move that discovery earlier: read the contract when it lands, turn the insurance article into structured requirements, compare them against the policy as issued, and put a gap list in front of a licensed person while the policy can still be endorsed.
This tutorial builds that pipeline. It assumes Python, an AWS account with Textract and Bedrock access, and read access to your AMS document store. It drafts; it never sends and never binds.
Scope: the insurance article, not the whole contract
Do not ask a model to read a 60-page subcontract and tell you about insurance. You will pay for 60 pages of tokens and get an answer you cannot trace to a page. Commercial contracts put insurance obligations in a small number of predictable places: an article titled "Insurance", an indemnity article next to it, and one or more exhibits that restate limits in a table.
Find those pages first with cheap text matching, then spend model tokens only there.
import re
import boto3
from pypdf import PdfReader
textract = boto3.client("textract")
TRIGGERS = re.compile(
r"\b(insurance|additional insured|waiver of subrogation|"
r"certificate of insurance|primary and non-?contributory|"
r"per occurrence|umbrella|excess liability|indemnif)",
re.I,
)
def candidate_pages(pdf_path: str) -> list[int]:
reader = PdfReader(pdf_path)
hits = []
for i, page in enumerate(reader.pages):
text = page.extract_text() or ""
if len(TRIGGERS.findall(text)) >= 2:
hits.append(i)
# include one page either side; insurance articles run over page breaks
spread = {p + d for p in hits for d in (-1, 0, 1)}
return sorted(p for p in spread if 0 <= p < len(reader.pages))
If extract_text returns nothing the contract is a scan, which is common when the GC's project manager prints and re-scans the signed version. Fall back to Textract DetectDocumentText for those pages and run the same regex over the returned lines. Keep the page numbers either way: every requirement you extract has to carry the page it came from, or a CSR cannot check your work.
Extract requirements into a schema, one coverage at a time
The output of this step is not prose. It is a row per coverage line with a limit, a basis, and the endorsement obligations attached to it. Define that shape before you prompt, and make the model fill it.
REQUIREMENT_SCHEMA = {
"type": "object",
"properties": {
"requirements": {
"type": "array",
"items": {
"type": "object",
"properties": {
"coverage": {
"type": "string",
"enum": [
"general_liability", "auto_liability", "workers_comp",
"employers_liability", "umbrella_excess",
"professional_liability", "pollution", "builders_risk",
"crime", "cyber", "other",
],
},
"limit_amount": {"type": ["integer", "null"]},
"limit_basis": {
"type": ["string", "null"],
"enum": ["per_occurrence", "aggregate", "per_claim",
"per_accident", "per_project", None],
},
"additional_insured": {"type": "boolean"},
"ai_scope": {
"type": ["string", "null"],
"enum": ["ongoing", "completed", "ongoing_and_completed", None],
},
"primary_noncontributory": {"type": "boolean"},
"waiver_of_subrogation": {"type": "boolean"},
"named_parties": {"type": "array", "items": {"type": "string"}},
"notice_of_cancellation_days": {"type": ["integer", "null"]},
"verbatim": {"type": "string"},
"page": {"type": "integer"},
"confidence": {"type": "number"},
},
"required": ["coverage", "verbatim", "page", "confidence"],
"additionalProperties": False,
},
}
},
"required": ["requirements"],
"additionalProperties": False,
}
Two fields do most of the work later. verbatim is the sentence the requirement came from, copied without paraphrase; it is what the CSR reads when they disagree with the agent. named_parties is the list of entities that must appear as additional insured or waiver beneficiary — owner, lender, construction manager, their "officers, directors and employees" — because that string is what ends up in the Description of Operations box on the ACORD 25.
Prompt against the candidate pages only, one call per page group, and constrain the output to the schema. If your runtime does not support a schema parameter, validate with jsonschema and retry once on failure rather than accepting a near-miss. Discard anything where verbatim does not appear in the source text for that page: that check alone kills most invented requirements.
import json, jsonschema
def extract(page_text: str, page_no: int) -> list[dict]:
body = {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 2000,
"temperature": 0,
"system": (
"Extract insurance requirements imposed on the subcontractor. "
"Copy the governing sentence into verbatim, unchanged. "
"Omit anything the text does not state. Do not infer standard practice."
),
"messages": [{"role": "user", "content": json.dumps(
{"page": page_no, "text": page_text, "schema": REQUIREMENT_SCHEMA}
)}],
}
resp = bedrock.invoke_model(modelId=MODEL_ID, body=json.dumps(body))
out = json.loads(json.loads(resp["body"].read())["content"][0]["text"])
jsonschema.validate(out, REQUIREMENT_SCHEMA)
return [r for r in out["requirements"] if r["verbatim"][:40].lower() in page_text.lower()]
A subcontract will usually yield six to twelve requirement rows. Flow-down clauses ("Subcontractor shall carry the insurance required of Contractor under the Prime Agreement") are a known miss: the obligation lives in a document you do not have. Emit those as a row with coverage: other and a confidence of zero so they surface as questions, not as silence.
Compare against the policy as issued
Now join the requirements to what the agency actually placed. Limits come from the policy record in the AMS. Endorsement obligations do not: additional insured status, primary and non-contributory wording, and waivers live in form numbers on the policy — CG 20 10, CG 20 37, CG 20 01, WC 00 03 13 and their carrier-specific equivalents — and those are in the document store, not in a tidy field.
If you have already indexed endorsement forms for certificate checking, reuse that index here. The comparison is a small amount of deterministic code per requirement, and it should be code rather than a model call, because a CSR will ask why a line was marked a gap.
def compare(req: dict, policy: dict, forms: list[str]) -> dict:
issues = []
if req.get("limit_amount") and policy.get("occurrence_limit"):
if policy["occurrence_limit"] < req["limit_amount"]:
issues.append(
f"Contract requires {req['limit_amount']:,}; policy shows "
f"{policy['occurrence_limit']:,}."
)
if req.get("additional_insured") and not any(f in forms for f in AI_FORMS):
issues.append("No additional insured endorsement on the policy.")
if req.get("ai_scope") == "ongoing_and_completed" and "CG 20 37" not in forms:
issues.append("Completed operations AI required; CG 20 37 not found.")
if req.get("waiver_of_subrogation") and not any(f in forms for f in WAIVER_FORMS):
issues.append("Waiver of subrogation required; no waiver endorsement found.")
if req.get("notice_of_cancellation_days"):
issues.append(
f"Contract asks for {req['notice_of_cancellation_days']} days' notice. "
"Carrier notice terms are policy-driven; confirm with the carrier."
)
status = "gap" if issues else "satisfied"
if policy.get("occurrence_limit") is None and req.get("limit_amount"):
status, issues = "cannot_determine", ["Policy limit not populated in the AMS."]
return {**req, "status": status, "issues": issues}
Keep cannot_determine as a first-class outcome. An agent that answers "satisfied" because a field was blank is worse than one that answers nothing, and the three-state result is the difference between a tool a CSR trusts and one they stop opening. Notice of cancellation is deliberately never marked satisfied: carriers rarely issue the 30-day notice a contract asks for, and that conversation belongs to a licensed person.
Land the result in the AMS and stop
Write one activity on the account — an Epic activity, a HawkSoft log note, an EZLynx task — with the contract name, the counterparty, the effective date, and a table of requirement, status, policy evidence, and page citation. Attach the extracted JSON. Make the activity idempotent on a key of account plus contract document ID plus a hash of the requirement set, so re-running the pipeline after a policy change updates one activity instead of breeding duplicates.
What the agent does not do: it does not order endorsements, email the carrier, email the GC, or issue a certificate. The gap list is a draft for a licensed person, who decides whether to request CG 20 37, negotiate the clause, or tell the insured the contract is unsignable as written. Every one of those is a judgment with E&O consequences.
One thing worth building on top: when a certificate request for that holder arrives later, the COI agent reads the stored requirement rows instead of guessing from the holder's name. The Description of Operations box then comes from the contract language, with a citation, which is the whole point of doing this six weeks early.
Measure it before anyone relies on it
Run the pipeline silently over 40 contracts already in the document store — ones where you know how the certificate dispute ended. Score three things, per requirement row rather than per contract:
- Recall on requirements. Of the obligations a senior account manager finds by reading the contract, what fraction did the agent emit? Missed requirements are the only failure that reaches the GC.
- Precision on gaps. Of the rows marked
gap, how many were real? False gaps burn CSR time and teach people to skim the list. - Citation accuracy. Does
verbatimappear on the cited page, every time? This one should be at 100%, because it is enforced in code, and any miss means the guard above is broken.
Expect recall in the high eighties on clean digital subcontracts and materially worse on scans with handwritten riders. Publish the numbers to the CSRs before go-live, including the bad ones. A team that knows the agent misses flow-down clauses reads the list the right way; a team told it catches everything reads it once.
Our COI Agent work typically runs three to five weeks from kickoff to supervised go-live, and the contract-requirements step adds about a week on top, most of it spent on the endorsement form index rather than the extraction. If you want to talk through it, tell us your AMS and roughly how many certificate requests you handle a week.