AI Strategy: Pilot to Production

From Pilot to Production in a Regulated Environment: What Changes

Move regulated AI pilots to production in healthcare, banking, and insurance with SR 26-2, EU AI Act compliance, HIPAA, and audit trails.

Fwdpod8 min read
From Pilot to Production in a Regulated Environment: What Changes — AI Strategy: Pilot to Production

In a regulated AI pilot to production, the engineering steps stay the same, but the evidence bar rises. Before go-live you must show independent validation, written documentation, defined human oversight, controlled change, a complete audit trail and vetted vendors, mapped to the rules your sector actually imposes. The model rarely blocks the launch. Missing evidence does.

This guide is part of our AI strategy: pilot to production series. It assumes you already know the generic stage gates (exit criteria, evals, LLMOps monitoring, handover), which are covered in AI POC to production: the gap nobody budgets for. Here we cover only what changes in healthcare, financial services and insurance, with the rules checked as of October 2026. This is general information, not legal advice; involve your counsel and compliance team for your specific obligations.

What is actually different in a regulated sector?

Three things change: someone outside the build team must sign off and needs artifacts, not demos; every later change becomes a controlled event with its own record; and your vendors' obligations become yours to evidence.

WorkstreamTypical unregulated go-liveRegulated go-live
ValidationBuild team runs evals and agrees a quality barValidation is performed or challenged by people independent of the builders, before first use
DocumentationArchitecture notes and a runbookIntended use, limitations, data lineage, test results and accepted risks, written for an examiner or auditor
Human oversightUsers can flag bad outputsNamed reviewers with authority to override or stop the system, and training on over-reliance
Change controlPrompt and model changes ship after regression tests passMaterial changes need re-validation and approval; some must be planned in advance with the regulator
Audit trailTraces kept for debuggingActivity on regulated data recorded and reviewable for years, linked to model and prompt versions
Vendor riskProcurement checks terms and priceContracts, due diligence and ongoing monitoring of every party that touches regulated data or models

Which rules apply, and what are the current dates?

Several frameworks changed in 2026. Here is where they stand.

Banking: SR 11-7 has been replaced by SR 26-2

On 17 April 2026 the Federal Reserve, OCC and FDIC issued revised model risk management guidance. The Fed's letter, SR 26-2, "supersedes and replaces" SR 11-7, the 2011 guidance most bank AI teams still cite. It is risk-based, tailored to the bank's model risk profile, and expected to be most relevant to banking organizations with over $30 billion in total assets.

It keeps effective challenge by people with sufficient independence to maintain objectivity, validation that generally occurs before a model's first use, ongoing monitoring and model inventories. A footnote states that generative AI and agentic AI models are "not within the scope of this guidance," but says a bank's own risk management and governance practices should still guide the controls it sets for such tools. Expect your model risk team to apply the same discipline to an LLM application anyway.

Vendor models fall under the same principles, and outside providers are also managed under the 2023 Interagency Guidance on Third-Party Relationships (Fed SR 23-4), which spans the whole relationship life cycle.

Insurance: written AI programs and the EU high-risk list

The NAIC adopted its Model Bulletin on the Use of Artificial Intelligence Systems by Insurers on 4 December 2023. It expects insurers to maintain a written AIS Program, and regulators may ask for third-party documentation on validation, testing and auditing. Adoption is state by state: the NAIC's adoption map listed 29 jurisdictions, including the District of Columbia and Puerto Rico, as having adopted it as of 8 October 2026, so check whether each state in your footprint is one of them.

Healthcare: HIPAA, and FDA if it is a device

Under HIPAA, a vendor that creates, receives, maintains or transmits ePHI for you is a business associate and needs a business associate agreement (BAA), even, per HHS cloud guidance, if it holds only encrypted data without the key. The Security Rule's audit controls standard (45 CFR 164.312(b)) requires mechanisms that record and examine activity in systems holding ePHI. HHS proposed a major Security Rule update in January 2025; the 2026 regulatory agenda projects final action in July 2027, so build to the current rule and watch for the final text.

If the AI is a medical device, FDA's December 2024 final guidance on Predetermined Change Control Plans lets manufacturers get planned model updates authorized up front. A PCCP has three parts: a description of the modifications, a modification protocol (data management, retraining, performance evaluation and update procedures), and an impact assessment.

The EU AI Act after the Digital Omnibus

Regulation (EU) 2026/1744, the Digital Omnibus on AI, was published in the Official Journal on 24 July 2026 and entered into force on 27 July 2026. It moved the main high-risk obligations:

CategoryOriginal dateDate after Regulation (EU) 2026/1744
Annex III high-risk systems (stand-alone, e.g. credit scoring, life and health insurance risk assessment and pricing, emergency patient triage)2 August 20262 December 2027
Annex I high-risk systems (AI in regulated products such as medical devices)2 August 20272 August 2028
Article 50 transparency obligations2 August 2026Unchanged: applied from 2 August 2026, with a grace period to 2 December 2026 for Article 50(2) on systems already on the market

The delay is a planning window, not a pause. High-risk systems still need risk management, technical documentation, automatic logging and human oversight. Under Article 14, overseers must be able to stay aware of over-reliance, decide not to use the system, override its output, and interrupt it through a "stop" button or similar procedure. Those controls are expensive to retrofit.

How do the six workstreams change in practice?

Validation. Build the validation set with the people who carry the risk: clinicians, credit officers, claims leads. Have someone outside the build team challenge the method and record the answers. With a hosted LLM, validate behavior on representative and adversarial inputs, plus the retrieval and tool layers you control.

Documentation. Record intended and excluded uses, data lineage, known limitations, test results and every accepted risk with its owner. Write for a reader who was not in the room.

Human oversight. Decide what the system may only draft and what it may never decide. Give reviewers the context to disagree and measure how often they do; an override rate near zero can mean they have stopped reading.

Change control. Pin model versions and classify changes in advance: a prompt typo fix is minor; a new model, data source or decision threshold is material and goes back through validation. Our guide to building a production AI agent covers the regression mechanics; the regulated addition is the approval record.

Audit trail. Log who asked what, what was retrieved, which model and prompt version answered, and what the reviewer did. Store logs under the same rules as the regulated data they contain. The design patterns are in our planned post on LLM audit trail architecture.

Vendor risk. List every party in the data path, from model API to development partner. Each needs the contract terms your sector requires and evidence an examiner can see. The 12 data security controls to demand from an offshore AI team are a practical starting checklist.

A pattern walk-through: a hypothetical health plan pilot

This is an illustrative composite, not a client engagement or fwdpod case study.

A US health plan pilots an LLM assistant that drafts summaries of prior-authorization requests for clinical reviewers. The pilot used de-identified records and a handful of reviewers, who liked it. The production plan had to add:

  1. Data and contracts. Real ePHI means the model provider, hosting platform and logging vendor each need a BAA. One observability tool would not sign, so ePHI traces moved in-house.
  2. Decision boundary. Compliance ruled that the assistant drafts summaries only. It never recommends approval or denial, and the reviewer's decision is recorded separately from the draft.
  3. Validation. A clinical lead outside the build team graded a stratified sample. Two error types (missed medication history, wrong dates) became blocking tests.
  4. Audit trail. Every summary is stored with the source documents retrieved, the model and prompt version, the reviewer and the time.
  5. Change control. A model upgrade is treated as a material change: rerun the validation sample and get clinical sign-off before release.
  6. Oversight metrics. The plan tracks how often reviewers edit drafts. A sharp drop triggers a spot check rather than celebration.

None of these steps needed a better model. All of them needed engineering time the pilot plan had not counted.

What should a healthcare AI engineering team bring?

A healthcare AI engineering team needs people who produce evidence as well as code: someone who designs logging and access control from day one, someone who runs validation with domain experts, and a lead who can hold their own in a compliance review. Treat any claim of a "HIPAA certified" team carefully. HHS states that it does not endorse or recognize private organizations' certifications regarding the Security Rule. Ask instead what a HIPAA compliant AI development team will sign, which safeguards it operates, and how it evidences them. Our healthcare AI partner checklist turns that into questions you can send. For sector-specific obligations, see our planned HIPAA compliance page.

How fwdpod approaches regulated AI pilot to production

Fwdpod builds dedicated AI engineering pods for startups and enterprises, covering LLM development, RAG systems, AI agents and product delivery, with engineering delivery from India for buyers in the US, UK, EU and Gulf. Each pod is 4–6 experts working toward one measurable outcome, and Compliance AI is one of the pod templates. Our position is that the validation plan, oversight design and audit trail belong in the pilot scope, not the production backlog. Questions about BAAs, regulated-sector controls and the evidence available to buyers are ones the team can walk you through on a first call.

If your pilot is approved and the compliance path is not, configure your AI engineering pod: answer five questions and get a custom proposal within 48 hours. For anything else, including pricing, talk to the team.

Frequently Asked Questions

1. Is SR 11-7 still the US model risk management standard?

No. On 17 April 2026 the Federal Reserve, OCC and FDIC issued revised model risk management guidance, and the Fed's SR 26-2 letter supersedes and replaces SR 11-7. Many of its principles, such as effective challenge, validation before first use and ongoing monitoring, carry over, but bank AI teams should cite the 2026 guidance.

2. Does the revised US model risk guidance cover large language models?

Not directly. A footnote in the 2026 guidance says generative AI and agentic AI models are not within its scope, but that a bank's own risk management and governance practices should still guide the governance and controls applied to such tools. Expect model risk teams to apply similar validation and documentation expectations to LLM applications.

3. When do EU AI Act high-risk obligations apply now?

Under Regulation (EU) 2026/1744, which entered into force on 27 July 2026, obligations for Annex III high-risk systems apply from 2 December 2027 and for Annex I systems, such as AI in medical devices, from 2 August 2028. Article 50 transparency obligations have applied since 2 August 2026.

4. Do we need a BAA with our LLM provider?

If the provider creates, receives, maintains or transmits ePHI on your behalf, it is a business associate under HIPAA and you need a business associate agreement. HHS guidance says this applies even when a cloud provider holds only encrypted data without the key. Confirm the specifics with counsel.

5. Can a regulated pilot use real patient or customer data?

Sometimes, but it raises the bar immediately: contracts, access controls and logging must already be in place. Many teams pilot on de-identified or synthetic data first and add regulated data once the production controls exist, accepting that results may shift when real data arrives.

6. What counts as a material change after go-live?

Define it before launch. A new model or model version, a new data source, a changed decision threshold or a new user population usually counts as material and should trigger re-validation and approval. Cosmetic prompt edits usually do not, but they should still pass regression tests and be logged.

7. Does the EU AI Act delay mean we can wait until 2027?

It means you have more time, not less work. Retrofitting logging, human oversight controls and technical documentation into a live system is harder than designing them in, and sector rules such as HIPAA and model risk guidance already apply today.

Planning an AI deployment?

Fwdpod places pre-vetted AI engineers from India inside your team, as one specialist or a full AI pod. Book a call about your roadmap, or browse the pods we deploy most.

View all →
Back to all insights

Tell us the role or the outcome.

We'll send matched profiles or a pod plan within 48 hours.

Book a 30-minute call