AI operations
Monitoring, drift detection, exception handling and monthly SLA reporting for everything we build. Every build ships with it.
What it does
A model that is not watched degrades quietly. Documents change, clients change their templates, a provider updates a model. The operations retainer exists so that none of this reaches your client unnoticed.
Model, prompt and schema versions are pinned per partner. Every change is gated on your evaluation set against the accuracy floor in the SOW. Drift signals raise an alert and feed the review queue. If a change fails, we roll back to the previous pinned version.
Each month you receive a report under your name: accuracy against the floor, exception rate against the ceiling, uptime, incidents and what changed. It is written so that you can hand it to your client.
Processes we have scoped this line for
- Accuracy monitoring against the floor in each SOW
- Drift detection on inputs, outputs and confidence distributions
- Exception handling and review-queue staffing plans
- Incident response with a severity table and a 72-hour partner notice
- Monthly SLA report by the fifth business day
Section 2 — what it does
Policy 09, AI model monitoring and drift, is the source for this page. The full text is in the security pack under NDA.
How it runs
| Step | Stage | Who | What happens |
|---|---|---|---|
| 1 | Pin | System | Model, prompt and schema versions recorded per partner environment. |
| 2 | Evaluate | Machine | Every change run against your evaluation set before it reaches production. |
| 3 | Watch | Machine | Accuracy, confidence distributions, exception rate and latency, compared with the baseline. |
| 4 | Alert | Person | Drift or a breach of the floor raises an alert to our operations engineer and to your named contact. |
| 5 | Roll back | System | The previous pinned version is restored while the cause is found. |
| 6 | Report | System | The monthly SLA report under your name, by the fifth business day. |
Severity table
| Severity | Means | Response |
|---|---|---|
| Sev 1 | Partner-wide outage or data exposure | 15 minutes (placeholder) |
| Sev 2 | One process down or accuracy below the floor | 1 hour (placeholder) |
| Sev 3 | Degraded | Next business day (placeholder) |
| Sev 4 | Cosmetic | Placeholder |
Section 3 — how it runs
Response times by severity: Sev 1 15 minutes, Sev 2 1 hour, Sev 3 next business day. Placeholders to confirm in the ops retainer schedule.
What the pilot builds
One process, two weeks, a fixed fee. Acceptance criteria are agreed before day one. Your reviewers work the queue. The report, the system and the code transfer to you.
| Process | What the pilot builds | What acceptance measures |
|---|---|---|
| Any process we have built for you | Monitoring, alerting, the exception queue and the monthly report | Accuracy against the floor, exception rate against the ceiling, response and resolution times by severity |
Section 4 — the pilot
The retainer starts at acceptance of the first build. Uptime, accuracy floor, exception ceiling and response times are placeholders until the SOW sets them.
The pilot fee is fixed and quoted in the scope. placeholder — replace with the pricing model.
What you receive
- The monthly SLA report under your name
- The runbook, the alert rules and the evaluation set
- Incident records with the timeline and the fix
- Transfer of all of it if the retainer ends
- We never contact the BPO's client.
- The BPO sets the price.
- Everything we build transfers to the BPO.
Section 5 — what you receive
Everything we build transfers to the BPO.
We do not contact your clients. Ever.