A North American labor union representing tens of thousands of licensed professionals runs 21 AI contract chatbots, each scoped to a single collective bargaining agreement.
They have answered 8,182 questions at an 87.4% answer rate over a six-month blend, with volume across the latest six months running 4.9 times the six months before. All 21 sit on one agreement with us. The union shared the deployment anonymously.
The same question has a different right answer in every unit
A member on a rest day is working out what a schedule change does to next month’s pay. A local representative is taking that same question from thirty people in a week, on an agreement he never sat at the table for. A staff officer at head office knows all of them and can answer any of them, one person at a time.
Every employer group bargains its own terms, so the answer that is correct for one member is wrong for the member sitting next to her.
This is the detail that makes unions a genuinely hard case. One assistant trained on all of the agreements would be worse than no assistant at all, because it would sound certain. The union’s problem was never a shortage of expertise. It was that the expertise lived in a few people’s heads and in documents nobody reads end to end.
One agreement per assistant, isolated from the rest
Each bargaining unit has its own assistant, grounded in that unit’s contract and nothing else. The corpus runs to 42.5 million words across 1,132 documents, held in 21 separate libraries so that no assistant can reach into terms that do not govern the member asking.
Members reach them inside the union’s own member site, behind the sign-on they already use, with page-level rules deciding which unit’s assistant a member gets. A separate destination would have been a destination nobody goes to.
Four of the 21 point inward rather than at members: HR, military leave rights, the union’s own constitution and admin manual, and the agreement that sits above the individual units. Internal assistants have no public downside, which makes them the easy place to start.
Accuracy, member trust and representative oversight
A union answering questions about its own agreements carries an obligation a publisher does not. A wrong answer about pay is a grievance.
Three requirements followed from that. The assistant cites the exact clause and knows when to escalate, so routine questions get answered and the hard ones get pointed at the clause and handed to a person. A union that answers everything will eventually be wrong in public.
Representatives had to trust it before members saw it. Each unit set an accuracy bar on its own agreement and had to clear it before the assistant was switched on. Nobody shipped on hope, and nobody has had to walk anything back.
And agreements change constantly. Amendments, side letters and new terms land all the time, so source documents sync from the union’s own document store. Without that, the whole estate goes stale inside a single bargaining round.
Short source libraries beat large ones
Every assistant is held to the few documents that actually decide the answer.
That constraint did more for accuracy than adding material would have. It is the opposite of the instinct most organizations bring to a corpus, and on contract interpretation it is the correct one.
One employer’s contract arrived in a layout the platform could not read. We fixed that on our side, nobody at the union touched a file, and every customer on the platform has that fix now.
The capability that made it possible: the wrong answer is not reachable
In a union, accuracy depends on who is asking. The same words about pay are correct for one member and a grievance for the member sitting next to her, and no amount of general intelligence resolves that, because the model has no way to know which contract governs the person typing.
Scoping each assistant to a single agreement removes the wrong answer from reach entirely. Putting the right assistant in front of the right member, using the access rules the union already runs on its own site, does the other half. That is why there are 21.
The estate today
| Measure | To Date |
| Grounded assistants in production | 21 |
| Questions asked | 8,182 |
| Questions in the busiest month | 1,991 |
| Growth, latest six months on the six before | 4.9× |
| Questions answered, six-month blend | 87.4% |
| Longest unbroken run of daily use | 71 days |
The number to read first is 4.9. Questions across the latest six months ran almost five times the six months before them, on an estate that was already live and already working. The busiest month on record is the most recent complete one.
Three quarters of questions arrive through assistants embedded in the union’s own member pages. Nobody had to be sent somewhere new.
The run of 71 straight days with at least one question is still going, and that is what it looks like once representatives stop treating something as a trial.
How the estate grew
The busiest unit went first and still carries the largest share of everything asked. Units came on one at a time, each with its own agreement and its own assistant, rather than as a single program with a launch date.
Depth arrived alongside breadth. Questions cluster hard around pay and scheduling, which is what made a second assistant over the heaviest unit’s scheduling reference material worth building before a twenty-second unit was added.
What this means for your union
Count the agreements, not the members.
If your members are covered by different agreements, a single assistant over all of them is the wrong shape at any size. Scope each to one set of terms and the wrong answer stops being reachable rather than merely unlikely.
Decide what it will not answer. This union answers the routine questions and points at the clause for the rest. That boundary is the reason its representatives trust it.
And set the bar before you switch anything on. Each unit here had to clear an accuracy threshold on its own agreement before its assistant went live to members. That sequence is worth copying exactly.
The member on a rest day, the local representative and the staff officer are all now working from the same clause, and it is the right one.
Read the full case study: How a National Union Gave Every Bargaining Unit Its Own Contract Assistant, on One Agreement
Figures from CustomGPT.ai product usage records. Shared anonymously at the customer’s request.
Arooj Ejaz is the Marketing Operations Lead at CustomGPT.ai, where she works on content, growth operations, and go-to-market programs for AI agent and chatbot solutions.