Uncaptured Association Knowledge: What 1,000 Member Meetings Reveal

In October, Alden Do Rosario’s team at CustomGPT.ai scanned 184 association websites and found that a firewall setting can quietly keep AI crawlers away from an association’s public pages. His question was whether AI can read what an association has published. Mine is about the knowledge that never gets published.

At RallyBoard, we run cohort-based peer programs for associations: group mentoring, committees, chapter leader groups, special interest groups and certification study groups. We reviewed 1,000 recorded meetings across 17 associations representing 928 hours of member-to-member engagement. Of the members who came to a cohort’s first session, 76% came back for another.

Then we read 30 of those meetings closely. Members compared vendors, traded benchmarks, worked through new regulations and helped each other make hard calls. That kind of knowledge rarely ends up in an association’s knowledge base, meaning the documents, courses, standards and FAQs that staff maintain and that a member AI assistant draws on.

How much member-to-member conversation happens in association peer programs?

A sample of 1,000 recorded meetings produced about 928 hours of peer conversation across 17 associations, 38 programs and 276 cohorts. The 769 meetings with full transcripts held more than 5.3 million spoken words, the length of more than 50 business books.

The meetings spanned the ways associations bring members together:

• Group mentoring, where a volunteer mentor meets with a small group of members over several months
• Committees and chapter leader groups, where volunteers do the association’s work
• Special interest groups and communities of practice built around a shared role or topic
• Standard-setting and certification work, from exam blueprint reviews to study groups

Members keep coming back. Of the 873 members who attended a cohort’s first session, 76% returned for at least one more. Repeat attendance is what lets a group build the trust to share what they wouldn’t post on a public forum.

What do members talk about when they meet?

We drew 30 meetings at random from 10 associations, held between late July and late October 2026, about 28 hours in all. The mix matched the full set: 13 group mentoring sessions, 10 peer cohorts and communities of practice, 5 committee and chapter leader meetings, one conference cohort and one certification study group. We coded each meeting by the themes that came up.

AI was the most common thread. The next three themes are the kind of knowledge that rarely makes it into a document:

• Shared resources. Meetings regularly ended with someone agreeing to pass along a template, checklist, study tracker, sponsorship packet or policy document.
• Judgment calls. Whether to stay through a restructuring, how to ask for an internal transfer, how to reorganize academic departments, when to launch a product.
• Vendor comparisons. Finance systems, CRMs, assessment platforms, event registration tools and job boards, reviewed by the people who use them.

Why doesn’t this knowledge reach the association knowledge base?

Much of it is shared only because the room is trusted, and much of it goes out of date within months. An association’s knowledge base holds what staff have had time to write, review and approve. Conversations between members move faster and get more specific.

Four patterns from the sample:

• Benchmarks. A group of academic administrators compared how they pay department chairs, from flat course releases to a tiered formula built by faculty. No institution would post its formula publicly. The group asked association staff to set up a shared folder so members could compare documents privately.
• New regulations. Institutional research staff worked through a new federal reporting requirement together. One member flagged a risk the official rollout had not made clear.
• Operating playbooks. Presidents of large chapters swapped sponsorship models, including putting potential buyers, rather than vendors, on the sponsorship committee.
• Vendor reviews. Members compared the systems they use with a candor no vendor case study offers, including tools they found too expensive or too hard to work with.

None of this fits neatly into a course or an FAQ. It’s situational, it changes quickly, and it depends on who is asking.

How are members using AI, and where do they say it falls short?

Members already use AI every day, and the most practical advice we heard combined AI with peers. In one certification study group, a recently certified member described how he passed: for every practice question he missed, he asked an AI tool to rephrase it several ways until he understood the concept, not just the answer. The others in the group were eager to try it.

Other uses came up across professions:

• Founders training an AI assistant on their own voice to triage email and draft first replies
• Mentors drafting bios and brand statements with AI, then editing out language that sounded machine-written
• Finance teams using AI inside their accounting systems to assemble board reporting packages

Members were just as specific about the limits. Hiring managers described receiving hundreds of near-identical AI-written applications for a single role. Interviewers said they can tell when an answer was scripted by AI. Mentors drew a firm line between using AI to present real experience well and using it to invent experience. Several members said referrals and trusted peers matter more to them now, precisely because so much else is generated.

That doesn’t argue against AI. It argues for giving AI better context: the questions members actually ask, the decisions they face and the resources they recommend to each other.

How can associations capture member knowledge without breaking trust?

Capturing member conversations is only worth doing if members trust how it’s done. The rules for capture have to protect that trust first.

In our sample, 25 of the 30 groups used anonymized summaries that identify people by role, not name. The other 5 were committees that chose attributed minutes, which suits their work. Members raised privacy on their own, too: one group set a policy on outside note-taking tools, another discussed state recording-consent laws, and one member proposed an anonymized webinar so a hard-won lesson could reach more peers.

We also learned that anonymizing isn’t enough on its own. A summary with no names can still point to one person through an unusual career path, a region or a one-of-a-kind event. That is why aggregation has to come before anything leaves the group.

Six principles we follow, and recommend:

1. Give notice and get consent before any meeting is recorded.
2. Let each group choose its note-taking mode: off, anonymized or attributed.
3. Identify people by role, never by name, in anything shared beyond the group.
4. Aggregate across groups before insights reach staff or the knowledge base.
5. Publish themes and resources, not quotes.
6. Keep raw transcripts out of any public-facing AI assistant.

This post follows the same rules. Every example is paraphrased, identified by role and stripped of names, places and figures.

How do a member AI assistant and peer learning work together?

They cover two layers of the same knowledge base, and each makes the other more useful.

Layer	What it holds	Best served by
Published knowledge	Standards, courses, policies, FAQs, research	A member AI assistant that cites the association’s own content
Conversational knowledge	Judgment calls, benchmarks, vendor reviews, emerging regulations	Peer cohorts, with anonymized, aggregated summaries

In practice, the two connect in a loop:

1. The assistant answers questions that have a documented answer, any time of day.
2. Questions without a single right answer go to peers who have faced them.
3. Recurring themes from peer groups, aggregated and approved, become new resources the assistant can cite.

Capture also leads back to people. In one community of practice, a member who missed a meeting read its anonymized summary, spotted a discussion relevant to their own work, and reached out to the colleague who raised it. The summary didn’t replace the conversation. It started a new one.

Association leaders gain something too: a view of what members are working through this month, in their own words, across every group at once. That view is usually hard to get. For more on how peer programs differ from community platforms, see our guide to community vs. peer learning platforms.

What does this data not show?

These figures describe RallyBoard programs, not every association. Read them with these limits in mind:

• The window. The analysis covers a subset of 1,000 recent recorded meetings, across 17 associations that use RallyBoard.
• Return rate. The 76% counts members who attended a cohort’s first session, in cohorts that started during the window and held at least three recorded sessions (873 members, 149 cohorts).
• The 30-meeting sample. Meetings were drawn at random and coded by one reviewer from AI-generated summaries, not raw transcripts. A meeting can count toward several themes.
• Capture is not citation. We measured what members discussed, not whether any of it has reached an association’s knowledge base or an AI assistant.

Related Resources:

• AI Knowledge Assistant for Association Staff: See the staff-side version, for the know-how that lives in long-tenured employees’ heads.
• What Member Questions Reveal: AI Conversation Analytics: See how the questions members ask an assistant show what your library is missing.
• ChatGPT vs. Your Own Member AI: See why a member assistant grounded in your own content beats pointing members to a general AI tool.
• How to Make Your Association’s Content Searchable with AI: See how to make the knowledge your association has already published searchable for members.