Traditional member surveys fail for one simple reason: they assume your members have the time, attention, interest, and emotional energy to help you run your chamber.
They don’t.
And before you think, “But our board wants data,” let’s break down what they really want and how you might accomplish that.
Yes, data matters. But a survey isn’t reality. Anyone who’s ever written a paper knows you can find statistics to support almost any angle. Survey results don’t reflect who you asked, they reflect who bothered to answer. Ask a teacher, a student, and a parent whether summers off are good or bad, and you’ll get three different answers. If you only look at those three responses, you might shrug and conclude summers off are “neutral.” But that conclusion would be based more on your tiny sample than on the truth of your community.
Now zoom out. Say you send the same question to an equal mix of 100 parents, 100 teachers, and 100 high school students. If 98 parents respond, but only 26 teachers and 5 students do, your results will be dominated by one group. You might end up reporting that “96% of respondents think summers off are a bad idea.” That doesn’t mean it’s true for your town. It only means it’s true for the people who replied.
In many chambers, the same is true of its member surveys. And yet the board may demand you act on the results of the survey when those results may only illustrate the desires of the most vocal.
Even if you cast a wide net, but you may have a much smaller group of respondents, your data will be skewed. And you’ll have a thin slice of reality served with a side of bias.
If you’ve been feeling like your surveys are less “member voice” and more “the same dozen people with opinions,” you’re not imagining it. Let’s talk about why and what can be done.
Why the Classic Member Survey Is Breaking Down
It sounds great to base your offerings upon what your members want. As a marketer, I tell businesses to focus on how they can solve their clients’ biggest problem. But the biggest problem for many businesses is not being in tune with what that is. It seems simple enough that you could just ask. But it isn’t. Here’s why:
1) We Live in the Age of Over-Surveying
Your members are getting hit from every direction:
- Their kid’s school asks for feedback.
- Their dentist asks for a rating.
- Their software tools send “2-question check-ins.”
- Every event has a post-event survey.
- Every purchase and phone call ends with “How did we do?”
By the time your chamber email shows up with “Please take 4 minutes,” their brain has already decided, “No.”
Even members who love you may skip it, not because they don’t care, but because they’re at capacity. Survey fatigue is far too real.
2) Time Poverty Makes “Helpful” Feel Heavy (and Exhausting)
Busy business owners are putting out fires every day: Payroll. Customers. Hiring. Cash flow. A roof leak. A surprise city permit issue. 1,000 emails. Their own burnout.
A survey feels like homework. Even the best-designed survey still asks them to stop what they’re doing, think abstractly, and type thoughtful responses. That’s a lot to ask for what often feels like an uncertain payoff.
Plus, most members don’t believe their answers will change anything. They see the annual member survey goes out and not much happens after that.
3) Your Sample Is Almost Always Skewed
Surveys rarely represent “your membership.” They represent:
- People with a complaint they want on record
- Your biggest fans who will do anything you ask
- The highly engaged members already in your orbit
- People with admin support who can click things during the workday
- People who love surveys (yes, that’s a thing)
Meanwhile, the members you most need to understand often don’t respond: the quiet renewers, the “I’m too busy” members, the ones drifting, the ones who want value but don’t know how to use you.
So you get an answer, but not necessarily the right answer.
4) The Loudest Voices Can Hijack the Narrative
Surveys tend to amplify extremes. If someone’s experience was “fine,” they won’t spend 10 minutes telling you “fine.” If they’re thrilled or furious, they will. However, even among those two (thrilled or furious), furious is more likely to respond (and write reviews, for that matter).
That creates a dangerous illusion.
Then the board sees a handful of sassy comments, and suddenly you’re rebuilding the whole program around what might be a very small (and very unrepresentative) group.
5) Surveys Ask People to Predict Their Behavior
A survey question like “What programs would you attend?” sounds great until you realize humans are bad at predicting themselves and sometimes they want to say what’s popular, expected, or on one of the options.
People answer based on aspiration, not actual behavior. They have a “might be nice” moment.
They’ll say they want leadership training, but they’ll show up for networking.
They’ll say they want advocacy updates, but they’ll click “small business grants.”
They’ll say they want more events, but renew because they got one great referral.
Surveys often measure intention or hypothetical interest. For example, a gym could ask me if I’d be interested in belly dancing classes. That would be an obvious yes because I find it fascinating and beautiful but do you think you’ll see me in those classes. Nope. They sound amazing, but they’re just not me. Your members may be answering the same way–that sounds great…for other people.
Chambers need to measure actual need and action.
6) The Format Is Built for Chambers, not Members
Most surveys are designed around what the chamber wants to know, things phrased like “Rate this. Rank that. Choose one.”
But members live in a different reality. Their preferences aren’t neatly boxed. Their needs change by season, cash flow, staffing, mood, and what’s currently on fire.
A static survey is like taking a close-up snapshot of the Rose Parade and calling it a garden. (Sure does look like one zoomed in).
7) They Don’t Know Until They Do
Let’s not forget, sometimes members don’t know what they want until they see it and even then they may just be picking from what you’ve made available. And as pointed out above, that’s not always reliable.
Think of it this way. You’re in line at a cafeteria. It’s really busy. There is no menu posted. Just one food after another. You’re hungry. You pass by the first few things because they’re unappealing. At some point, you settle for what’s in front of you. You didn’t like what you’d already passed, and you’re not sure what’s ahead. But you liked what was in front of you well enough and if you don’t choose, you’ll eventually get to the end of the line and have to pick something before it’s time to pay. Sometimes that means you’ve picked something you really like. Sometimes it just means you picked something because it was better than what came before it.
If you give your members 3-5 options to choose from on a survey, they may not pick their fave. It may just be the best of what they see in front of them.
Are Surveys a Thing of the Past?
I wish, but not exactly. However, the era of “annual survey = member insight” is fading.
Surveys still have a place when:
- you’re making a specific decision and need directional input
- you’re validating a hypothesis, not fishing for ideas
- you can clearly say what you’ll do with the results
- you keep it short, targeted, and timed well
- you combine it with other methods so it’s not the only “voice”
- you’re picking a theme for the next event (everyone can get excited about that)
The issue isn’t surveys. It’s over-reliance on them as your main prognostication tool.
Creative, Lower-friction Ways to Learn What Members Want (sans survey)
What your board may not realize is that your members are giving you data all the time. You just have to collect it in ways that don’t feel like extra work to them.
The “two-question” renewal conversation
During renewal (or pre-renewal), ask two questions on a quick call or even on the invoice:
- “What’s one challenge you want less of this year?”
- “What’s one outcome you’d love more of?”
That language keeps it grounded in real life, not abstract “programming preferences.” Plus, it’s a fill-in-the-blank so they won’t select the best option out of what’s available. It’s also not a bad question for them to be asking themselves anyway.
Micro-polls in the moment
Instead of a survey link, use a one-click poll:
- in an email
- on LinkedIn
- in your event check-in QR
- in a text message
One question. One tap.
Example:
“What would help you most in Q2?”
A) More customers
B) Hiring help
C) Cost control
D) City/state issues
That single question can guide months of content and programming. (While you still may have a respondent bias, more people are apt to answer short, off-the-cuff types of interactions.)
“Listening posts” at your events
Set up a simple board at networking events:
“What’s the thing you wish you had time to fix in your business right now?”
Give sticky notes. Let people answer anonymously. Take photos. Categorize later.
Low effort for them. High signal for you.
Member journey mapping from your own data
Look at behaviors:
- who opens emails and what do they click?
- which events draw which industries?
- who uses directory listings?
- who attends ribbon cuttings but not workshops?
- who shows up once and disappears?
Behavior data is often more honest than survey data because it costs members zero extra effort. (What is it that people say about actions speaking louder than words?)
Five-member “kitchen cabinet” calls
Pick 5 members from different segments and do a 30-minute roundtable on Zoom.
But here’s the key: don’t invite only the usual suspects.
Invite:
- one brand-new member
- one long-time quiet member
- one member who’s on the edge of leaving
- one highly engaged member
- one “we never see them” member
Ask them what’s working, what’s confusing, and what would make membership feel essential.
The “walk-and-talk” strategy
If you’re in a downtown corridor, walk in (but be respectful of busy times in their industry).
Ask, “What’s one thing that would make the chamber more useful to you this quarter?”
Capture the answers in a running log. Patterns will appear fast.
“Voice note feedback” instead of typed answers
People hate typing. They don’t hate talking.
Invite members to reply with a 30-second voice memo answering one prompt. It feels personal, and you’ll get richer language and emotion.
Customer-style “why did you join” interviews
New member onboarding should include a short interview:
- “What problem were you hoping membership would solve?”
- “What would make you say ‘this was worth it’ in six months?”
That’s intelligence.
The Bigger Question Chamber Pros Should Debate
If surveys are producing low response rates, biased samples, and vague “nice ideas,” are you measuring member value… or member opinions?
Maybe the future isn’t “more surveying.” Maybe it’s building a listening system with:
- small, frequent inputs
- real behavior data
- direct conversations
- visible follow-through
The most powerful member insight isn’t what they say they want. It’s what they repeatedly show you they need.
