The October 9th Marketing Insider on Media Post, leads with SLRG’s Jon Last discussing why it’s important for sports properties to avoid the mistakes of oversimplification when measuring fan satisfaction.
Beyond the buzz surrounding the Philadelphia Flyers’ controversial new mascot, another piece of Philadelphia-related sports news caught my attention this past week: the installation of 75 “Happy or Not” kiosks at The Wells Fargo Center. These allow fans to evaluate restrooms, retail locations and concessions by selecting one of four smiley or frowning face icons.
As a classically trained marketing researcher, I’m always glad to see properties measure fan satisfaction.
But these kiosks, which I have also seen (and avoided) in a number of airport restrooms, strike me as evidence of the old adage that often no information is better than potentially misleading information.
Proper sampling is a fundamental of good research: defining specific quotas that assure a random and representative sample of respondents who reflect the full onsite attendee base, rather than just those who selectively choose to engage with a kiosk.
A professional interview team has the mobility to cover multiple areas of a sports venue and brings expertise in sampling and fan interaction that enables them to engage respondents across all demographic segments.
A kiosk-based approach runs the risks of multiple responses by the same person, which can skew results, not to mention the introduction of other self-selection biases (intoxicated or otherwise unqualified participants). It does not accurately represent the mix of event attendees, as we’ve proven in tests where parallel intercept and kiosk-based methodologies were deployed at major sporting events, yielding vastly different respondent profiles.
A thoughtfully designed intercept methodology assures representative and stable bases of respondents across numerous demographic criteria and attendance behaviors that may be of particular importance to the management of a property. An intercept survey strategy also eliminates self-selection and non-respondent bias that inaccurately skew results derived from a non-managed data collection strategy. Non-managed or self-selected samples tend to result in over-representation of respondents with strong opinions or strong motivation to participate in surveys.
In other words, we’ve observed over-representation of those with extreme opinions (very happy or angry) with a static kiosk-based approach.
This also surfaces the methodological flaws inherent in the simplistic approach of four emoji-like smiley face icons to represent sentiment. Ten-point scales have proven to yield significantly more robust analysis than four or five choices. They allow respondents to offer nuance in quantifying their satisfaction levels. Limiting response to “very or somewhat happy or sad” as these kiosks allow, leaves no room for neutrality and fails to get a solid read on whether slightly positive or negative respondents are closer to the extremes or middle range.
Understanding in-game fan experience is mission-critical in a sports environment littered with a proliferation of consumption options. Isn’t it worth a more thoughtful approach?