Work in Progress
This case study is currently being developed. More content and details coming soon!
Simplifying the creation and management of survey quotas and improving scalability.
Survey Response Quota Management
Qualtrics
- • UX/Interaction Model Design
- • Information Architecture
- • User Testing
Panels and Quotas are key elements in the bedrock of Qualtrics’ XM Platform. When discovering insights to close experience gaps, users must first ensure they are receiving high quality responses and insights from their target audience. Creating Quotas allow users to indicate their target demographic breakdowns and track how many respondents from the panels meet their specified target audience conditions.
The current Quota set up process and Quota Reporting Dashboard do not scale to projects that require nested Quotas (ex: age range buckets for each country in the study) and projects that are run in waves, and are often unreadable by new users. This increases time to value and drives up program delivery operational costs because users become responsible for re-creating the information in a user-friendly way, keeping the data updated, and regularly communicating the information to external stakeholders.
I spearheaded the design effort and worked with my team to overhaul the quota management and reporting features within the Qualtrics platform for survey projects, with the overarching goal of streamlining the quota setup process and revamping reporting dashboards. This would significantly reduce the total cost of ownership and expedite the time to value for projects that rely on panels and quotas.
Addressing Key Pain Points & Experience Gaps
The redesign specifically addresses several critical experience gaps, including the current system's inability to handle complex nested and wave-based quotas, which often leads to manual workarounds. A major pain point is the unreadable quota reporting dashboard, forcing Qualtrics Program Managers to manually re-create and update reports for clients and panel providers. This lack of self-service for real-time quota status information, combined with manual report generation and constant communication, has resulted in increased operational costs and prolonged time to value. Furthermore, panel vendors currently struggle to accurately target quotas due to unclear status information, extending fielding times.
I think it would be really nice for us to be able to get some of those details on the panel completed without having to go through our team if that was in the system and I could self-serve, so like "What buckets are filled as of right now? or, How many do we need out of the quotas for this week?" I can't really access any of that without talking to the team, and since it's real-time data that I need, by the time they respond, and the brand team is very fast, but even if it's later in the day, it could be different than what I wanted as of this morning.
Constraints and Challenges: Navigating a Complex Landscape
We faced some challenges with this effort, no the smallest of which was the sheer volume of projects our Research Services (RS) manages weekly
- Handling both simple quotas and advanced logic quotas.
- Substantial time investments in report creation (minimum 83 hours/week) and constant communication regarding quota progress (415 hours/week).
- Balancing the need for comprehensive data with the demand for a human-readable and concise dashboard display.
Success Metrics & Adoption Goals: Defining Our Impact
We’re consistently measuring our customer’s satisfaction with our product as it continues to roll out, using tools like Amplitude and our own internal tools and services. Or targets include:
- 100% decrease in time spent generating reports (from 1 hour/week per research services rep to 0)
- 60% decrease in time spent communicating with clients and vendors (from 5 hours/week per research services rep to 2) - the equivalent time of 8 full-time employees.
- Eliminating extended fielding times and automating quota status communication.
- 80% of users transferring over to our new Quota process and Quota Reporting Dashboard