Total Survey Design

In this episode of Total Survey Design, Dr. Azdren Coma sits down with Dr. Rubén Ángel Arias Rueda, Project Manager and Lead Researcher at Washington State University's Social and Economic Sciences Research Center, for the second part of a two-part series on multilingual surveys. Dr. Arias Rueda specializes in mixed-methods equity research, bilingual data collection, and community-engaged fie...

In this episode of Total Survey Design, Dr. Azdren Coma explores what it takes to design surveys for populations that speak two or more languages. The episode explains why measurement equivalence (ensuring responses mean the same thing across languages) matters more than literal translation, and why the goal should be conceptual equivalence rather than word-for-word matching. Drawing on a personal...

In this episode, I explore research by Lena Le and Thom Allen at Washington State University on the use of personal appeals in survey contact letters — the persuasive statements that encourage people to participate, like explaining how results will be used or telling respondents their opinion matters. Analyzing 190 surveys, their data shows response rates climb from 21.6% with no appeals to 51.6%...

In this episode, Dr. Azdren Coma explores how the Dunning-Kruger effect distorts self-reported skill ratings in hiring and surveys. A seemingly straightforward question like “On a scale of 0 to 10, how skilled are you in Microsoft Excel?” can produce misleading data because undefined scale points allow respondents to interpret them differently. Beginners often overestimate their abilities due to l...

In this episode, I will be talking about open-ended questions. Along with closed-ended questions, open-ended questions are the other major type of question used in surveys. This episode covers the purpose of open-ended questions, their pros and cons, the different types of open-ended questions, several examples, and finally some tips and best practices for implementing them effectively in surveys....

What happens when you are faced with a list of options but do not have a strong preference? In this episode, I explore how the simple order of a list can nudge a person's choice. I move beyond the idea of favorite flavors to look at the undecided voter and the hungry diner. I examine why being first on a ballot can sway an election by as much as 10 percent in some local races. I discuss why t...

This episode is a live guest lecture recorded for a Marketing Research class at Washington State University.  The guest lecture is meant to introduce complete newcomers to general ideas and considerations for survey design. I also offer a few general tips on survey questionnaire design. Support the show Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/...

In this episode of the Total Survey Design Podcast, I explore whether to include a neutral midpoint option in survey questions or force respondents to pick a side. I define forced-choice formats, from even-numbered scales and binary picks to pairwise comparisons. I discuss key evidence like the 2019 Pew Research experiment, which found forced yes/no questions increased reporting on sensitive topic...

Should you ever force respondents to answer a survey question? In this episode of Total Survey Design, I examine forced questions. While these features are often used under the assumption that more complete data means better data, forcing responses frequently leads to worse outcomes by increasing survey breakoffs, encouraging satisficing, and undermining respondents’ trust. I outline more effectiv...

On this episode of the Total Survey Design podcast, I put generative AI to the test by asking it to critique and improve the Microsoft Word Feedback survey: a short, everyday questionnaire that many of us have seen after using the program. I compare the suggestions from Grok, ChatGPT, and Gemini, highlight where they agree, where they differ, and share my own thoughts on what they got right and wh...

On this bonus episode of the Total Survey Design podcast, I follow up on Trump and Rogan’s polling critiques with a listener email highlighting nonresponse bias, the real issue behind low response rates, where responders often differ systematically from non-responders. I explain how rigorous weighting corrects for this bias, why it’s not foolproof, and how strong weighting helped make the 2024 ele...

On this episode of the Total Survey Design podcast, I examine an episode of the Joe Rogan Experience podcast, #2219, which aired on October 25th, 2024, where he discusses survey sampling with President Donald Trump. They both express skepticism about the reliability of polls for making broad national claims. They also refute the ability to make predictions from small samples and criticize polling...

In this episode, Azdren creates a feedback survey from scratch, showing the process from idea formation, to messy first draft of the questionnaire, to final product.  To see the final version of the questionnaire and to give your valuable feedback on this podcast, please visit https://tinyurl.com/totalsurveydesign  Support the show Contact us at: totalsurveydesign@gmail.com Find us online at: inst...

This episode of the Total Survey Design podcast demystifies the statistical technique of survey weighting (or post-stratification), explaining how it corrects sample imbalances by giving "quiet groups" more influence. The hosts define weighting, detail when it's an effective correction tool for probability samples, and provide practical advice on avoiding common pitfalls like over-a...

This episode explains the important distinction between response rate and nonresponse error in survey research. Drawing on principles from Dillman et al.(2014), it clarifies that a high response rate does not necessarily mean low nonresponse error, and vice versa. Through two practical examples, the episode illustrates how systematic differences between respondents and nonrespondents -- not the nu...

In this episode, the hosts speak with Mark Miazga, Director of University Survey and Assessment Services at the University of Minnesota, about his journey into survey research. Mark shares how his upbringing in rural Wisconsin and a background in law and nonprofit work shaped his perspective on data and public service. The conversation covers his transition from corporate law to survey methodology...

In this episode, we talk with Robert Santos, Director of the U.S. Census Bureau from 2022 to 2025, about his professional journey and philosophy toward data and public service. Rob shares how his upbringing in a Mexican American community in San Antonio and how his early exposure to music and math shaped his path into statistics. The conversation explores his commitment to equity in data, the impo...

In this episode, I talk about a survey I didn’t plan to analyze—one that popped up the moment I opened Microsoft Word. Microsoft asked for feedback, and well… I had some. I break down the flaws in their opening question, including directional bias, vague wording, and unlabeled scale points. I also reflect on how these design choices impact data quality and what could have been done better. It’s a...

In this episode of Total Survey Design, Dr. Azdren Coma and Dr. Seon Yup Lee interview Dr. Jon Krosnick, a leading expert on political behavior, survey methodology, and public opinion. Dr. Krosnick is a professor at Stanford University who has advised the U.S. Census Bureau, led national studies on voting behavior and climate change, and published extensively on how people form attitudes and make...

In this episode I sat down with Thom to talk about surveys. We talked about his time at the Social and Economic Sciences Research Center, working with Don Dillman on various projects including the 2000 U.S. Census, some of his favorite projects, my dissertation survey, the application of the Tailored Design Method on a wide variety of cultures, and much more. Support the show Contact us at: totals...

In this episode, we explore essential considerations for crafting effective surveys.  We begin by emphasizing the importance of defining the purpose of your survey, which serves as a guide through each step of its creation. We discuss how understanding your main goal—whether it's measuring customer satisfaction or academic variables—shapes your questions and improves data quality. Additionall...

In this episode, we discuss the target population. We emphasize the importance of a well-defined target population, in context of the goal of your survey, and in relation to your sampling frame. We offer examples of target populations of various studies, and offer suggestions for the types of variables that you may want to consider when thinking about your target population.  Support the show Cont...

In this episode, we interview James McCall, a survey methodologist at Westat. James shares what first drew him to surveys, and what drove him into a career of surveys. We talk about his role as a survey methodologist in industry. We discuss cognitive testing, data scripts, and other elements of the art and science of survey design. We also discuss our pet-peeves, such as matrix questions, check-al...

In this episode, we explore the key design elements of effective surveys, emphasizing the importance of user-friendly experiences and reliable data collection. But first, in our free talk segment we discuss the compensation structure of the YouGov platform. Then, we highlight practical tips like strategic question ordering, clear language use, and consistent visual design. Additionally, the episod...

In this episode, we introduce a new free talk segment, where we discuss and celebrate Seon Yup's new job as chief survey methodologist at Kansas State University. We then explore why many people distrust surveys and some things we can do to regain their trust. We discuss key issues like sensitive topics, the impact of interviewer identity, and the prevalence of poorly designed surveys that fa...