by Zoë Billington, User Research & Customer Insights, Santa Monica, California, zbillington@gmail.com
This quarter, I Zoomed across the pond with Ray Poynter. Poynter has spent the last 46 years at the intersection of research, innovation, and business, having been involved in the development of Computer Assisted Personal Interviewing (CAPI), online systems, online surveys, social media research, and now AI-enhanced research. He is the author of The Handbook of Online and Social Media Research and The Handbook of Mobile Market Research, the editor of ESOMAR’s book Answers to Contemporary Market Research Questions, a content author for the University of Georgia’s Principles of Market Research course, and the founder of NewMR. Last, but not least, Poynter is a fellow of the MRS and current president of ESOMAR.
Zoë: Let’s start with the 10,000-foot view of your career. How do you tell the story of your career and how you got to where you are today?
Ray Poynter: After graduating from university, I started creating statistical programs, then formed my own agency to support the industry. For quite a long time, I was mostly a quantitative researcher. We pioneered CAPI and were doing it across the world, in Japan, Thailand, and all sorts of interesting places. Then we started to do a lot of online dial-up stuff where we could create synthetic data and do modeling with that.
We were lucky to be acquired by a company called the Research Business, headed by two of the most amazing people in the research industry: Wendy Gordon and Colleen Ryan. They were qualitative researchers right down to their toes. They bought us as a quantitative agency, but working with them, I saw the two sides and how they work together. From there, I’ve gone through various jobs: I’ve been with Millward Brown, IntelliQuest in the U.S., and Vision Critical—usually in managing director roles of a country or something like that.
But for the last 10 years, I have been focusing more on training and less on research. I wrote a textbook on how to do online research. A few years later, with Sue York and Navin Williams, I wrote an industry textbook for mobile research. I’ve also just started a joint venture, creating an AI tool to do automated analysis of simple surveys.
So that’s all one side of my life. The other side is that I was elected president of ESOMAR for the 2023–2024 term. There was a slightly fraught situation the previous year that had a few issues. I felt that maybe I could help, and enough people thought likewise and elected me. That has been the main thing I did from April last year until the end of last year, and it’s going to be a big part of my life this year as well.
Zoë: I imagine you get this question a lot, but our readers will want to know: how have you seen qualitative research as a practice and/or as an industry evolve over the course of your career?
Ray: There’s been a lot of deskilling in qual. There used to be the giants—people who were really immersed in the theory and who, themselves, had learned from working with the people who more or less invented qualitative research in the 1950s and the 1960s. I think there was a lot more theory involved in the analysis of qualitative research. Because the data collection process took longer, there was a lot more time from data collection to reporting, so people would talk more about grounded theory, discourse analysis, and these sorts of things. Now, there is a lot more acquired tradecraft. People do a lot of projects. They use an intuitive understanding, but it’s probably a little bit less formal than it used to be.
Zoë: From your perspective as a teacher and trainer, how are researchers getting their training nowadays, if not in those more formal ways of the past?
Ray: Yeah, training is a really interesting thing. In addition to conducting trainings, I’ve taught a semester on market research at a Japanese university. I’ve taught new research approaches at the business school in Nottingham—so not just within the industry but around it. If I look at the last 20 years of running courses for things like the MRS in the U.K., the big shift is that it used to be almost all agencies sending their people; now, it’s almost all end clients. There’s a massive hunger among end clients to understand these processes more.
The other change, of course, is that people want shorter courses. So, I now do a lot of one-hour and two-hour courses. For example, in “Finding the Story in the Data,” we talk about what you’re looking for, how to make sense of what’s there, and how to communicate what’s there.
I think we can teach these individual skills rather than saying: here’s a 40-hour course, and then you’re an accredited researcher. I think a lot of learning in the future is going to be bite-sized, and it’s a good way to bring some of the theory back into it while still making it accessible.
Zoë: As we’re touching on some big changes you’ve witnessed in the industry, it feels like a natural time to talk about your work around scenario planning. What is scenario planning, and why is it important?
Ray: Scenario planning is really powerful. Instead of trying to always be right, which is impossible, it tries to never be wrong. So, you take a topic, and you say: here are three possible versions of how the future might go. We say to ourselves, it’s going to head in this direction or this direction or this direction. For example, AI could really boost the individual qualitative researcher, it could replace them, or it could displace them in two or three different ways. We’ve got some really big differences going on in each scenario. So, we need to envision all three scenarios and look at how they unfold and move forward. Very often, this is a great place to mix qual and quant skills because you have people from the quant background who will pull out some of the data. Then qualitative research will come in when you’re trying to think about what sort of worker would actually do this, how one could actually benefit from this, and how we could describe them. All of that is part of scenario thinking.
But I’m just going to back up for a second to also say that we should always remember when we get it really wrong or we don’t spot something enormous. I’ve never yet met an ethnographer, anthropologist, qualitative researcher, or quantitative researcher who predicted that something like 50 percent of people aged 20 to 40 would have tattoos in the U.S. and the U.K. It was not on anybody’s radar. It happened before people even talked about it. It’s just really a useful reminder that sometimes things are going on right around us, and we’re not picking them up.
Zoë: What are some of the big scenarios that you think we as researchers need to be prepared for?
Ray: There are some really positive scenarios for the qualitative researcher: one is to be able to use AI to reduce the time it takes to get to a result and to improve the product. If we just start with basics, I pretty much now always use ChatGPT to double-check my early notes on a project. So, if I write a discussion guide, I will say to ChatGPT: Have I missed anything? Have I got the language right for this group of people? Maybe I’m going to be interviewing Australians, and ChatGPT will say that, actually, in Australia, they don’t use that phrase. For example, I’m talking about beds. The thing you put over yourself in bed is called a duvet in the U.K. In Australia, it’s a doona. So, ChatGPT picks up that sort of stuff. It will also say, “Okay, you’re doing these interviews in France. You’ve got these questions about ethnicity that are illegal in France.”
This year there are also going to be better and better tools for taking your video recording of the interview, creating a transcript, and then looking at that and saying it has the following main themes. If you’re a single consultant, you’ve done 10 in-depth interviews, and you probably know where all that stuff is, and you can handle it yourself. But if you’ve got five consultants, each of whom has done five depth interviews, now you really do want a tool that tells you: did Joe find anything that was different to Sally?
Zoë: That sounds pretty positive! What about the negatives of AI?
Ray: The biggest downside scenario is called Smart DIY. When I’m talking to clients, I call it “smart self-serve.” We’ve had DIY quantitative like SurveyMonkey, Qualtrics, and many other options. There is a little bit of DIY qualitative, but it hasn’t really taken off because it’s just a method of collecting information. It doesn’t tell you what to do with the information you collect. For example, it collects all these video interviews, but then you don’t have the intellectual training to know what you’re supposed to be looking for and how you’re supposed to work it out. So, that’s really why, despite having some impressive qual tools out there, like Voxpopme, they haven’t really changed the industry very much.
But when we bring in AI, things change. Let’s start off with SurveyMonkey. It will certainly write the survey for you and tell you who to interview, and it will give you a hand in finding the results. Similarly, if we’re using smart qualitative tools, it will start analyzing the videos, and it will become a lot easier for end clients to do their own simple research. One of the places this is going to make a really big difference is in usability research for design teams. At the moment, the best practice is to embed a researcher in the design team as the UX research specialist. Worse practice at the moment, and it’s quite common, is the designers think they can run focus groups, or they think they can do observational research—and they can’t. But when we start to move these smart tools around, then more and more of these designers will be able to conduct research because the machine will guide them toward how many observations they should do and what they should be looking for.
So, I think we are going to see a lot more simple projects move to nonpractitioners being able to do them. It may not shrink the industry because there will still be, for the foreseeable future, all sorts of complicated things that AI won’t do.
Zoë: Can you think of any examples of things AI won’t do so that our readers can think about how to ensure their skills are complemented—not replaced—by AI?
Ray: One of my favorite examples is when I was talking to a qualitative researcher from Salomon, the sports brand. She’s a runner, and she goes running with a customer or a couple of customers. They run for 10 kilometers talking about the product and how it’s feeling in the moment. Say it’s a sports bra: Where’s it rubbing? Does it make you feel like a runner? It’ll be a long time before she’s replaced.
I could imagine, within a year, being quite happy to go into McDonald’s and have my order taken by a bot. The burger, Big Mac, may be created by an automated AI system and served by AI. But if I’m alive 50 years from now, the best sushi I will eat will be in Tokyo, served by a human who spent 40 years learning their trade. So, there are going to be things that will definitely need to stay human, and we need to spot what they are. It won’t be everything.
We had a fascinating ESOMAR conversation about the impact of AI. Somebody was saying that if we got to a position in five years where a machine could do the qualitative interview as well as a human and produce results as good as a human, would we use the machine? The answer was hell yes. We have to produce something different. If we look back in time, something fascinating happened in the 19th century that’s a good model for how we might think about qualitative research. We got photography, and, of course, a photograph is more accurate than a painting. So, where could it go? The direction that the camera couldn’t follow. Art had impressionists, then surrealism, and various types of Cubism and alternatives. So, the camera didn’t end art. It actually enlivened art because it made us go in a new direction—and that is what we are going to need to do as researchers. We are going to need to be better at using the tools, and I think qualitative researchers are going to be, as a group, better at talking to generative AI than quantitative researchers, some of whom have difficulty talking to people.
Zoë: Thanks, Ray! Now, excuse me while I go plan for all these scenarios you’ve shared with me today.
