The testing process
To kick-off testing, think about the question (or questions) you wanted to answer about your message. This is your research question and here’s how you can identify it.
STEP 1
Outline your reasons for testing
The first thing you need to know is what you are looking for.What are you trying to understand? This is what will go into your research question
STEP 2
Define your research question
The first thing you need to know is what you are looking for.What are you trying to understand? This is what will go into your research question
STEP 1
Outline your reasons for testing
Know what you are looking for
Research questions help you know what answers you’re searching for in your test. They guide you towards achieving the mission of your testing activity, and it is from this mission that the research questions are crafted. The most important element to decide in the early stages of research is to understand if your research is formative or summative. Formative research is the first stage in evaluation, where you seek to explore and understand the context of a problem, how your audience responds to said problem, and
Research questions help you know what answers you’re searching for in your test. They guide you towards achieving the mission of your testing activity, and it is from this mission that the research questions are crafted. The most important element to decide in the early stages of research is to understand if your research is formative or summative. Formative research is the first stage in evaluation, where you seek to explore and understand the context of a problem, how your audience responds to said problem, and determine the best ways to reach them. In this phase, you may use tools like in-depth interviews and focus group discussions to understand the problem and audience. Formative research is the “understand” phase. Summative research is near the last stages of research, when you want to test if and how your designed solution works. In this phase, you may use more quantitative methods like surveys and A/B testing to reach more conclusive insights.
determine the best ways to reach them. In this phase, you may use tools like in-depth interviews and focus group discussions to understand the problem and audience. Formative research is the “understand” phase. Summative research is near the last stages of research, when you want to test if and how your designed solution works. In this phase, you may use more quantitative methods like surveys and A/B testing to reach more conclusive insights.
What goes into a research question?
When testing your message, a research question needs to show the relationship between two variables. The first variable (called “independent variable”) is the key feature that you are hoping to test. The second variable (called the “dependent variable”) is the outcome you think the key feature will have.
For example, if you want to test whether a positively-framed message can increase the reporting of gender-based violence, your research question can be as follows: “Do positively-framed messages increase the reporting of gender-based violence?” The positively framed message is your key feature and the increase in reporting of gender based violence is the outcome you hope to achieve.
How you choose to measure your outcomes:
Which approach would be easy and clear to the respondent? Would quantitative or qualitative approach be better to the questions you are interested in? In both approaches, how are you going to choose to operationalise (or measure) the things you most care about, especially concepts like trust, sustained interest, and behavior change?
Channel of communication:
Will it be online, SMS, in-person, or phone call? Or will you test all four channels to see how they affect your outcome? In this case, a good research question will be: “Do SMS invitations increase the number of people who sign up for the gender-based violence campaign?”
Features of your message:
This has to do with what your message has to look like to achieve your outcome. For example, you might want to add images or a sound to see if it will increase the number of sign-ups. Agood research question here would be: “Will images in the invitation message increase the number of people who sign up for the gender-based violence campaign?”
STEP 2
Choose your testing method
Which approach do I use?
Now that you know your research question, which method should you use to try to answer it? There are different resource dependent methods that you can use to effectively test your messages. Over the next two pages a snapshot of 4 key methods that you can use, including what the method looks like under a low budget, is provided.
- Focus Group Discussions
- In-Depth Interviews
- Surveys
- A/B Testing
Share your perspective
