Download Links

A guide to . messaging & testing

About the guide

This is a resource to help you and your organization test your communications.

What is this guide?

This guide will take you through the basics of testing. Specifically:

1

It will help you understand the purpose and needs of testing and how testing can improve your communications design.

2

It will take you through a step-by-step process of what you need to decide from designing communications to testing materials. We will also help you choose which testing method is most suitable for your needs.

3

It will help you understand the purpose and needs of testing and how testing can improve your communications design.

LATEST

News & Publications

Tafakari Yearbook 2024

OVERVIEW DOI: http://doi.org/10.62372/JDVN6905 As an organization dedicated to behavioral science in pursuit of poverty alleviation, we feel that pressure from all around us. There is pressure from the outside world: climate

Read More »

Who is this guide for?

This guide is for organizations with little to no experience with testing

Civil society organizations (CSOs) like yours often work with limited funds, time, and resources. There’s also a misperception that testing is expensive. We’d like to show you that this isn’t always the case. This guide presents some simple ways to test your messaging and campaigns to identify how to best achieve your desired results – all while preserving your resources. This guidebook is meant for advocacy, communications, research and programs teams. These can be officers, managers and directors working with nascent or even established CSOs, social enterprises and other non-profits. It is particularly useful to officers and teams that take up the roles of program and project implementation, communications, public affairs, research, stakeholder engagement and advocacy. These can be Communications Officers, Research teams, Communications Leads, Project Managers, Program Managers and Coordinators, Grant Managers, Advocacy Leads, Engagement Managers and Directors, Chiefs and Directors of Advocacy and other similar roles. Executive Directors and Board Members of these organizations also have a role to play as they are the custodians of the overall organizational strategy and vision. When communicating to an organization’s primary stakeholders and partners, this guidebook is a free resource that can be used to complement the communications and advocacy strategies. The guidebook therefore can be used as a tool when crafting advocacy messages and campaigns, when releasing project information on interventions, reporting implementation and communicating impact.

Ready to test your campaign?

Yes. At the end of this guide, we will provide you with online resources to assist you in testing. 

Understanding testing

What is testing?

Testing is a series of investigations that evaluate your assumptions on why something might or might not work. In CSO messaging, testing measures your priority audience’s reaction and helps determine whether they will find your message understandable, believable, and appealing.

You should test to see if your messaging and programming will perform as well as you think it will.

As people who work in communications we are under pressure to create ideas that will resonate with our audiences and change how they think. Testing lets us know whether the idea we pick works before we roll it out.

What could I test for?

You should test the elements in your campaign that you think could drive the most impact with your audience to see how they respond to them. On the right are some elements that matter in how effectively you communicate.

... elements that drive the most impact with your audience.

Comprehension

Will your intended audience understand the message relayed?

Attractiveness

Is the message related to the issues faced by them?

Relevance

Is the message related to the issues faced by them?

Improvement

Will your intended audience understand the message relayed?

Believability

Is the message believable and realistic to them?

Acceptance

Is the message related to the issues faced by them?

Share your perspective

Using behavioral science to strengthen messaging

There are some concrete steps you should complete before assessing how you can integrate behavioral science into your messaging and/or testing approach. Here’s what you should have already done:

1

Identify the goals of the project

2

Identify the audience(s) you need to engage in order for change to happen;​

3

Create some initial draft messages appropriate for your different audiences

4

Identify what the success outcomes of your campaign are (e.g., certain behavioral or perspective changes and actions in your audience);

Behavioral science framing can really help with that last step. Behavioral science studies how human beings make decisions and provides powerful insights that can improve how your audience responds to your messaging.

Here are some framings below you could try when creating messaging:

Social norms

As humans we tend to follow behaviors that are largely accepted or desirable in our society. The goal of using social norms framing is to show that a majority of people within a group are doing something. This makes the thing desirable to do as it is now “acceptable”

Social norms can backfire if it highlights the wrong behavior, so only use this when there is an overwhelming majority, e.g. 60% of women doing Action X (which suggests 40% don’t!). It is also useful to highlight personal stories and testimonials and bring them to life with color and photos.

Simplification

Remove jargon and reduce text;
Give very specific instructions, not requests e.g. ‘Call 999’ instead of ‘Please Ring’;

Break actions into steps using numbers, bullet points or checklists;

Use colours, images, bold or larger fonts to highlight key phrases.

Framing

There are two types of framing;

Gain framing is where you emphasize what the recipient of the message could gain from taking the desired action. E.g If you report gender based violence today, you help by saving a life.

Loss framing is where you emphasize what the recipient of the message stands to lose by not taking action. E.g If you fail to report gender based violence today, you are responsible for the harm caused to the victim. It is important to test whether gain and loss framing to see which gives you better results.

Share your perspective

The testing process

To kick-off testing, think about the question (or questions) you wanted to answer about your message. This is your research question and here’s how you can identify it.

STEP 1

Outline your reasons for testing

The first thing you need to know is what you are looking for.What are you trying to understand? This is what will go into your research question

STEP 2

Define your research question

The first thing you need to know is what you are looking for.What are you trying to understand? This is what will go into your research question

STEP 1

Outline your reasons for testing

Know what you are looking for

Research questions help you know what answers you’re searching for in your test. They guide you towards achieving the mission of your testing activity, and it is from this mission that the research questions are crafted. The most important element to decide in the early stages of research is to understand if your research is formative or summative. Formative research is the first stage in evaluation, where you seek to explore and understand the context of a problem, how your audience responds to said problem, and

Research questions help you know what answers you’re searching for in your test. They guide you towards achieving the mission of your testing activity, and it is from this mission that the research questions are crafted. The most important element to decide in the early stages of research is to understand if your research is formative or summative. Formative research is the first stage in evaluation, where you seek to explore and understand the context of a problem, how your audience responds to said problem, and determine the best ways to reach them. In this phase, you may use tools like in-depth interviews and focus group discussions to understand the problem and audience. Formative research is the “understand” phase. Summative research is near the last stages of research, when you want to test if and how your designed solution works. In this phase, you may use more quantitative methods like surveys and A/B testing to reach more conclusive insights.

determine the best ways to reach them. In this phase, you may use tools like in-depth interviews and focus group discussions to understand the problem and audience. Formative research is the “understand” phase. Summative research is near the last stages of research, when you want to test if and how your designed solution works. In this phase, you may use more quantitative methods like surveys and A/B testing to reach more conclusive insights.

What goes into a research question?

When testing your message, a research question needs to show the relationship between two variables. The first variable (called “independent variable”) is the key feature that you are hoping to test. The second variable (called the “dependent variable”) is the outcome you think the key feature will have.

For example, if you want to test whether a positively-framed message can increase the reporting of gender-based violence, your research question can be as follows: “Do positively-framed messages increase the reporting of gender-based violence?” The positively framed message is your key feature and the increase in reporting of gender based violence is the outcome you hope to achieve.

How you choose to measure your outcomes:

Which approach would be easy and clear to the respondent? Would quantitative or qualitative approach be better to the questions you are interested in? In both approaches, how are you going to choose to operationalise (or measure) the things you most care about, especially concepts like trust, sustained interest, and behavior change?

Channel of communication:

Will it be online, SMS, in-person, or phone call? Or will you test all four channels to see how they affect your outcome? In this case, a good research question will be: “Do SMS invitations increase the number of people who sign up for the gender-based violence campaign?”

Features of your message:

This has to do with what your message has to look like to achieve your outcome. For example, you might want to add images or a sound to see if it will increase the number of sign-ups. Agood research question here would be: “Will images in the invitation message increase the number of people who sign up for the gender-based violence campaign?”

STEP 2

Choose your testing method

Which approach do I use?

Now that you know your research question, which method should you use to try to answer it? There are different resource dependent methods that you can use to effectively test your messages. Over the next two pages a snapshot of 4 key methods that you can use, including what the method looks like under a low budget, is provided.

Share your perspective

After testing

Iterations and fine tuning of communications

Testing takes time

The first thing you should think about after testing is whether the test answered your question.

This guide will take you through the basics of testing. Specifically:

1

It will help you understand the purpose and needs of testing and how testing can improve your communications design.

2

It will help you understand the purpose and needs of testing and how testing can improve your communications design.

3

It will help you understand the purpose and needs of testing and how testing can improve your communications design.

If the results are positive it means your message is working! Well done! Roll it out. If the results are negative or inconclusive the first thing you should do is find out what you can learn from the data.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.

Does it give you more insight into what you need? After this you might have to make some adjustments.

There are two scenarios to think about here

You need to make minor changes

This means you may need to use simpler words or maybe tailor your content a little better. In this scenario the results will highlight instances where elements of your message campaign may be hard to understand or relate with by the audience.

You need to make major changes

This means your project team will need to reassess all the elements of the campaign. You may need to restructure your entire strategy to achieve your goals. These changes will warrant further testing to see if the new strategy is effective.

Share your perspective

Testimonials

FAQs

Low-Cost Message Testing.

Message testing is a process that helps CSOs evaluate the effectiveness of their communication strategies. It’s essential for ensuring your messages resonate with your target audience and drive your desired outcomes.

You can learn more about testing, the different components of your message that can be tested as well as qualitative and quantitative testing methods using the Low-Cost Message Testing Guide. You can visit our website to access other resources on communication testing.

Testing is instrumental in enabling organizations to curate clear messages for its intended audience. Through message testing, you can identify components of a message that can be enhanced to effectively reach the intended audience.

Get access to the offline Guidebook

Download Links