Data can do everything – and nothing

RNW Media brings young people together on digital platforms—with websites, WhatsApp, Facebook, Twitter and other channels. Through our platforms they get comprehensive evidence- and rights-based information. They also have safe places where they can engage in constructive dialogue and focus on their aspirations. When young people engage on our platforms they generate huge amounts of data and that data is a resource that can enable us to refine our work and strengthen our impact.

The raw data generated on our platforms is a resource but, as RNW Media Data Analyst and Researcher Kyle Snyder says, “data can do everything and it can do nothing.” The essential key is knowing exactly what it is we want to know. A mass of data is only useful for the insights it can give us. Those insights are only available through careful analysis, and good data analysis starts with clear questions.

One of RNW Media’s aims is to stimulate constructive dialogue between our users with the aim of lessening polarisation and strengthening social cohesion. By gathering data from our Facebook pages, we can see how many people have hit the Comment or Reply buttons – an indicator of how often people engage with our content and with each other. But there’s no Facebook button for Constructive Comment or Reply. So, explains Snyder, “we have to unpack what we mean by constructive – the simple fact of replying to someone? But what if that reply is insulting? And are people engaging in dialogue or just voicing their own opinions without listening to others?”

Knowing where to look
Text analysis is a method that aims to qualify text in order to get a better understanding of users’ attitudes and behaviour online. Sentiment analysis for example can assign emotional weighting, such as negative, positive or neutral to comments. This information can then be categorised in a number of ways such as by topic or by format. Text analysis is useful when there is a lot of data available – over the course of a year there will be tens of thousands of comments on our platforms – and it’s not possible for an individual team member to read and contextualise all these comments. But text and data analysis can identify where we should look for insights. If analytics make it clear that a particular topic attracts a great deal of negative attention, our programme teams and content creators can then consider why that’s so. Some topics may simply by their very nature be negative – people are unlikely for instance to make positive comments about war. But it may be that a topic is taboo or controversial and then we can try different ways of approaching the topic to generate a more positive response.

Language matters
One of Snyder’s first research projects with RNW Media was an evaluation of a year’s worth of data from the Yaga Burundi platform which found that different topics were being discussed in different languages. Content was being published most often in French while the user-base speaks mostly Kirundi – almost 15,000 comments were in Kirundi and only 3,439 in French. Many Burundians neither read, write or speak French meaning much of Yaga’s content was inaccessible to them. However, the analysis also showed that when discussing topics like the Economy, Education, Media, Security and Sexuality, users prefer to use French. These findings showed that in order to include more people in the discussion, the Yaga team should carefully consider the language in which they choose to publish different types of content and develop strategies that open up the debate to as many people as possible. Yaga now publishes regularly in Kirundi as well as French.

Snyder’s recent analysis of RNW Media content published on the Chinese social media channel, Weibo, showed that there was no consistency across topics when comparing content published as text or as images. It’s a rule of thumb across digital media, that images will perform better than text – but the research in China showed this wasn’t the case and that users preferred text for certain topics. This means we have to ask ourselves whether users might prefer not to see images of certain topics – if a topic is sensitive an image may seem threatening whereas a user will find it easier to engage with text on the same topic.

Tried and tested
The commercial world has been applying data and text analysis for years in an attempt to better understand their customers. As a result, the kind of digital data mining and social listening we are doing has been well tried and tested. These aren’t common methods in the development sector, but they can be very effective for questioning our assumptions and understanding if our strategies are genuinely effective. Thanks to these digital approaches we can take monitoring and evaluation to the next level.