The Geneva Social Media Index (GSMI) is based on analysis of the use of Twitter, which is the most frequently used social media tool in diplomacy, politics and social developments. The GSMI balances Twitter activities and the impact these activities create. It aims to promote smart and impactful use of social media. The Index was developed by Dr Goran S. Milovanović, Data Scientist, DiploFoundation, for DiploFoundation and the Geneva Internet Platform.

[Read the full GSMI Report for Geneva Engage Award]

1. Introduction: Social Media in Digital Diplomacy

Social media and the continuous information and knowledge dynamics ...[1]), around 320 million active Twitter users monthly [2], and more than one billion daily active users of Facebook in September 2015 [3], digital diplomacy clearly needs to be focused on near real-time information dissemination and analysis. The World does not change only in terms of how many citizens – or how often - consume social media or contribute to them. With these numbers, thecomplexity of the human information environment increases tremendously. The task of digital diplomacy, among others, is to adapt to this fast paced and complex environment by developing appropriate digital strategies to reach its target groups, keep them informed (but not overloaded with information), and - as in the case of traditional diplomacy - ensure the most valuable representation of actors, be it states, international, or non-governmental organisations.

While many social media platforms are present today, Twitter still seems to occupy a specific position in the world of diplomacy, politics, and social developments in general. Of all social media, Twitter will be the most cited one in both traditional and online media when it comes to actual developments, and especially in times of crisis. It has almost achieved the status of the primary online channel used to disseminate information on ongoing critical social and political developments in real time. It forces its users to be concise and pack information in a dense textual format, and is probably the most frequently used source to analyze the online presence of various subjects in the contemporary context of big data and data science. Managing a successful representation on this social network is a challenging task indeed, for any digital strategist.

The GSMI is based on a simple and concise quantitative analysis of the social media activity on Twitter. The Index presents an attempt to establish a balance between (a) measuring the efforts to manage social media representationappropriately and (b) measuring the results achieved by such management efforts. It by no means favorites exclusively those Twitter accounts that simply tweet a lot, neither it scores them solely on the basis of numbers of followers or favorited statuses. By taking a multidimensional approach to the analysis of Twitter activity, the Index aims at compressing a multitude of information on that activity in its total ranking, while the whole GSMI methodology provides an in-depth review of the many sub-indicators that are comprised by the analysis of a particular Twitter account. Many of these indicators are derived by following the simple idea to estimate what was done relative to what could have been done online. Thus, the aim of the Index is rather to promote intelligent and original use of social media than to merely describe and rank social media activity and reception in quantitative terms. In January 2016, the GSMI was used to determine the winners of theGeneva Engage Award in three categories: permanent diplomatic missions to the UN in Geneva, Non-governmental organisations, and International organisations. The GSMI analysis for the Geneva Engage Award has covered 48 permanent missions to the United Nations, 43 non-governmental organisations (NGOs), and 43 international organisations (IOs), all located in Geneva.


Figure 1. Permanent Missions to the UN in Geneva Twitter accounts, their Twitter Activity and Reception (percentile ranks), total score (size), and the GSMI Contribution indicator (color). Contribution represents the proportion of original tweets in the total Twitter production the respective account.

2. Methodology

2.1. Data Acquisition

The Twitter accounts of 48 permanent missions to the UN in Geneva were identified from two public Twitter account lists: (1) Diplo Missions in Geneva (by International Geneva [4]), and (2) Diplo Missions in Geneva (by Twiplomacy [5]). The Twitter accounts of 43 NGOs and 43 IOs were identified from the following public Twitter account lists: Peace and Security, Economic affairs, Human rights, Global Health, Environment & SD (by International Geneva [6]), and international-development (by NonprofitOrgs [7]).  Other means (e.g. Twitter search, personal recommendations, expert knowledge) were used as well to identify relevant accounts. In cases where both the country’s mission itself and the permanent representative personally manage separate Twitter accounts, data were aggregated. When an IO had multiple Twitter accounts in different languages, the English language account was selected for analysis. For the UN system, the accounts of various departments and programmes were also included. No personal accounts (e.g. directors, program managers, public relations managers) were considered in the category of NGOs and IOs.

Twitter Search API [8] was then used to (a) collect relevant data on the permanent mission user accounts, and to (b) gather samples of tweets from the respective Twitter timelines. Because this is the first time the GSMI is computed for the three categories of accounts under consideration (permanent missions to the UN, NGOs, and IOs), all tweets that could have been retrieved from the Twitter Search API were taken into consideration [9]. This was done in order to obtain a solid baseline result for future comparisons. Data acquisition took place between 19 and 21. December 2015.

In order to assess the extent and depth of the social media presence, two sets of quantitative indicators were developed, addressing social media activity and reception. The computation of the indicators and data visualization were developed in the programming language R [10]. All Twitter accounts in the scope of this study were ranked according to their scores on activity and reception indicators to determine their success in social media management.

Twitter was accessed from inside the programming language R’s environment [11]and through the Twitter Search API. For each Twitter account, the following user account data were collected:

  • number of statuses posted since the registration of the account;
  • number of followers;
  • number of other accounts the account under analysis follows; number of public lists that list the respective actor’s account;
  • account lifetime (length) in weeks.

In addition to these data, the following were collected from the respective accounts’ timelines:

  • number of original tweets (excluding re-tweets from other accounts);
  • number of the re-tweets of the account’s original tweets;
  • number of times that the account’s original tweets were favorited;
  • number of replies made on the behalf of the account.

The collected data were combined to develop a set of social media activity and reception indicators.


Figure 2. The 50 most popular hashtags used by NGOs and IOs, correlations across accounts. The rank-order correlations from the account-term frequency matrix were computed; only significant (p<.05) and higher (r>.55) correlations are represented.

2.2. GSMI Indicators

2.2.1. Activity indicators

  • Production: number of tweets published since the registration of the account, including all re-tweets and replies, divided by the account’s current lifetime in weeks. Production is the most straightforward measurement of Twitter activity used in this study. It provides information on user activity per week.
  • Contribution: proportion of original tweets (account’s production after removing all re-tweets) relative to the total number of tweets retrieved from the account’s timeline. This criterion provides an assessment of the amount of new content provided by a particular account.
  • Responsiveness: proportion of replies to other Twitter users relative to the total number of original tweets. How often does the user of the account engage in communication?
  • Interest: number of Twitter followers (other Twitter accounts that follow the account under analysis), divided by the account’s current lifetime in weeks. Essentially, this is a weekly rate of engagement in following Twitter content and developments.

2.2.2. Reception indicators

  • Retweets received: total number of retweets of all original tweets made on the behalf of the account, divided by the number of original tweets. How much of the account’s original production is passed on to other users by the account’s followers?
  • Popularity: total number of the account’s tweets favorited by other users, divided by the number of original tweets.
  • Follower level: total number of followers on Twitter, divided by the account’s current lifetime in weeks. This is a weekly rate of audience gain.
  • Publicity: total number of public enlistments on Twitter, divided by the account’s current lifetime in weeks. Another form of rating weekly audience gain, this time based on the count of public lists that list the account.

2.3. Aggregation and ranking

All accounts were first ranked on both the activity and reception indicators. The aggregate activity and reception scores were calculated in three steps: (1) first by summing up the ranks of the indicators, then by (2) reverse-scoring the sums, and finally (3) by being expressed as percentile ranks. The total score for each account was computed in the same way - except that all eight indicators were aggregated at once.

3. Rationale of the GSMI

In the following figure, the activity percentile ranks are plotted against the reception percentile ranks. The size of the marker is proportional to a particular Twitter account’s total GSMI ranking, while the color scale represents the distribution of one of the sub-indicators, namely, contribution: the proportion of original tweets found in the account’s total Twitter production. The data set comprises the accounts of 86 NGOs and IOs.


Figure 3. GSMI sub-indicators. Activity vs. Reception percentile ranks. Data set: 86 Twitter accounts of NGOs and IOs.

If success in managing social media would be measured in terms of reception only, than many (potentially unaccounted for) factors influencing performance would turn out to be of decisive importance. Certainly, there are actors in the digital arena who gain a share of their popularity simply because they are traditionally influential, or because the cause they represent is of universal importance to many. A priori, there is nothing wrong in gaining popularity online by simply inheriting the visibility that was achieved offline. However, we wish to recognize not only those who are popular and visible, but those who own at least a significant part of their popularity and visibility to the efforts invested in smart social media management. In the upper left corner we can see a group of NGOs and IOs who score low on the activity sub-indicator of the GSMI, but nevertheless exhibit success in terms of reception. In the lower right corner, we find a group of Twitter accounts who, on the contrary, perform well in terms of activity, but score poor on reception. The two - activity and reception - are only slightly correlated [12]. The total GSMI score is derived from a combination of both sets of indicators, trying to balance out the possibility that some accounts score high in the total rankings by simply inheriting the reception they would receive no matter how actively their social media accounts were managed. In the upper right corner, we find those accounts who score high on both activity and reception, and that is what the index distribution places its highest total scores. Once again, the idea behind GSMI is to promote clever social media management and influence actors in digital diplomacy to try hard to reach the goals of good online representation: we are looking to recognize those who are ready to go beyond what they would probably achieve anyways.

NOTES

[1] Source: Internet World Stats, URL: http://www.internetworldstats.com/stats.htm, accessed on January 06, 2016.

[2] Source: Twitter, URL: https://about.twitter.com/company, accessed on January 06, 2016.

[3] Source: Facebook, URL: http://newsroom.fb.com/company-info/, accessed on January 06, 2016.

[4] URL: https://twitter.com/Geneve_int/lists/diplo-missions-in-geneva/members.

[5] URL: https://twitter.com/Twiplomacy/lists/diplo-missions-in-geneva.

[6] URL: https://twitter.com/Geneve_int.

[7] URL: https://twitter.com/nonprofitorgs.

[8] The documentation on Twitter Search API is found online on:https://dev.twitter.com/rest/public/search; the API was accessed from inside the programming language R, using the TwitteR package, documented on:https://cran.r-project.org/web/packages/twitteR/twitteR.pdf.

[9] The Twitter Search API does not return an exhaustive list of statuses published on the behalf of any Twitter account. The current limit on the number of statuses that can be retrieved from a particular timeline is 3,200 (cf. https://cran.r-project.org/web/packages/twitteR/twitteR.pdf). We were thus able to retrieve (a) all tweets from any particular account that did not publish beyond the current search constraint, and (b) a sample of 3,200 tweets for several accounts that did.

[10] The R Project for Statistical Computing: https://www.r-project.org/.

[11] TwitteR R package was used to query the Twitter Search API from inside R:https://cran.r-project.org/web/packages/twitteR/index.html.

[12] Linear regression is certainly not a method of choice to describe the relationship between activity and reception expressed as percentile ranks; the regression line in Figure 3. is included only to help the reader spot the weak positive correlation between the two components of the GSMI. In terms of Spearman’s R coefficient, R = .41.