Country Coding Considerations for Dataset Harmonization and Applied Uses

Introduction

Managing country codes is one of the most difficult tasks in environment-security analysis, and each dataset treats the concept differently. Country code designations may be very strict and only respect internationally recognized sovereign nations starting with their first official day of independence. Other datasets may include separate observations and codes for territories, disputed states, colonial nations, and other quasi-independent states. Coding systems like the Correlates of War¹ or Gleditsch and Ward² fall into the former with strict, to the day, international recognition, while refugee and asylum datasets provided by the United Nations³ falls into the latter. UN migration and refugee datasets include nearly every conceivable territorial designation/disaggregation (e.g. Puerto Rico, Taiwan, Faroe Islands, etc.).

When using an individual dataset, users need a general awareness for the conceptual approach to the country code specification and any potential inconsistencies, however, when attempting to harmonize multiple datasets, more intricate knowledge is required. Merging multiple datasets without first addressing disparate approaches to country coding will lead to inconsistencies, numerous NA values, and compound downstream error in the analysis. It takes time and experience to anticipate country coding issues and know what to be on the lookout for when undergoing project planning. The goal of this vignette is to outline common issues and considerations for harmonizing multiple country-year datasets. This walkthrough uses the Varieties of Democracy (V-Dem)⁴ and Correlates of War coding schemes to illustrate recurrent challenges. They are good datasets to work with while learning for several reasons:

• They represent opposite ends of the country-coding ideological spectrum.
  • Because V-Dem is focused on polities, it is highly disaggregated and observes long standing claims of independence.
  • Conversely, CoW is very strict and adheres to official and internationally recognized claims of independence.
• Both datasets are consistent and have little to no errors and inconsistencies. This is helpful for learning, but not entirely representative of real world experiences. On top of differences in coding ideologies, many datasets have a slew of errors and inconsistent applications that further complicate pre-processing.
• They both have direct programmatic access via R that makes management, exploration, and visualization easier.
  • The vdemdata package.⁵
  • The states package⁶ featuring CoW and G&W code access and manipulation.
  • The cshapes package;^7,8 vectorized historic state boundaries using either the CoW or G&W schemes.
• V-Dem provides reference materials that are extremely helpful for learners.
  • The Country Coding Manual and Historic Name column in the raw data provide a lot of context for the year to year regime changes, and their reasoning for important decisions.

V-Dem Country Codes

The Varieties of Democracy (V-Dem) presents a collection of highly disaggregated indicators (400+) depicting wide-ranging measures of democracy and institutional characteristics dating back to 1789. In contrast to several other political, social, and environmental datasets, V-Dem demonstrates an extreme level of transparency in their methods and copious documentation. This includes a dedicated supplementary manual depicting their country coding approach; V-Dem Country Coding Units v11.1.⁹ This tutorial will highlight the most relevant (in my opinion) portions of their approach and country-year coding decisions.

V-Dem defines a country as

…a political unit enjoying at least some degree of functional and/or formal sovereignty.

Generally speaking, V-Dem will have a country-year observation for a country if:

1. The state made a formal declaration of independence; even if not yet fully recognized by the international community.
2. In modern times this concerns states like Kosovo, Taiwan, and Western Sahel, in historic times this concerns states such as Colonial Asia and Africa.
3. If the state in question operates with some degree of autonomy that distinguishes itself, at least in its polities and institutions, from the parent nation.

In my limited experience with V-Dem, this conceptual approach is carried out very consistently, but additional pre-processing is required to successfully merge with more strict datasets. Moreover, the Country Coding Units Manual contains information that helps users construct historical time series of nations that have limited independent observations in the dataset. For example, Bangladesh is coded independently starting in 1971, but to construct a longer historic record, the Country Coding Units Manual states that the following combination of observations may be used:

1. India (Princely state of British India (1910 – 1947))
2. Pakistan (1947 - 1971)
3. Bangladesh (1971 - )

The Country Coding Units Manual is a great reference if you need to combine historical democracy indicators for nations with intermittent internationally recognized sovereignty with other historically disaggregated environmental, migration, or conflict data.

The histname variable in the raw V-Dem dataset is also an excellent source of context for the true nature of regime changes in annual polity. While the country_name field is static and typically reflects the modern, common, country name in the V-Dem dataset, the histname reflects official or historic names and potential states of occupation. Take Somalia for example:

This provides a clear understanding of Somalia’s historical sovereignty and polity for the past 120 years. When combined with the Country Coding Units Manual, the user is able to make more informative choices regarding their research interests. It also serves as excellent reference material for historical nation-states that is much faster than being sucked into a Wikipedia black hole.

Now we will walk through some of the more important country coding considerations for applied cases using Varieties of Democracy and Correlates of War for 1900-2020.

The Americas

Europe

The Balkans

The Balkans are consistently a source of frustration when attempting to merge disparate datasets. The affected states include Serbia/Yugoslavia, Bosnia and Herzegovina, Kosovo, Croatia, North Macedonia, Slovenia, and Montenegro. V-Dem disaggregates each component for as long as possible while CoW uses the official, internationally recognized, aggregated states. Because of this, when merging with most datasets, V-Dem will require some additional processing. At a minimum this may include averaging across multiple states to calculate Yugoslavia or the State Union of Serbia and Montenegro. Given time and resources, users may also consider calculating weighted averages using population data as not to give Kosovo or Slovenia equal weighting with Serbia/Yugoslavia or Montenegro. The V-Dem Country Coding manual is very helpful in reconstructing the complicated historic record of these nation-states.

• Montenegro
  • V-Dem (1789-1918; 1998-2019)
  • CoW (2006- )
• Macedonia
  • V-Dem (1991- )
  • CoW (1993-)
• Croatia
  • V-Dem (1941-1945 Nazi puppet state, 1991-)
  • CoW (1992-)
• Yugoslavia / Serbia
  • V-Dem (1804-)
  • CoW (1878-) no observations for 1942-1943. CoW maintains the YUG character code during the State Union of Serbia and Montenegro (2003-2006) and Republic of Serbia (2006-).
• Bosnia and Herzegovina
  • V-Dem (1992-)
  • CoW (1992-)
• Kosovo
  • V-Dem (1999-)
  • CoW (2008-)
• Slovenia
  • V-Dem (1989-)
  • CoW (1992-)

Visual timeline of significant secessions from the former Yugoslavian state as internationally recognized and coded by the Correlates of War and cshapes project. The top panel depicts Yugoslavia (highlighted teal) in its 1989 unified state. Highlighted countries in remaining panels illustrate countries that gained sovereignty since the previous date.

Middle East and North Africa

Yemen(s)

CoW includes separate codes for the Yemen Arab Republic/North Yemen (678; 1926-1990), unified modern Yemen (679; 1990-), and Yemen People’s Democratic Republic/South Yemen (680; 1967-).

Similar to the Czech Republic, V-Dem tracks historic, North, and modern Yemen on a single code (14; 1789-1850 and 1918-). They provide South Yemen a separate code (23; 1900-1990). V-Dem’s greater historic record for South Yemen includes only the city of Aden and its immediate surroundings from 1900-1963. There is no CoW equivalent for South Yemen/Aden during this time.

Visual timeline of internationally recognized sovereign borders of Yemen, North Yemen, and South Yemen. Historic boundaries are determined by the cshapes packages. Note that cshapes does not include land area that lacks international independence. Hence, in the 1946 (left) panel, British Aden (South Yemen) and Djibouti are not present until they’ve gained their respective independence.

Non-MENA Africa

There are not many complicated distinctions throughout Africa. V-Dem codes most African nations independently starting in 1900 while indicating their colonial protectorate. Depending on the colonial parent nation, most African countries acquired internationally recognized independence between 1955-1975; this is when the CoW record begins. Some notable African countries:

• Somaliland
  • Recognized by V-Dem as an independent state from 1900-1960 and again starting in 1991-.
  • CoW (and several other datasets) does not include Somaliland.
• Ethiopia
  • Coded continuously by V-Dem starting in 1789; including Italian occupation from 1936-1941.
  • Coded intermittently by CoW (1898-1936; 1941-); excludes Italian occupation.
• Sudan and South Sudan
  • They are coded identically by V-Dem and CoW; a separate code for South Sudan starting in 2011.

Asia

V-Dem includes a greater historical record for most Asian countries, because, similar to Africa and the Caribbean, there was less centralized control exercised over their polities and institutions during colonial rule. As a result, most Asian states are recorded in V-Dem beginning sometime 1789-1900, and in CoW during one of the waves of colonial (1945-1950) or regional (1971-1975) independence. However, there are a few nation-states with more complicated records in V-Dem and CoW: