COVID-19 Global Response Index Methodology

The Foreign Policy Analytics (FPA) team took a specific methodological approach in constructing the COVID-19 Global Response Index. The overall government response score, also referred to as the overall Index score, comprises three categories: COVID-19 public health directives, financial responses, and fact-based communications and accountability by political leaders. There is also a score that measures a country’s pre-pandemic conditions, including socio-economic and healthcare access and quality; this score provides a means to estimate countries’ capacity to respond to the COVID-19 shock. Finally, there is a score that measures a country’s status in terms of COVID-19 case and death rates, and rate of positive test results, as of the last date that data for the Index was pulled.

---

Scores and Normalization

All the scores are on a 0–100 scale, with 100 indicating positive outcomes (e.g., supportive debt-forbearance policy, comprehensive contact-tracing policy). The scales are normalized to the data, using minimum-maximum normalization. The “min-max” methodology is commonly used in Index construction to transform indicators expressed in different units into a common scale while preserving relative distance between values. This approach assigns the lowest value in the data a score of 0, the highest 100, with the rest falling in between.

Specifically, for any given value x, the calculation to obtain the normalized x is: (x-minimum)/(maximum-minimum) x 100. For instance, if the highest healthcare access score in the Index is 85, the minimum is 10, and a particular country’s value is 50, its normalized value is: ((50-10)/(85-10)) x 100 = 53.3.

In cases where worse outcomes are associated with higher scores (e.g., infant mortality rates, COVID-19 death rates), we rely on reverse-normalization. For instance, the highest death rate would normally be assigned a 100 but gets reversed to a 0. This procedure simply involves subtracting the normalized score from 100 for the reverse-normalized score.

---

Pre-COVID-19 Conditions

This score is composed of several categories: national debt, infant mortality rate, hospital beds per 100,000 people, GDP per capita, inequality, and healthcare access and quality. These factors are included as they provide an indication of the socio-economic strength and health security of a country and have data that is systematically reported and reliably tracked. For each of these, we take the actual values in each country and normalize those values using the min-max method. We then take the average of all the normalized scores to obtain the pre-COVID-19 conditions overall score. Finally, we normalize those scores for ease of comparisons.

For example, the highest average score in this category was 84.3. If this is adjusted to be a 100, and the lowest (24.3) adjusted to 0, it becomes easier to interpret how countries performed relative to one another. Thus, a country with a score of 100 in this category was in the strongest relative position based on those metrics prior to COVID-19 impact, compared to others in the Index; a country with a score of 0 was, likewise, the worst situated to handle COVID-19. Countries’ pre-COVID-19 scores do not factor into the overall Index score.

• Debt As a Share of GDP: This factor is used to measure how much flexibility a country may have had to pass fiscal policies to address challenges resulting from COVID-19. Countries with high shares of debt relative to their economies’ overall GDP, in general, are assumed to be less able to readily allocate resources to help mitigate the impact of COVID-19. 2019 debt statistics were sourced from the International Monetary Fund’s Global Debt Database; the measure represents a combination of central government debt and general government debt. In most cases, countries only report one or the other (non-federal systems, for example, would only have central government debt), but in the case of federal systems reporting both, we use the higher number.
• Infant Mortality Rates (IMR): IMR is used as a proxy measure for a population’s health— countries with high infant mortality rates routinely have high degrees of unmet needs, including healthcare, welfare, sanitation, nutrition, and education. Infant mortality data was sourced from UNICEF’s most recent State of the World’s Children report and measures the number of deaths per 1,000 live births, as of 2018.
• Healthcare Capacity: Hospital beds per 100,000 people were used as a proxy measure for the capacity of a country’s healthcare system. Limited availability of hospital beds has constrained countries’ abilities to manage an influx of patients, a factor that has emerged as a serious issue during this pandemic as hospital systems around the world have been overwhelmed with COVID-19 patients. The 2019 Global Health Security Index, which pulls this data from a combination of World Bank data and country data, was used as a reference.
• National Wealth: National GDP per capita is incorporated as a measure of wealth, an approach routinely used in social science research. Richer countries should, on average, have more resources available to address challenges associated with COVID-19. 2020 GDP per capita data, which was based on projections and sourced from the October 2019 IMF World Economic Outlook Database, is in current prices, using purchasing power parity in international dollars.
• Inequality: Gini coefficients are used as a measure of inequality in a country. As a Gini coefficient approaches 100, it suggests that wealth disparities in a country are very high. While GDP per capita addresses overall wealth, countries with high levels of inequality likely have major pockets of poverty. Based on studies and analysis to date, poorer, less economically stable populations are at particular risk for COVID-19, from having fewer resources, commonly living in close quarters (with poorer sanitation), and working in jobs that cannot be performed from home. Gini coefficients were incorporated into the methodology in an effort to account for these populations, and countries with greater inequality are likely to be more vulnerable to COVID-19, compared to similar countries with less inequality. Each country’s Gini coefficient is based on the most recent year of data available for that country, as not every country has this measure available every year. The data is drawn from the World Bank.
• Health Security: Finally, the Healthcare Access and Quality (HAQ) Index is utilized as a measure of health security in a country. Countries with accessible quality healthcare have systems in place that would be best positioned to handle an increase in patient needs after COVID-19 reached them, including the ability to handle the rise in healthcare needs that are being neglected due to COVID-19. The HAQ Index scores countries from 0 to 100 by using death rates for thirty-two causes of death that could have been prevented with timely and effective care. The data, routinely cited, is from the Institute of Health Metrics and Evaluation and is provided through 2015. While this data point is more dated, it is, in our view, the best estimation of the concept of healthcare access and quality we aim to measure.

---

Government Response Policy Score

The core of the Index aims to score the policies and actions that national leaders have taken in their efforts to mitigate the effects of COVID-19. The overall country score comprises three different components: public health directives related to COVID-19, financial response to COVID-19, reliance on facts, and enabling press freedom to ensure communication of science-based information informing the public’s actions and public accountability of leadership during the COVID-19 pandemic. The overall government response score takes each of these three scores and averages them. Finally, these average scores are then normalized in the same min-max approach to produce the overall government response score. Hence, the country with the highest average score in the Index will have a value of 100.

---

Time Dynamic for Policy Scores

For policy measures where time is relevant (e.g., when a country implements a stay-at-home order versus the most current testing rate per capita), the Index makes an adjustment to account for these differences. If two countries implement the same policy, but one implements it very early in that country’s crisis, that country will receive a higher score. In light of that, the Index uses quartiles of case rates to account for this difference. It references the case rate for each country on the date its policy was implemented, then divides those rates into quartiles. If a country implemented its policy in the first quartile (i.e., very early in their crisis, when its case rate was very low), its coded score is boosted by one standard deviation of the coded scores to account for early action. If a country implemented its policy in the fourth quartile (i.e., relatively late), it loses one standard deviation from its score. For coded scores in the second or third quartile, no adjustments are made. After factoring for time, scores are normalized using the min-max method for the subcategory’s final score.

For example, if a country is assigned a score of 2 for its school closings, its case rate on the date of implementation is in the first quartile, and the standard deviation for school closings is .5, its time-adjusted score would be: 2 + (.5) = 2.5. This score would then be normalized via the min-max method.

One general note about case rates: If there is not a case rate on a specific date of a policy for a country, the Index relies on the nearest date. For example, if a country implemented a policy on March 12th, but said country’s first case rate was reported on March 18th, the Index uses the data from March 18th.

---

Time of Policy Measurement

The Index accounts for three different time measures for each policy in most cases: the initial government response, the maximal government response, and the current government response. It weights the initial response time measure highest (50 percent of the final score), given the importance of early initial response to the pandemic. As such, for policy decisions, where time is relevant, the Index uses the first policy that a government implements in a specific issue area and allows ten days for that government to accelerate its response. The Index uses ten days as a reference, which was determined in consultation with public health experts as a reasonable threshold, to allow time for country leaders to adjust their initial policies if they were not sufficient to handle the mounting crisis, but not too much time to miss a critical point of intervention in the early phase of a country’s crisis. The calculations factor the more stringent measure and date it was implemented. If a country has no clear policy in place, but implements its first measure weeks or months into the crisis, the action is accounted for as a late intervention, factoring for time utilizing the same approach as described above. Other factors, including emergency healthcare spending, are accounted for similarly and are based on total spending within the first ten days of the initial amount spent.

The second and third time measures each account for 25 percent of the final score. The maximal response measures the most stringent policy a government implements for a specific issue area, and then accounts for time in the same manner noted above (examining quartiles for case rates on the date of implementation). The current policy measure takes the government’s policy on the date of the Index update, and the rolling seven-day average of new cases/capita. For both, it assigns each country into quartiles, and then it subtracts the policy quartile from the new cases quartile. Countries with very few new cases are thus not penalized for having relaxed policies months into the epidemic, whereas those that have scaled back policies despite having high cases have their scores negatively impacted. This measure does not adjust for time like the other two, since it is evaluating countries at exactly the same time (i.e. date of the new Index update).

For each of these three time measures (initial, maximal, current), scores are normalized using the min-max method. The Index then takes the average of all three, with the weighting described above (50 percent for initial response, 25 percent for maximal response, 25 percent for current response), and normalizing that score using the min-max method. This final score is the score for the subcategory.

For emergency healthcare spending, the Index uses the initial response (60 percent of the total score) and the total spending as of the most recent data pull, minus this initial spending (40 percent of the total score). We opt for a 60/40 split because early responses are, again, critical, but since most countries have had high cases for months, spending beyond the early days of a country’s crisis is also critical to monitor. For testing/capita, COVID-19 stimulus, and all measures in the fact-based communications category, the Index only uses total or current data and does not account for the three time periods. All other policy measures account for the three time periods described above.

---

Accounting for Low Cases

While many countries have implemented a range of stringent policy measures to limit the impact of COVID-19, others that took early action and/or have not experienced a spike in cases had a more muted response. Most notably, South Korea and Taiwan did not implement strict lockdowns, instead relying on particularly strong contact tracing to control the spread of the virus. Unfortunately, the contact tracing data does not differentiate countries’ responses enough to utilize it effectively. Instead, the Index examines case rates in conjunction with policies implemented. Specifically, if a country has a relaxed, or no, COVID-19-specific policy (1st quartile), coupled with a very low case rate at the time of that policy (1st quartile), the Index “rewards” those countries by utilizing the higher of its normal score or the average of the highest quartile score for that policy. The measure is incorporated so as to not unduly penalize countries for not taking more aggressive action when their low case rates would reasonably not prompt them to do so.

For instance, if a country’s normal code for a policy is a 1, the average for the 4th quartile for this policy is a 2.75, and this country’s policy score falls in the 1st quartile and its case rate for the date of policy implementation is also 1st quartile, it would be coded as a 2.75 for that policy, which would then be normalized using the min-max method.

In cases where countries did not implement any policy, we use the case rate on the date of their contact-tracing policy, given the use of strong contact tracing as a proxy for other policies. This approach allows the Index to not penalize countries for not taking aggressive policy measures when relatively low COVID-19 cases would not have compelled them to do so. However, countries that implement strong policies with low cases still score higher on this measure, as they get a boost from implementing those policies early, whereas countries with relaxed policies do not.

In subcategories where there is not sufficient policy variation among countries (school closings and cancelling public events are the two policies where this happened, as almost every country implemented the same level of policy), there is no difference among quartiles in terms of policy scores (meaning the average policy score for each quartile was the same), and the Index does not implement the adjustment for countries with relaxed policy and low cases.

---

COVID-19 Public Health Directives Score

The first of three government response measures examines a country’s public health directives implemented to combat COVID-19. This score and the other subcategories were normalized using the min-max method, following the time-adjustment explained above, where relevant. The subcategories are: stay-at-home orders, school-closings, public-gathering restrictions, cancellation of public events, testing policy and testing rates, travel restrictions, emergency healthcare expenditures, and contact tracing. The data was sourced from the University of Oxford.

• Stay-at Home Orders: Orders were accounted for by utilizing data from the Oxford University Coronavirus Government Response Tracker. The stringency of the orders is coded on a 0–3 scale, with 0 being no measure, and 3 requiring people to stay home with minimal exceptions. Strict lockdowns were clearly beneficial to limiting the spread of COVID-19.
• School-Closing Policy: Policies were coded on a 0–3 scale, with 0 indicating no measures, and 3 reflecting school closure mandates.
• Public-Gathering Restrictions: Gathering restrictions were coded on a 0–4 scale, with 0 indicating no restrictions, and 4 indicating restrictions of gatherings of ten or more people.
• Cancellation of Public Events: Directives are coded on a 0–2 scale, with zero reflecting no measure, and 2 indicating broad-based cancellations. Restrictions on public gatherings have been widely recognized as limiting the spread of COVID-19.

  For cancelling public events, several countries did not have reported case rates on the day of their policy implementation (Kenya, Indonesia, New Zealand, and Taiwan). All of those countries had rates near the date of their policy, with no difference between rates surrounding the actual policy date, so we used those instead.

• Testing Policy and Testing Rates: Each country’s testing policy is coded on a 0–3 scale, with 0 reflecting no testing policy, and 3 reflecting open public testing available to anyone. Countries with more targeted and broad-based testing policies are likely to test the asymptomatic or not severely ill, which would, in theory, help to minimize spread. The Index includes data points for COVID-19 tests per 1,000 people, using the Our World in Data COVID-19 data.

  Several countries had issues with dates with this measure. Ethiopia, Hungary, and Turkey had initial case rates that were reported after these policy implementations, but those rates were all 0, so we accounted for that. Saudi Arabia and South Africa reported case rates on days surrounding the policy date, and those rates were same (again, 0), so we used that.

• Emergency Healthcare Spending Per Capita: In U.S. dollars, also from the Oxford data. (We use Worldometer, which uses the United Nations Population Division estimates, for current population measures.) Emergency healthcare spending includes, but is not limited to, spending on personal protective equipment (PPE) and other COVID-19-related medical supplies. This measure captures spending within the first date of any emergency healthcare spending for each country and ten days after that, consistent with the time measurement for the other policies. Germany represented an outlier on this measure, spending considerably more than all other countries and significantly skewing the scores. To account for this and to minimize the extreme data dispersion, the square root of the per capita spending was used. We initially attempted a log transformation; while this normalized the data somewhat, the dispersion was still substantial, so we opted for the square root transformation instead. This transformation produces data that still clearly differentiates the highest spenders from others, but it creates a score distribution that is slightly less extreme.
• Travel Restrictions: This measure is coded on a 0–4 scale, with 0 reflecting no restrictions, and 4 reflecting a ban on all regions or a border shutdown. While we recognize that there have likely been some political motivations for some travel measures, particularly border shutdowns, it is also highly likely that many of these restrictions have been intended to limit the spread of COVID-19.

  Ethiopia, Ghana, Kenya, Turkey and South Africa’s travel restrictions predate their first case rate by several months (0 for all)—we use 0 for them accordingly.

• Contact Tracing: Data for this measure was sourced from Oxford and coded on a 0–2 scale, with 0 reflecting no contact tracing, and 2 indicating comprehensive contact tracing. Contact tracing has been identified by public health experts and epidemiologists as being vital to mitigating the spread of COVID-19, and countries with the most sophisticated forms of contact tracing (particularly Taiwan, South Korea, and Germany) have been able to keep their case and death rates low without implementing as many other extensive public health policies.

  Ethiopia, Ghana (both by one day), Hungary, Saudi Arabia, and Turkey implemented contact-tracing policies before their first data entry for case rate (all 0).

---

COVID-19 Financial Response Score

The second of three government response measures examines each country’s financial policies implemented to address COVID-19. Each of these subcategories is normalized using the min-max method, following the time-adjustment explained above, where relevant. The subcategories are: stimulus size, income-support policy, and debt-forbearance policy. University of Oxford data was utilized for these measures.

• COVID-19 Stimulus Packages: This measure assesses the size of a country’s stimulus package relative to its economy. Countries with more generous stimulus packages have devoted more resources to mitigating the crisis as an indication of their commitment to improving the situation.
• Income Support: This measure was coded on a 0–2 scale, with 0 reflecting no income support, and 2 reflecting government replacement of 50 percent or more of lost salary for workers.
• Debt-Forbearance Policy: This measure was coded on a 0–2 scale, with 0 reflecting no freezing of financial obligations, and 2 reflecting broad debt and contract relief.

These measures are assumed to mitigate the financial burden on many in countries impacted by COVID-19, and to enable more people to stay at home longer.

As noted above, due to aggressive early response policies, a number of countries have been able to maintain a very low case rate and have not needed to implement as extensive a financial response. To account for this effect, for countries that rank in the 1st quartile for case rates (low number of cases per population) on the policy dates and in the 1st quartile for policy (not very stringent), they were coded with the higher of two numbers: either their normal coding, or the average number of the highest quartile for that specific policy. This is then normalized, along with the other values, using the min-max method, for the overall score in the subcategory.

---

COVID-19 Fact-Based Communications and Accountability

The third of three government response measures examined a national leadership’s communication of fact-based information with the public and freedom of the press with respect to COVID-19. No other assessment of governments’ responses to COVID-19 incorporates such a measure. It was considered critical for inclusion, given the impact that national leadership’s public communication has on individuals’ and groups’ actions that ultimately contribute to or mitigate the virus’s spread. Each of the two factors in this subcategory is normalized using the min-max method. Here both examples require reverse-normalization.

• Fact-Based Public Communication: For this measure, Lexis-Nexis news searches were conducted to identify statements made by key leaders, specifically executives and health ministers who have autonomy to make policy and influence national decisions related to COVID-19 in each country. Eight specific topics associated with disinformation related to the coronavirus were measured, including: claims that it is caused by 5G, claims that it is a man-made weapon, claims that it is like the flu, claims that it can be cured by unproven drugs or remedies, actions or claims downplaying the importance of social distancing or personal protective equipment (PPE), false claims about its spread or containment in-country, actions to spread disinformation about it globally, and claims that it will be gone soon or is not a major risk. If a national leader made any claims or acted based on any of these claims, the country would be coded as a 1 (0 if they did not). Scores were cumulative up to 8, based on the number of claims. Leaders who have consistently been communicating with the public with fact or science-based information received a score of 0; those engaging in each type of claim were scored an 8. Scores were then reverse-normalized.
• Press Freedom Regarding COVID-19: For this measure, the Index utilizes unpublished data from the International Press Institute that is part of IPI’s publicly available COVID-19 press tracker. IPI has tracked claims of press limitations in every country, relying on a global network of editors, journalists, and media executives, and placed everything into several categories. The Index codes IPI data based on whether countries engage in any of four specific media limitations based on COVID-19: arrests or charges of journalists, restrictions on access to information, censorship, or fake news regulations. Countries that do not engage in any of these actions get a score of 0, and those engaging in all of these get a score of 4. This score is then reverse-normalized.

---

In-Country COVID-19 Status

The last score incorporated into the country profiles, but not an explicit factor in the Index itself, pertains to the current status of COVID-19 in countries as of October 1, 2020, the date when the last round of data for the Index was pulled. It comprises four measures: the death rate per capita, the case rate per capita, the positivity rate of testing, and the change in positivity rate over time. It will be updated periodically, along with all of the factors in the Index. Data for this score comes from the Our World in Data website.

• Death Rates and Case Rates: The COVID-19 death rate per capita measures the number of confirmed COVID-19 deaths per 1 million people in a country, and the case rate measures the number of confirmed COVID-19 cases per 1 million people in a country. This data is adjusted for population size to put the scope of the crisis in context. For example, 1,000 deaths in Italy are of greater magnitude relative to the size of the population than the same number of deaths in the United States, as the latter has a population more than five times the size of the former.
• Positivity Rates: The positivity rate measures the percent of COVID-19 tests that resulted in a positive result. This rate is noted as the “most reliable way to determine if a government is testing enough,” according to the Johns Hopkins University Coronavirus Resource Center. A high positivity rate suggests that a country is only testing its sickest population. The World Health Organization suggested that a positivity test rate of 10 percent is adequate and that a positivity rate below 5 percent for fourteen days is needed before relaxing social distancing. Countries that are not testing widely will likely have far more asymptomatic patients out and potentially spreading the virus. The change in the positivity rate is also measured from the previous month (this specific measure was, accordingly, first made available with the second round of the Index) and used in the final calculation of the category score. Countries that are making major improvements in their positivity scores will likely be much more able to minimize the spread. For purposes of the Index, all of these scores are reverse normalized.