Measurement of Performance Indices or Misrepresentation of India?

According to the recently published (May 24, 2023) Global Slavery Index 2023 (GSI), compiled by the Australian human rights organisation Walk Free, six members of the Group of 20 nations have the largest number of people in modern slavery – either in forced labour or forced marriage. India tops the list with 11 million followed by China with 5.8 million, Russia with 1.9 million, Indonesia with 1.8 million, Turkey with 1.3 million and the United States with 1.1 million.

From the appearance of the first sovereign credit ratings in the 1930s, there is now a plethora of rankings for various performance indicators. Rankings of states, for qualities (ex. transparency), activities (ex. press freedom), policies (ex.corruption), are being used to measure everything from ‘happiness’ to ‘democracy.’ Such ‘performance’ indicators have lately proliferated and strive to act as a pervasive sort of social pressure, almost exclusively for developing countries. These rankings tend to simplify a complex reality while attempting to appear objective.

The Walk Free foundation has been coming up with its Global Slavery Index (GSI) for more than a decade now. But its findings are not free of controversy. According to an article  in Social Inclusion ‘The GSI aims to, among other objectives, recognize the forms, size, and scope of slavery worldwide as well as the strengths and weaknesses of individual countries. An analysis of the GSI’s methods exposes significant and critical weaknesses and raises questions into its replicability and validity.’ But what’s more troublesome is that the publicity given to the Index is leading to the ‘use of this poor data not only by popular culture and reputable magazines and news organisations […], but also by academic journals and high level policy makers […], which can lead to inaccurate policy formulation.’

Earlier, the 2023 ‘World Happiness Report’ was released and India was ranked at 126th position out of 137 countries. Published by the Sustainable Development Solutions Network to the United Nations (UN) these rankings are based on data from the Gallup World Poll, which measures how happy citizens perceive themselves to be. Webster dictionary defines happiness as a state of well being and contentment so an obvious problem with measuring happiness is that it means something different to different people. Yet Finland, almost always, remains the happiest country in the world, followed by Denmark, Iceland, Netherlands Sweden, Norway, Switzerland, Luxembourg and Israel. Apparently crisis ridden Sri Lanka and inflation hit Pakistan are happier than India. As is China despite the evidence of rare protests breaking out in multiple cities against the government’s zero-covid policy. And oddly ‘happiness’ even for countries at war, Russia and Ukraine, has been ranked higher than India in the World Happiness Report, with Russia ranked 70th and Ukraine ranked 92nd.  During an event in Bengaluru, External Affairs Minister Dr S Jaishankar dismissed the report and said that he “does not understand the method” to find these indexes which are clearly “mind games.”

A similar gloomy picture is painted by other international think tanks. Sweden-based Varieties of Democracy Institute (V-Dem) had said that India has joined the ranks of Afghanistan, Brazil and Myanmar in “witnessing the most dramatic increases” in political polarisation. V-Dem categorises India as one of the “worst autocratisers” of the last decade. The  annual report by Freedom House, the Washington-based pro-democracy think tank and watchdog rated India ‘partially free’ its  report for a third straight year, with a cumulative score of 66 out of 100, on parameters such as political rights and civil liberties. The Economist Intelligence Unit (EIU), a division of UK’s Economist Group categorised India as a “flawed democracy” in its ‘Democracy Index.’ Is there robust evidence that democracy is in decline in India? What is the measure of democracy?

Sociologist Dr Salvatore Babones of the University of Sydney believes that India is an ‘extraordinarily successful democracy.’ At the India Today Conclave in Mumbai in November 2022, he said that, “India is the world’s biggest democratic success story. It is the only postcolonial well-institutionalised democracy.” He has argued that in a statistical sense, India being the world’s ‘largest’ democracy is irrelevant because democracy is unrelated to country size. The relevant factor for India is that it is the world’s ‘poorest’ democracy. If we take peaceful transfer of power with free and transparent elections, as the most basic measures of democracy then India can be categorised as a well institutionalised democracy. It is the only democracy in the world with a GDP for capital of less than about $10000 per head. Other democracies at these levels of income exist in eastern Europe which became independent only about three decades ago, and had to become democratic in order to join the European union.  Truly, unambiguously indigenously developed democracies then exist in the West. India is also the world’s only post colonial state to remain a democracy throughout its entire 75 years of independence.

Indices have gained favour in the past two decades as indicators of  regional and national growth of a country, and have become widely popular symbols of multidimensional welfare. These indices are built on a set of indicators using the most recent published articles. Data is then usually submitted for an handpicked ‘expert’ consultation to validate the ascribed rankings, but not necessarily alter them. Scandinavian countries Denmark, Iceland, Netherlands Sweden, Norway invariably top the indices. That poor, developing countries tend to be at the bottom of these rankings is no longer surprising. It becomes pertinent to ask whether these indices reflect generalisations based on a small number of widely published but non representative media articles?

Rankings have significant limitations when it comes to recognising the achievements of countries beyond the Anglosphere. For instance, the Global Hunger Index 2022 released by Concern Worldwide and Welt Hunger Hilfe, Non-Government Organisations from Ireland and Germany respectively, has ranked India at 107 among 121 countries. The report chooses to deliberately ignore efforts made by the Government to ensure food Security for the population especially during the Covid Pandemic. Three out of the four indicators used for calculation of the index are related to the health of Children and cannot be representative of the entire population. The fourth indicator estimate of proportion of undernourished population is based on an opinion poll conducted on a very small sample size of 3000, in which respondents were asked questions like ‘During the last 12 months, was there a time when, because of lack of money or other resources: You were worried you would not have enough food to eat? You ate less than you thought you should?’

Another example is the Climate Change Performance Index (CCPI) 2023 which has ranked India in 8th place. India is the only G20 country in the top ten climate change performers in this index the first three ranks were purposely kept left out as no country performed well enough to be placed as first, second or third,  Denmark And Sweden were placed on fourth and fifth place respectively.  Compare this to the Environment performance index 2022, measured by Yale and Columbia University which has ranked India at the very bottom of the list at 180, after Vietnam (178), Bangladesh (177), and Pakistan (176). The dichotomy in the ratings indicates a clear unscientific bias in the methodology.

Human Development Index for 2021-22 released by the UN developed program UNDP, India was ranked 132nd out of 191 countries. World Press Freedom Index, 2022 released by Reporters Without Borders ranked India at 150th out of 180 countries. Even Hong Kong, where Beijing has led a brutal crackdown to punish critics and silence dissenters, fundamentally altering the life for Hong Kongers, ranks above India at 148.

Organisations that build indexes are reluctant to share the data through which they arrive at their conclusions. The Union government tried to engage with the Economist Intelligence Unit (EIU) which has ranked India on the 53rd spot and as a ‘flawed democracy’  seeking details on the parameters used in the rankings. However, according to a report in the media, the EIU did not share the methodology adopted to deduce the rankings. Most of these rankings can be criticised on methodological ground with the need to improve indicators and the underlying calculative operations. There is a possibility that growth systematically mis-measured or measured less reliability for developing countries. For instance, growth is more volatile in countries where agriculture constitutes a large part of the economy. Numerous index designs have been put forward, yet little is understood about their reliability. Ideally an index construction involves decisions related to indice selection, scale of analysis, measurement errors, data transformation and weighting – each of which is imbued with uncertainty due to decisions made by the index developer.

Rankings hold interest for multiple audiences, and whether we think they should, or not it seems there is an appetite for them. Factors like culture, language and resources strongly influence the rankings and buttress the dominance of the global north. The only remarkable thing about these global rankings is that they are so unremarkable, reinforcing our intuitive expectation that European, especially Scandinavian countries rank high, and countries from the global south figure lower.  Suffice to say that they are neither scientific nor multilaterally validated.

Vaishali Basu Sharma
Vaishali Basu Sharma
Vaishali Basu Sharma is an analyst of strategic and economic affairs. She has worked as a consultant with India’s National Security Council Secretariat (NSCS) for nearly a decade. She is presently associated with New Delhi based think tank Policy Perspectives Foundation.