The production of the WJP Rule of Law Index® may be summarized in ten steps:
- The WJP developed the conceptual framework summarized in the Index’s 9 factors and 48 sub-factors, in consultation with academics, practitioners, and community leaders from around the world.
- The Index team developed a set of five questionnaires based on the Index’s conceptual framework, to be administered to experts and the general public. Questionnaires were translated into several languages and adapted to reflect commonly used terms and expressions. These instruments were piloted in six countries in 2008.
- The team identified, on average, more than 300 potential local experts per country to respond to the qualified respondents’ questionnaires, and engaged the services of leading local polling companies.
- Polling companies conducted pre-test pilot surveys of the general public in consultation with the Index team, and launched the final survey.
- The team sent the questionnaires to local experts and engaged in continual interaction with them.
- The Index team collected and mapped the data onto the 48 sub-factors.
- The Index team constructed the final scores using a five-step process: a. Codified the questionnaire items as numeric values. b. Produced raw country scores by aggregating the responses from several individuals (experts or general public). c. Normalized the raw scores. d. Aggregated the normalized scores into sub-factors and factors using simple averages. e. Produced the final rankings using the normalized scores.
- The data were subject to a series of tests to identify possible biases and errors. For example, the Index team cross-checked all sub-factors against more than 60 third-party sources, including quantitative data and qualitative assessments drawn from local and international organizations.
- A sensitivity analysis was conducted by the Econometrics and Applied Statistics Unit of the European Commission’s Joint Research Centre, in collaboration with the Index team, to assess the statistical reliability of the results.
- Finally, the data were organized into country reports, tables, and figures to facilitate their presentation and interpretation.
A detailed description of the process by which data is collected and the rule of law is measured is provided in the document below.