Bridging the Gap: Can the World Governance Indicators Reliably Predict WJP Rule of Law Scores?

In the complex landscape of international development and political science, researchers often find themselves caught between two worlds: the desire for long-term, comprehensive data coverage and the need for high-precision, specific institutional metrics. Two of the most prominent tools in this field—the World Bank’s World Governance Indicators (WGI) and the World Justice Project (WJP) Rule of Law Index—are frequently used to gauge the health of a nation’s legal framework.

While the former offers the advantage of vast historical breadth and global reach, the latter is often lauded for its granular, survey-based precision. A persistent question has emerged among policy analysts: Is it possible to use the WGI as a reliable proxy for the WJP, not just for relative country rankings, but for the actual, cardinal levels of Rule of Law scores? Recent statistical analysis suggests that the answer is a qualified "yes," provided that researchers are willing to embrace a bit of non-linear complexity.

The Core Problem: Precision vs. Coverage

The WGI, a flagship product of the World Bank, provides a long-running time series that captures the perceptions of governance across hundreds of countries. Conversely, the WJP’s Rule of Law Index, while highly regarded for its methodology, covers a shorter temporal span.

When researchers seek to perform longitudinal studies on how the rule of law influences economic growth or social stability, they often prefer the WGI for its length. However, critics—including advanced AI models tasked with reviewing such methodologies—have frequently pointed out that confirming "general agreement" between the two indices is insufficient. The critical test is whether the WGI can predict the specific values of the WJP index. Predicting levels is a far more demanding task than merely predicting a country’s rank, as it requires the WGI to mirror the specific variance and distributional characteristics of the WJP.

A Chronology of the Inquiry

The investigation into this substitution began as a simple comparative study, intended to see if the two metrics were "close enough" for casual use. However, the feedback loop between the researcher and analytical tools (specifically, Claude) highlighted a methodological gap.

Initially, the study confirmed that the two indices agreed on the global hierarchy of lawfulness. But after being challenged to prove that the WGI could act as a quantitative substitute for the WJP’s levels, the research expanded. What began as a brief blog post evolved into a more rigorous statistical validation process, conducted during a research retreat in the Northern Territory.

The process involved:

1. Data Harmonization: Ensuring that country codes (ISO3C), regional classifications, and historical years were perfectly aligned between the two disparate datasets.
2. Cardinal Substitution Testing: Building a linear regression model to map WGI inputs to WJP outputs.
3. Validation: Utilizing Leave-One-Out Cross-Validation (LOO RMSE) to ensure that the model’s predictive capabilities weren’t simply overfitting to the existing dataset.
4. Bias Detection: Identifying and correcting for systematic inaccuracies, particularly at the extreme ends of the spectrum (the highest and lowest performers).

Supporting Data and Methodology

The analysis utilized the 2025 releases of both the WGI and the WJP historical datasets. The technical implementation, executed in R, prioritized the standardization of country names and the systematic removal of extraneous data, ensuring that the unified dataset was robust.

The Linear Model

At the outset, a simple linear model provided an R-squared value of 0.97, suggesting that 97% of the variation in WJP scores could be explained by WGI scores. While this is mathematically impressive, it is only the first step. The true test of a proxy is how it performs when predicting values for countries not currently in the training set.

The LOO RMSE calculations revealed a typical error rate of less than 3%. When scaled against the standard deviation of the WJP scores, this represents a mere 19% of the cross-country spread. For most policy-making contexts, this level of error is well within the acceptable margin, confirming that the WGI is a highly competent surrogate for the WJP.

The "Smile" Bias

However, a deeper look at the residuals—the difference between the predicted value and the actual observed value—revealed a subtle "smile" effect. The model was under-predicting the WJP scores for countries at both the extreme high end (very strong rule of law) and the extreme low end (very weak rule of law).

This systematic bias indicates that the relationship between the two indices is not strictly linear. When a country achieves an exceptionally high score in the WGI, the WJP score tends to climb at a slightly different rate than the model predicts.

Correcting for Non-Linearity

The solution, as applied in the study, was to introduce a quadratic term into the regression model. By accounting for non-linearity, the "smile" disappeared. The residuals leveled out, showing that the model’s accuracy became consistent across the entire spectrum. This adjustment is crucial for researchers: if one intends to use the WGI to estimate WJP scores, a simple linear model will suffice for the "average" country, but a quadratic model is necessary to maintain accuracy for nations at the extreme edges of the legal landscape.

Temporal Stability: Does the Proxy Drift?

A common concern with using one index as a proxy for another over time is "drift"—the possibility that the relationship between the two metrics changes as global standards evolve or as the indices update their survey methodologies.

To test this, the researchers compared a "flat" model (where the relationship between WGI and WJP is constant) against a "year-varying" model. Using an ANOVA test, the study yielded a p-value of 0.09. In the parlance of statistical significance, this is generally considered high enough to fail to reject the null hypothesis. In plain terms, the model with constant coefficients is sufficient; there is no compelling evidence that the predictive power of the WGI significantly decays or drifts relative to the WJP over the time period studied.

Implications for Policy and Research

The findings of this analysis have significant implications for the future of international development research:

1. Increased Efficiency: Researchers no longer need to fear the "missing data" problem when working with the WJP index. They can reliably extend their analysis into the past by utilizing the WGI as a calibrated substitute.
2. Enhanced Modeling: By applying the non-linear adjustment identified in this study, institutions can build more accurate models that account for the nuances of high-performing and low-performing legal systems.
3. Confidence in Historical Trends: The lack of significant temporal drift provides a green light for using these combined datasets to track the evolution of governance over decades.

A Cautionary Note on "Composite Precision"

Despite these positive findings, the study concludes with a necessary caveat. As the author noted, one should always be wary of the "illusion of precision" inherent in composite indices. Both the WGI and the WJP represent human perceptions and aggregate survey data; they are not absolute physical constants. While this analysis confirms that the WGI can act as a reliable proxy, it does not suggest that the WGI possesses a level of granular accuracy that the underlying data does not support.

Final Reflections on AI-Assisted Research

The development of this analysis highlights a growing trend in academic and policy research: the collaborative "feedback loop" between human experts and AI systems. The initial identification of the methodological weakness—the failure to address "levels" versus "rankings"—was prompted by an AI, while the technical execution and theoretical framing were completed by the human researcher.

This symbiotic process ensures that analytical blind spots are caught early, while the final output remains grounded in human judgment and contextual understanding. As datasets grow in complexity, the ability to rapidly test, validate, and refine statistical models using AI support will likely become a cornerstone of modern, high-quality economic research.

Ultimately, for those navigating the murky waters of international rule-of-law metrics, the path is now clearer: the WGI is a capable, reliable, and stable proxy for the WJP—provided you account for the curves at the ends of the road.