From Manual Sifting to Algorithmic Precision: How Bellingcat Revolutionized Civilian Harm Tracking

Between February 2022 and September 2025, the digital landscape of the war in Ukraine became an unrelenting deluge of information. For open-source intelligence (OSINT) researchers, the task of documenting civilian harm—defined not merely as immediate fatalities but as the ripple effects of trauma, displacement, and infrastructural ruin—became an existential challenge. Bellingcat staff and volunteers, tasked with the monumental duty of geolocating and verifying over 2,500 incidents of civilian harm, found that the sheer volume of user-generated content was outpacing the human capacity for analysis.

To address this, Bellingcat pioneered a machine learning (ML) framework designed to rank Telegram posts by their likelihood of containing evidence of civilian harm. This shift from manual, exhaustive scouring to an AI-assisted triage system has fundamentally altered how the organization conducts investigations, providing a scalable model for conflict monitoring globally.

The Chronology of an Open-Source Breakthrough

The transition toward automated detection began as a response to a growing "verification backlog." As the intensity of the conflict persisted, researchers were trapped in a cycle of constant monitoring, leaving little time for the deep-dive forensic analysis required to confirm specific incidents.

• 2022–2023: The initial phase of the invasion necessitated a reliance on manual collection. Researchers manually reviewed Telegram channels chronologically, flagging posts for verification. This process was high-stakes but dangerously slow, often forcing researchers to prioritize "live" feeds over historical analysis.
• 2024: Bellingcat began formalizing its dataset. By treating verified incidents as "positive instances" and surrounding, unverified posts as "negative instances," the team curated a dataset of over 54,000 unique Telegram entries.
• 2025: The implementation of the ML model reached maturity. The system was integrated into a custom user interface, allowing researchers to view prioritized feeds of potential incidents, which were then instantly linked to Bellingcat’s internal Auto Archiver and ATLOS verification platform.
• 2026: The methodology was stress-tested against Large Language Models (LLMs) to determine the efficacy of proprietary AI versus custom-trained, interpretable algorithms.

The Architecture of Evidence: Feature Engineering

Training an AI to detect human suffering is a delicate exercise in mathematical translation. To make the model "understand" the gravity of a Telegram post, Bellingcat’s data scientists employed a process known as feature engineering. This involved converting qualitative cues—such as a specific emoji or a keyword—into quantitative data points.

Converting Language into Logic

The team utilized a two-pronged approach to text analysis. First, researchers compiled a lexicon of high-impact keywords in both Russian and Ukrainian, such as "Shahed" (referencing loitering munitions) or "guided aerial bomb." These were converted into numerical frequency scores.

Second, the team employed advanced natural language processing (NLP). By using a Sentence Transformer model, they calculated "semantic similarity scores." This allowed the machine to understand that the word "hurt" is semantically proximate to "injured," while "residential" bears no direct relationship to harm, despite appearing in the same context. Finally, a multilingual text transformer from the BERT family represented each post as a vector of 768 numerical values, capturing the nuances of war-torn communication across linguistic barriers.

Metadata and Engagement Signals

Beyond text, the model analyzed the structural metadata of Telegram posts. Variables such as media_type, day_of_week, and forwarded status were fed into the algorithm. Intriguingly, engagement metrics—views, forwards, and specific reaction counts (such as the "crying face" emoji)—served as high-weight features, confirming the hypothesis that civilian-harm-related content often generates specific, recognizable patterns of social engagement.

The Battle of the Algorithms: Why XGBoost Prevailed

With a massive matrix of nearly 900 features, the research team tested four distinct machine learning algorithms: Logistic Regression, Random Forest, XGBoost, and LightGBM. The goal was not just predictive power, but interpretability.

Neural networks, often touted for their performance, were rejected due to their "black box" nature. Bellingcat required a model where researchers could audit why a specific post was flagged. After rigorous testing—utilizing a stratified split to maintain an 11% ratio of positive harm instances—XGBoost emerged as the clear victor. It outperformed competitors across all key metrics, specifically in Precision-Recall AUC (PR-AUC), which is vital for minimizing false negatives in sensitive human rights documentation.

The LLM Comparison: A Retrospective Analysis

As Large Language Models (LLMs) gained prominence, Bellingcat conducted a comparative study to see if models like Google’s Gemini 2.5/3.5 Flash or Gemma 3 could outperform their custom-built XGBoost model.

The results were revealing: while LLMs proved capable of processing engineered features and significantly outperforming baseline models, they did not surpass the custom-trained XGBoost model in PR-AUC. The study suggested that while generative AI is powerful, specialized models trained on specific domain-knowledge datasets remain superior for high-stakes, low-frequency event detection.

Ethical Considerations: Automation and Trauma

The introduction of automation into the study of war is fraught with ethical peril. Bellingcat acknowledged two primary risks:

1. Automation Bias: The tendency for researchers to trust the algorithm implicitly, potentially overlooking nuances that only a human eye can detect.
2. Algorithmic Bias: The risk that the model might mirror the limitations of the training data, potentially failing to identify under-reported or less "visible" forms of harm.

To combat these risks, the team implemented "human-in-the-loop" protocols, including the random auditing of posts not flagged by the model and the deliberate exclusion of visual media analysis for this prototype. Furthermore, the system served a secondary, humanitarian purpose: it reduced the frequency with which researchers were forced to view highly distressing, graphic imagery by filtering out irrelevant content, thereby mitigating secondary trauma.

Implications for Global Conflict Monitoring

The success of the Telegram Civilian Harm Detector has profound implications for human rights organizations worldwide. As conflicts in Sudan, the Middle East, and beyond continue to generate an ocean of digital evidence, the ability to rapidly filter this data is no longer a luxury—it is a necessity.

The model is designed to be "platform-agnostic" in logic, if not in execution. While Telegram’s API was essential for this specific project, the methodology—using semantic similarity, metadata feature engineering, and interpretable ML—can be adapted to any platform that allows for data collection.

Looking Toward the Future

The model is currently an ongoing project. Future iterations aim to integrate multi-modal analysis, incorporating the visual content of photos and videos. However, the organization remains cautious. As noted by the project leads, the integration of image analysis brings a new layer of complexity regarding bias and privacy.

For the OSINT community, Bellingcat’s work serves as a blueprint for the future of investigative journalism. By bridging the gap between raw data and human intelligence, they have demonstrated that technology can be a powerful ally in the fight for accountability, provided it is deployed with rigorous ethical oversight and a commitment to transparency.

Editor’s Note: This article details a prototype system. The model described is a work in progress and requires continuous re-evaluation and human validation before deployment in any new geographic or political context. Bellingcat remains a non-profit entity dependent on public support; those interested in the technical specifics of this implementation are encouraged to consult the open-source Code Notebooks provided by the organization.