This article examines the main mistakes that are often made in open source intelligence (OSINT). Learn how to avoid common mistakes and improve the quality of your research to produce reliable and proven results.
When news spreads quickly, many turn to open source experts to make sense of events. This highlights the importance of research using publicly available resources such as satellite imagery and flight tracking websites. However, the rise in popularity of OSINT has led to misuse of the term and confusion due to imperfect methods of analysis.
Good open source research is not dependent on number of followers or status, but is based on collaboration and transparency so that others can replicate the results.
Open source research has long-term importance because it can help prosecute criminals. This raises the bar for both researchers and accountability in conflict settings.
There are certain mistakes open source researchers make, especially when monitoring armed conflicts, but they are applicable to any area of OSINT. It is important to recognize these errors and work to eliminate them in order to improve the quality of research.
If you are a researcher or just a consumer of information, paying attention to these mistakes will help you evaluate the quality of open source research and improve your own skills in this field.
The main principle of open source research is that information should be transparent and publicly available. This allows anyone to verify the authenticity of the data and the source of its origin.
Since the start of Russia’s full-scale invasion of Ukraine in 2022, many OSINT aggregators on Twitter have gained popularity by reposting Telegram videos without crediting the original source, making it difficult to verify and trace the origin of the content.
It is important to store metadata because it can contain important information needed for investigations. Metadata is often lost when videos or images are re-uploaded or shared without proper links.
While there are times when it may not be ethical to provide a link, the general rule is to share the source of the content whenever possible. This helps maintain the accuracy and transparency of future research.
All people have biases, but open source researchers must try to separate them from the evidence they analyze. Even if the research aims to achieve a specific goal, it is important to recognize when the data do not support that goal and to be transparent about the uncertainty.
Confirmation bias causes us to accept as true information that supports our beliefs and reject information that contradicts them. This can influence research regardless of political or social positions.
In order to maintain trust, it is important to acknowledge the limitations of your knowledge, even if you have strong beliefs. Failure to recognize this can lead to erroneous conclusions and biased investigations.
Internet content is often ephemeral: the Internet is littered with links to pages that no longer exist. This could be because the web domain owner stopped paying their bills. This could be because the website has changed the way the pages are organized. A content hosting platform may have decided to delete a large number of its files either intentionally or accidentally. Social media posts are often deleted either by the account that created the post or by the social media platform’s moderation team.
Third-party archiving platforms like the Internet Archive’s Wayback Machine or archive.today are the most reliable way to archive content, although they often fail to properly archive content from multiple social media platforms, as well as video in general. If all else fails, a screenshot is better than nothing.
In the context of conflict monitoring, events are often taken out of context and exaggerated. For example, researchers who are unfamiliar with the analysis of NASA FIRMS data may mistakenly interpret routine scheduled fires or other temperature changes as dangerous. During crises, such routine events can be misinterpreted by people who are not experienced with such information, leading to excessive significance of such data.
An example of this trend is the recent situation with baseball player Shohei Ohtani. In December 2023, a private flight from Anaheim to Toronto led online detectives to speculate that Ohtani might sign with the Toronto Blue Jays. In fact, the flight was carrying a Canadian businessman and had nothing to do with Ohtani or baseball.
Distinguishing between normal and unusual events requires considerable expertise in a particular field, whether it is monitoring conflicts, natural disasters, or other fields of research.
There are many open source tools and users need experience and training to use them effectively. Problems with new tools often arise from a lack of understanding of their limitations. For example, facial recognition software has its strengths and weaknesses, and results can be unreliable without additional context.
Photo manipulation detection tools can also give false results, leading to wrong conclusions.
Even after mastering the tools, it is important to learn how to interpret the data correctly. For example, footage taken by a drone was mistaken for UFOs, and clouds on satellite images were mistaken for craters.
Sometimes OSINT accounts redact the footage in ways that could mislead or reduce the quality of the analysis. For example, they can overlay audio tracks, create compilations, or trim original videos. One of the common practices of such accounts is watermarking videos and images. This makes it difficult to reverse frame lookup to determine the source of the video, which can lead to errors in determining its authenticity.
When working with open source content, it’s important not to edit the footage in a way that might hide or remove useful information. Even if the changes do not seem to affect important details, it is impossible to predict whether this information will not be useful in the future.
For example, the audio in the video recording the shooting of Colombian journalist Abelardo Liz contained important audio clues that helped identify the location of the shooting. Editing, by adding another soundtrack, could have hidden these key elements.
It’s easy to get caught up in the rapid flow of news, especially during times of terrorist attacks or military conflicts. Social media platforms, where open source public research is often conducted, encourage this behavior. There is a great temptation to be the first person to make a “breakthrough” in an investigation, or to quickly analyze a developing event, which can lead to hasty conclusions. However, content validation should always take priority over speed.
One of the most high-profile and damaging examples of this is the repeated cases where amateur researchers have mistakenly identified innocent people as the perpetrators of terrorist attacks. Most recently, this happened during the Bondi Junction attack, as well as the 2013 Boston Marathon bombings and the 2023 Allen, Texas mall shooting. Such false results are based on the fact that an innocent person has the same name or a similar face as the guilty party—neither of which alone is sufficient evidence, given the seriousness of such identifications.
When speed is prioritized, fact-checking is often overlooked. This can lead to misinformation and confusion, instead of ensuring accuracy and clarity in the unfolding of events. In the pursuit of quick results, there is a risk of spreading unverified or incorrect information, which can worsen the situation instead of helping to understand it.