Open Source Intelligence(OSINT) for Cyber Security

What is Open-source Intelligence?

Open-source Intelligence(OSI) is the intelligence produced by collecting and analyzing information from publicly available sources to address a specific security requirement [1]. The OSINT life cycle consists of 5 steps: Planning & Direction, Collection, Processing, Analysis & Production, and Dissemination. OSINT research can be divided into active and passive OSINT:

1. Active: Information is gathered quietly from online sources without engaging with the target such as gathering information present on the target’s social media.
2. Passive: Information is gathered by directly engaging with the target such as commenting on the target’s social media posts.  

Sources of Open-Source Intelligence

Some of the open-source Intelligence sources are [2]:

1. Media Sources: Newspapers, Magazines and journals, Television and radio broadcasts, Online news websites and aggregators
2. Internet Sources: Search engines such as Google, websites and blogs, social media platforms such as Facebook, forums and discussion boards, video sharing platforms such as YouTube.
3. Research Institutions: Research papers and studies, think tank publications, university, and academic institution websites.
4. Databases: Whois databases for domain ownership, Patent and trademark databases, and Business registration databases.
5. Public Government Data: Government reports and publications, regulatory and administrative filings, government websites, public records, and datasets.

Tools and Techniques Used in OSINT Research

Some of the tools used in OSINT research are as follows[3]:

1. Shodan: Shodan is a database of interconnected devices. Shodan is a search engine tool that helps in discovering devices, servers connected to the network. 
2. Google Dorking: It is a technique used to find vulnerabilities and exposed private data using the Google search engine. An example of a Google Dorking command is “filetype: PDF <query>”, this command lists all the PDF links related to the searched query.
3. Maltego: It is an all-in-one tool for link analysis that offers real-time data mining and information gathering, as well as the representation of this information on a node-based graph, making patterns and multiple-order connections between said information easily identifiable.
4. The Harvester: A tool for gathering subdomain names, e-mail addresses, virtual hosts, open ports/ banners, and employee names from different public sources such as search engines, and PGP key servers.
5. SpiderFoot: It automatically queries public data sources to gather information such as IP address, domain, email address, etc.
6. HaveIBeenPwned: Used to check if an email address or phone number is compromised in a data breach. This service collects and analyzes information from numerous database dumps to check for compromised accounts.
7. TinEye: Reverse image search tool i,e. users can search using images.
8. Social Searcher: Monitor user mentions across social media platforms.
9. GeoCreepy: Gather information through social networking platforms.
10. Wappalyzer: Search technology stack of a website.
11. BuiltWith: Website profiler tool similar to Wappalyzer, gathers the tech stack of a website.

Major Use Cases of OSINT

Open Source Intelligence (OSINT) has broad applicability across various sectors, providing valuable insights without the need for covert or clandestine methods. Here’s how OSINT can be leveraged in the mentioned use cases:

Threat Intelligence

1. OSINT can help organizations understand how much of their data is publicly available for security posture management.
2. Spot potential phishing campaigns or identify leaked credentials by monitoring forums, social media, and paste websites.
3. Tracing the malware back to its source, or understanding trends and patterns to uncover larger attack campaigns.
4. Open-source data helps in attributing cyberattacks to specific threat actors or groups by analyzing tactics, techniques, and procedures (TTPs).

Business Intelligence

1. OSINT can help businesses conduct Competitive Analysis.
2. Companies employ reputation management strategies by monitoring public sentiment about their brand, products, or executives.
3. Businesses identify emerging market trends by analyzing public discussions, reviews, and other open-source data. 
4. Businesses monitor their supply chain risks by tracking and evaluating public data about suppliers.

Law Enforcement and Intelligence Agencies

1. Law enforcement can use OSINT for criminal investigations.
2. OSINT can aid in tracking down missing persons.
3. Investigate fraudulent activities, money laundering, and other financial crimes by analyzing public financial data or tracking digital footprints.

Scientific Research

1. Data Gathering from public databases, websites, or publications.
2. OSINT can help identify potential research collaborators, based on their public works and publications.
3. OSINT provides a rich dataset for public sentiment analysis, cultural trends, or societal responses to certain events.

Challenges with OSINT

OSINT (Open Source Intelligence) offers vast potential, but its use also comes with various challenges. Here are some of the key issues with OSINT:

1. The volume of Information: Looking at the vast amount of publicly available information, filtering out the noise, and identifying relevant data is a significant challenge.
2. Information Reliability: Due to the uncontrolled nature of OSINT sources, false or misleading information is prevalent, especially on social media. It is very challenging to verify the authenticity of the information.
3. Ephemeral Nature of Data: Capturing and archiving relevant data before it disappears can be challenging. For example, websites can go offline, posts can get deleted, and social media accounts may become private or deactivated.
4. Legal and Ethical Concerns: Even if the information is publicly available, accessing, storing, or using it might violate terms of service, data protection regulations, or privacy laws.
5. Language and Cultural Barriers: Significant information might be in languages unfamiliar to the researcher. Even with translation tools, cultural contexts, idioms, and colloquialisms can be missed or misunderstood.
6. Data Redundancy: Multiple sources might provide the same or overlapping data. Additional efforts are required to differentiate between unique and redundant sources.
7. Technical Limitations: Many websites block web scraping tools and employ mechanisms such as anti-bots, CAPTCHAs, and rate limiting to avoid automated data collection.

How companies are using OSINT to protect themselves?

OSINT helps organizations by harnessing publicly available information for strategic advantages. It assists in understanding the digital footprint, assessing vulnerabilities, and dodging potential threats. OSINT can also provide insights into competitors’ strategies, market trends, and public sentiment. When used in security frameworks and solutions, it identifies potential attack vectors and mitigates cyber risks. For businesses, it provides an insight into customer preferences. Furthermore, by monitoring global events, organizations can anticipate market disruptions, ensuring proactive response, especially in sectors such as trading, and investment banking. Essentially, OSINT transforms huge amounts of distributed unintelligible public data into collective actionable intelligence.

Conclusion

The potential of OSINT is vast. Whether for understanding potential threats, gaining a competitive edge, enforcing the law, or pushing the boundaries of human knowledge, OSINT offers invaluable insights from the vast amounts of public data. Despite all the challenges listed in this article, OSINT remains a crucial resource for businesses in various sectors. However, Its challenges do emphasize the need for proper training, advanced tools, and a methodical approach to collecting, analyzing, and interpreting open-source data.