Extracting information from servers with shodan

Shodan is a search engine that allows you to scan the internet for open ports and services. This can be a valuable tool for security researchers, penetration testers, and anyone who wants to learn more about the overall security posture of the internet.

Extracting information from servers with Shodan can be done using a variety of methods, including:

• Searching by IP address or hostname: You can search for servers by their IP address or hostname. This can be useful for finding specific servers that you are interested in.
• Searching by port: You can search for servers by the port that they are open on. This can be useful for finding servers that are running specific services.
• Searching by banner: You can search for servers by the banner that they return when you connect to them. This can be useful for identifying specific types of servers.
• Searching by location: You can search for servers by their location. This can be useful for finding servers in specific countries or regions.

Once you have found the servers that you are interested in, you can extract information from them using a variety of methods, including:

• Using the Shodan API: Shodan provides an API that allows you to query its database of servers. This can be a useful way to extract information from servers in an automated way.
• Using a Shodan parser: There are a number of Shodan parsers available that allow you to parse the output of Shodan scans. This can be a useful way to extract information from Shodan scans in a structured format.
• Using a Shodan client: There are a number of Shodan clients available that allow you to scan the internet for open ports and services. These clients can be a useful way to extract information from servers in a manual way.

Once you have extracted information from servers with Shodan, you can use that information to:

• Identify vulnerabilities: You can use the information you have extracted from Shodan to identify vulnerabilities in servers. This can help you to prioritize your security efforts and protect your systems from attack.
• Perform penetration tests: You can use the information you have extracted from Shodan to perform penetration tests against servers. This can help you to identify and exploit vulnerabilities in servers.
• Gather intelligence: You can use the information you have extracted from Shodan to gather intelligence about the overall security posture of the internet. This can help you to make informed decisions about your security posture.

Some additional tips for extracting information from servers with Shodan:

• Use filters: Shodan allows you to filter your search results by a variety of criteria. This can help you to narrow down your results and find the servers that you are most interested in.
• Use the Shodan API: The Shodan API is a powerful tool that can be used to extract information from Shodan in an automated way.
• Use a Shodan parser: Shodan parsers can be used to parse the output of Shodan scans in a structured format. This can make it easier to extract and analyze information from Shodan scans.
• Use a Shodan client: Shodan clients can be used to scan the internet for open ports and services. This can be a useful way to extract information from servers in a manual way.

Shodan is a powerful tool that can be used to extract information from servers. By using Shodan and the methods described above, you can gather valuable intelligence about the overall security posture of the internet.

Shodan offers a suite of services that provide access to its vast database of internet-connected devices, enabling users to gather insights into the overall security posture of the internet. These services cater to different needs and skill levels, offering a range of functionalities and usage options.

Accessing Shodan’s primary services:

1. Shodan Search:

• Description: Shodan Search is the core service that allows users to query Shodan’s database using various criteria, including IP address, port, banner, location, and organization.
• Usage: This service is ideal for researchers, security professionals, and network administrators who need to identify and analyze open ports, services, and vulnerabilities on the internet.

2. Shodan Monitor:

• Description: Shodan Monitor enables users to track and monitor changes in their internet-connected devices, providing real-time alerts for newly discovered devices, port modifications, and vulnerability disclosures.
• Usage: This service is particularly useful for organizations that want to stay ahead of potential security threats and maintain a proactive approach to network security.

3. Shodan Maps:

• Description: Shodan Maps provides a visual representation of internet-connected devices, allowing users to explore and analyze device distribution, vulnerabilities, and trends based on geographical location.
• Usage: This service is valuable for researchers, security analysts, and network administrators who want to gain a comprehensive understanding of the cyber landscape and identify potential risks in specific regions.

4. Shodan Images:

• Description: Shodan Images collects and archives screenshots of websites and web applications, enabling users to visualize the appearance of these online resources over time.
• Usage: This service is helpful for researchers, security analysts, and web developers who want to track changes in website design, identify potential security flaws, and gain insights into website development trends.

5. Shodan API:

• Description: The Shodan API provides a programmatic interface for accessing Shodan’s data and functionalities, allowing developers and integrators to embed Shodan’s capabilities into their own applications and workflows.
• Usage: This service is geared towards developers, security researchers, and organizations that want to integrate Shodan’s data and features into their own tools and systems.

6. Shodan Developer Portal:

• Description: The Shodan Developer Portal serves as a resource center for developers, providing documentation, tutorials, and code samples to assist them in building applications and integrations using the Shodan API.
• Usage: This portal is specifically designed for developers who are interested in exploring the Shodan API and incorporating its capabilities into their own projects.

Shodan’s services offer valuable tools for researchers, security professionals, and organizations to gain insights into the internet’s connected devices, assess potential security risks, and track changes in the cyber landscape. By utilizing these services effectively, users can enhance their understanding of the ever-evolving digital world and make informed decisions to protect their systems and data.

The Shodan RESTful API

The Shodan RESTful API is a powerful tool that allows users to programmatically access Shodan’s vast database of internet-connected devices. It provides a comprehensive set of endpoints for searching, filtering, and analyzing Shodan data, enabling developers, researchers, and security professionals to integrate Shodan’s capabilities into their own applications and workflows.

Key Features of the Shodan RESTful API:

• Search for devices: Query Shodan’s database using various criteria, including IP address, port, banner, location, organization, and tags.
• Filter results: Narrow down search results based on specific criteria, such as operating system, vulnerability type, and service version.
• Retrieve device information: Gather detailed information about discovered devices, including their IP addresses, ports, banners, operating systems, and vulnerabilities.
• Analyze device trends: Analyze trends in device discovery, vulnerability exposure, and cyber activity over time.
• Visualize device data: Generate charts and maps to visualize device distribution, vulnerability trends, and cyber activity patterns.

Prerequisites for Using the Shodan RESTful API:

• A Shodan account: Create a Shodan account to obtain an API key, which is required to authenticate API requests.
• Familiarity with RESTful APIs: Understanding the basic principles of RESTful APIs is essential for making effective use of the Shodan API.
• Programming skills: Familiarity with programming languages like Python, JavaScript, or Java is necessary for building applications that interact with the Shodan API.

Getting Started with the Shodan RESTful API:

1. Create a Shodan account and obtain an API key.
2. Explore the Shodan API documentation: The Shodan API documentation provides detailed explanations of API endpoints, parameters, and response formats.
3. Choose a programming language and development environment: Select a programming language that suits your preferred development workflow and tools.
4. Install the Shodan API client library: Install the appropriate API client library for your chosen programming language to simplify API interactions.
5. Write code to interact with the Shodan API: Use the Shodan API client library to make API requests, parse responses, and process results.

Setting Shodan API Key as an Environment Variable

This Python script demonstrates how to set the Shodan API key as an environment variable for use in other scripts.

1. Importing the Required Module:

import os

The script imports the os library for handling environment variables.

2. Setting the Shodan API Key as an Environment Variable:

# Set the Shodan API key as an environment variable
os.environ['SHODAN_API_KEY'] = 'SET_YOUR_OWN_API_KEY'

The Shodan API key is set as an environment variable using os.environ['SHODAN_API_KEY']. Make sure to replace 'SET_YOUR_OWN_API_KEY' with your actual Shodan API key.

3. Verifying the Environment Variable:

# Verify that the environment variable is set correctly
if 'SHODAN_API_KEY' not in os.environ:
    print("Error: SHODAN_API_KEY environment variable not set.")
    exit(1)

The script checks if the environment variable 'SHODAN_API_KEY' is set. If not, it prints an error message and exits with code 1.

4. Success Message:

print("SHODAN_API_KEY environment variable set successfully.")

If the environment variable is set correctly, the script prints a success message.

Explanation Summary:

• The script sets the Shodan API key as an environment variable using os.environ['SHODAN_API_KEY'].
• It checks whether the environment variable is set correctly and prints an error message if not.
• If the variable is set successfully, it prints a success message.

Before running this script, make sure to replace 'SET_YOUR_OWN_API_KEY' with your actual Shodan API key. This script is typically run once to set the environment variable for use in other scripts.

import os

# Set the Shodan API key as an environment variable
os.environ['SHODAN_API_KEY'] = 'SET_YOUR_OWN_API_KEY'

# Verify that the environment variable is set correctly
if 'SHODAN_API_KEY' not in os.environ:
    print("Error: SHODAN_API_KEY environment variable not set.")
    exit(1)

print("SHODAN_API_KEY environment variable set successfully.")

import os

shodan_api_key = os.environ['SHODAN_API_KEY']
print(shodan_api_key)

Using Shodan API to Obtain Information for an IP Address

This Python script utilizes the Shodan API to retrieve information about a specified IP address.

1. Importing the Required Modules:

import requests
import os

The script imports the requests library for making HTTP requests and the os library for handling environment variables.

2. Retrieving Shodan API Key and Setting Target IP:

SHODAN_API_KEY = os.environ['SHODAN_API_KEY']
ip = '1.1.1.1'

The Shodan API key is obtained from the environment variables using os.environ. The target IP address (ip) is set to ‘1.1.1.1’ for demonstration purposes.

3. Defining ShodanInfo Function:

def ShodanInfo(ip):
    try:
        result = requests.get(f"https://api.shodan.io/shodan/host/{ip}?key={SHODAN_API_KEY}&minify=True").json()
    except Exception as exception:
        result = {"error": "Information not available"}
    return result

A function ShodanInfo is defined, which takes an IP address as an argument. It makes a GET request to the Shodan API endpoint for the specified IP address, passing the API key. The minify=True parameter reduces the amount of data returned. Any exceptions are caught, and an error message is returned if information is not available.

4. Calling the ShodanInfo Function:

print(ShodanInfo(ip))

The script calls the ShodanInfo function with the target IP address and prints the result.

#!/usr/bin/env python

import requests
import os

SHODAN_API_KEY = os.environ['SHODAN_API_KEY']
ip = '1.1.1.1'

def ShodanInfo(ip):
    try:
        result = requests.get(f"https://api.shodan.io/shodan/host/{ip}?key={SHODAN_API_KEY}&minify=True").json()
    except Exception as exception:
        result = {"error":"Information not available"}
    return result

print(ShodanInfo(ip))

Explanation Summary:

• The script uses the Shodan API to obtain information about a specified IP address.
• It retrieves the Shodan API key from the environment variables.
• The ShodanInfo function makes a GET request to the Shodan API endpoint for the specified IP address and returns the result.
• The result is printed, providing information about the target IP address.

Using Shodan Python Library to Perform a Shodan Search

This Python script demonstrates how to use the Shodan Python library to perform a Shodan search for devices using the keyword ‘nginx’. It assumes you have set your Shodan API key as an environment variable.

1. Importing Required Modules:

#!/usr/bin/python

import shodan
import os

The script imports the shodan library for interfacing with the Shodan API and the os library for handling environment variables.

2. Retrieving Shodan API Key:

SHODAN_API_KEY = os.environ['SHODAN_API_KEY']
print(SHODAN_API_KEY)

The script retrieves the Shodan API key from the environment variable 'SHODAN_API_KEY' and prints it for verification.

3. Initializing Shodan API Object:

shodan = shodan.Shodan(SHODAN_API_KEY)

An instance of the Shodan class is created using the API key.

4. Performing Shodan Search:

try:
    resultados = shodan.search('nginx')
    print("results :",resultados.items())
except Exception as exception:
    print(str(exception))

A Shodan search is performed for devices with the keyword ‘nginx’ using the search method. The results are printed, and any exceptions are caught and printed.

Explanation Summary:

• The script uses the Shodan Python library to interact with the Shodan API.
• It retrieves the Shodan API key from the environment variable.
• An instance of the Shodan class is created with the API key.
• A Shodan search is performed for devices with the keyword ‘nginx’.
• The script prints the results if successful, or it prints any exceptions that occur during the search.

#!/usr/bin/python

import shodan
import os

SHODAN_API_KEY = os.environ['SHODAN_API_KEY']
print(SHODAN_API_KEY)
shodan = shodan.Shodan(SHODAN_API_KEY)

try:
    resultados = shodan.search('nginx')
    print("results :",resultados.items())
except Exception as exception:
    print(str(exception))

Shodan Search Script with Argument Parsing

This Python script performs Shodan searches based on user-provided arguments. It uses the Shodan Python library for interacting with the Shodan API and the argparse module for parsing command-line arguments.

1. Importing Required Modules:

#!/usr/bin/env python

import shodan
import argparse
import socket
import sys
import os

The script imports the necessary modules: shodan for accessing the Shodan API, argparse for parsing command-line arguments, socket for resolving hostnames to IP addresses, sys for system-related functions, and os for handling environment variables.

2. Setting Shodan API Key:

SHODAN_API_KEY = os.environ['SHODAN_API_KEY']
api = shodan.Shodan(SHODAN_API_KEY)

The Shodan API key is retrieved from the environment variable 'SHODAN_API_KEY', and an instance of the Shodan class is created using this key.

3. Command-Line Argument Parsing:

parser = argparse.ArgumentParser(description='Shodan search')

parser.add_argument("--target", dest="target", help="target IP / domain", required=None)
parser.add_argument("--search", dest="search", help="search", required=None)

parsed_args = parser.parse_args()

The script uses the argparse module to create a command-line parser. It defines two optional arguments, --target for specifying a target IP/domain and --search for performing a Shodan search. The parsed arguments are stored in the parsed_args variable.

4. Shodan Search Based on Arguments:

if len(sys.argv)>1 and sys.argv[1] == '--search':
    try:
        results = api.search(parsed_args.search)
        print('Results: %s' % results['total'])
        for result in results['matches']:
            print('IP: %s' % result['ip_str'])
            print(result['data'])
    except shodan.APIError as exception:
        print('Error: %s' % exception)

If the --search argument is provided, a Shodan search is performed using the specified search query. The results, including IP addresses and data, are printed.

5. Shodan Host Information Based on Arguments:

if len(sys.argv)>1 and sys.argv[1] == '--target':
    try:
        hostname = socket.gethostbyname(parsed_args.target)
        results = api.host(hostname)
        print("""
                IP: %s
                Organization: %s
                Operating System: %s
        """ % (results['ip_str'], results.get('org', 'n/a'), results.get('os', 'n/a')))

        for item in results['data']:
            print("""Port: %s Banner: %s""" % (item['port'], item['data']))

    except shodan.APIError as exception:
        print('Error: %s' % exception)

If the --target argument is provided, the script resolves the target hostname to an IP address, retrieves information about the host from Shodan, and prints relevant details, including IP, organization, and operating system. Additionally, it prints information about open ports and banners.

Explanation Summary:

• The script combines the power of the Shodan Python library and the argparse module for flexible Shodan searches based on user-provided arguments.
• It supports two modes: --search for Shodan searches and --target for detailed information about a specific host.
• The Shodan API key is retrieved from the environment variable.
• The script utilizes exception handling to handle potential errors during API calls.

Usage Examples:

• To perform a Shodan search: python script.py --search "query"
• To get information about a specific host: python script.py --target example.com

#!/usr/bin/env python

import shodan
import argparse
import socket
import sys
import os

SHODAN_API_KEY = os.environ['SHODAN_API_KEY']

api = shodan.Shodan(SHODAN_API_KEY)

parser = argparse.ArgumentParser(description='Shodan search')

parser.add_argument("--target", dest="target", help="target IP / domain", required=None)
parser.add_argument("--search", dest="search", help="search", required=None)

parsed_args = parser.parse_args()

if len(sys.argv)>1 and sys.argv[1] == '--search':
    try:
        results = api.search(parsed_args.search)
        print('Results: %s' % results['total'])
        for result in results['matches']:
            print('IP: %s' % result['ip_str'])
            print(result['data'])
    except shodan.APIError as exception:
        print('Error: %s' % exception)
        
if len(sys.argv)>1 and sys.argv[1] == '--target':
    try:
        hostname = socket.gethostbyname(parsed_args.target)
        results = api.host(hostname)
        print("""
                IP: %s
                Organization: %s
                Operating System: %s
        """ % (results['ip_str'], results.get('org', 'n/a'), results.get('os', 'n/a')))

        for item in results['data']:
            print("""Port: %s Banner: %s""" % (item['port'], item['data']))
        
    except shodan.APIError as exception:
        print('Error: %s' % exception)       

Shodan DNS Resolution and Host Information Script

This Python script utilizes the Shodan API to perform DNS resolution for a given domain and retrieves detailed information about the corresponding IP address.

1. Importing Required Modules:

import shodan
import requests
import os

The script imports the necessary modules: shodan for accessing the Shodan API, requests for making HTTP requests, and os for handling environment variables.

2. Setting Shodan API Key:

SHODAN_API_KEY = os.environ['Shodan_api_key'] 
api = shodan.Shodan(SHODAN_API_KEY)

The Shodan API key is retrieved from the environment variable 'Shodan_api_key', and an instance of the Shodan class is created using this key.

3. Performing DNS Resolution and Retrieving Host Information:

domain = 'www.python.org'
dnsResolve = f"https://api.shodan.io/dns/resolve?hostnames={domain}&key={SHODAN_API_KEY}"

try:
    resolved = requests.get(dnsResolve)
    hostIP = resolved.json()[domain]

    host = api.host(hostIP)
    print("IP: %s" % host['ip_str'])
    print("Organization: %s" % host.get('org', 'n/a'))
    print("Operating System: %s" % host.get('os', 'n/a'))

    for item in host['data']:
        print("Port: %s" % item['port'])
        print("Banner: %s" % item['data'])

except shodan.APIError as exception:
    print('Error: %s' % exception)

• The script constructs the Shodan DNS resolve API URL using the provided domain and API key.
• It makes an HTTP request to the Shodan API to perform DNS resolution and obtain the corresponding IP address.
• The obtained IP address is then used to fetch detailed information about the host using the api.host method.
• The script prints relevant details such as IP, organization, operating system, open ports, and banners.

Usage Example:

• Replace 'www.python.org' with the desired domain.
• Ensure the Shodan API key is set correctly in the environment variable 'Shodan_api_key'.
• Run the script to perform DNS resolution and retrieve host information.

Summary:

• This script demonstrates how to leverage the Shodan API for DNS resolution and gathering detailed information about a specific host.
• It combines the Shodan Python library and the requests module to interact with the Shodan API and make HTTP requests.
• The script handles potential API errors using exception handling.

import shodan
import requests
import os

SHODAN_API_KEY = os.environ['Shodan_api_key'] 
api = shodan.Shodan(SHODAN_API_KEY)

domain = 'www.python.org'

dnsResolve = f"https://api.shodan.io/dns/resolve?hostnames={domain}&key={SHODAN_API_KEY}"

try:
    resolved = requests.get(dnsResolve)
    hostIP = resolved.json()[domain]
   
    host = api.host(hostIP)
    print("IP: %s" % host['ip_str'])
    print("Organization: %s" % host.get('org', 'n/a'))
    print("Operating System: %s" % host.get('os', 'n/a'))


    for item in host['data']:
        print("Port: %s" % item['port'])
        print("Banner: %s" % item['data'])

except shodan.APIError as exception:
        print('Error: %s' % exception)

Shodan Search Script for FTP Servers

This Python script utilizes the Shodan API to search for FTP servers that have anonymous users logged in.

1. Importing Required Modules:

import shodan
import re
import os

The script imports the necessary modules: shodan for accessing the Shodan API, re for regular expressions, and os for handling environment variables.

2. Setting Shodan API Key and Initializing Shodan API:

servers =[]
shodanKeyString = os.environ['SHODAN_API_KEY']
shodanApi = shodan.Shodan(shodanKeyString)

The Shodan API key is retrieved from the environment variable 'SHODAN_API_KEY', and an instance of the Shodan class is created using this key.

3. Performing Shodan Search:

results = shodanApi.search("port: 21 Anonymous user logged in")
print("hosts number: " + str(len( results['matches'])))
for result in results['matches']:
    if result['ip_str'] is not None:
        servers.append(result['ip_str'])

The script performs a Shodan search using the query string "port: 21 Anonymous user logged in". It retrieves matching results and prints the total number of hosts found. The IP addresses of the matching hosts are stored in the servers list.

4. Displaying FTP Server IP Addresses:

for server in servers:
    print(server)

Finally, the script iterates over the servers list and prints each FTP server’s IP address.

Usage Example:

• Ensure the Shodan API key is set correctly in the environment variable 'SHODAN_API_KEY'.
• Run the script to perform a Shodan search for FTP servers with anonymous users logged in.
• The script outputs the total number of matching hosts and their respective IP addresses.

Summary:

• This script demonstrates how to use the Shodan API to search for FTP servers based on specific criteria.
• It leverages the Shodan Python library and regular expressions to formulate a search query.
• The script prints the total number of matching hosts and displays the IP addresses of the identified FTP servers.

#!/usr/bin/env python

import shodan
import re
import os

servers =[]
shodanKeyString = os.environ['SHODAN_API_KEY']
shodanApi = shodan.Shodan(shodanKeyString)

results = shodanApi.search("port: 21 Anonymous user logged in")
print("hosts number: " + str(len( results['matches'])))
for result in results['matches']:
	if result['ip_str'] is not None:
		servers.append(result['ip_str'])
		
for server in servers:
    print(server)