Subdomain Enumeration Techniques

What is sub-domain Enumeration?

Subdomain enumeration is a process of finding subdomains for one or more domains.

Why need sub-domain enumeration?

Types of sub-domain enumeration

There are two types of enumeration techniques available which consist of other sub techniques.

1. Passive sub-domain enumeration

In passive sub-domain enumeration, an adversary or tester gathers the sub-domain information without directly connecting to the infrastructure managed by the organization. In this process, the adversary or tester gathers the information from third parties like, the information is gathered by Virustotal, DNSDumster, certificate, etc. Generally, any alerts of flags are not raised during such probing.

Passive sub-domain enumeration techniques:

2. Active sub-domain enumeration

In active sub-domain enumeration, the adversary or tester gathers the information by directly probing the infrastructure managed by the organization. In active enumeration, the detection of adversary or tester may be possible by the organization. Such kind of probing may raise alerts and/or flags.

Active sub-domain enumeration techniques:

Passive sub-domain enumeration techniques

By Certificate Transparency

Before understanding the certificate transparency, we would first need to understand the basic terminology related to digital certificates.

What is a digital certificate?

A digital certificate is a certificate that is issued by the trusted third party which is called a certificate authority to verify the identity of the certificate sender or holder.

For example: The driving license assigned by the government authority which is a third party whom we trust. The traffic police check if the ride can or cannot ride by checking the driving license. Here in this example traffic police are like users who access the website and government authority is the certificate authority whereas the rider is the website.

Who are certificate authorities?

Certificate authorities or CA are the globally trusted third-party companies like VeriSign, GeoTrust, GoDaddy, etc that manage three major tasks:

What is a digital signature?

A digital signature is a mathematical way for verifying the authenticity of digital messages or documents.

The steps followed in creating a digital signature are:

How the certificate works when a user accesses a website?

What is Certificate Transparency?

Certificate Transparency is the open-source framework for the certificate authorities (CAs) under which they log the certificates to the domain name owners. In this way, anyone can see which CA has issued a certificate for which domains. It is like the inventory of all certificates, certificate authorities, and domains.

Why we need Certificate Transparency?

The problem with the previous CAs infrastructure was that there was no effective way to audit or monitor SSL certificates in real-time. So, when any missteps or malicious activities happen, the suspect certificate was not usually detected and revoked for weeks or months. These miss issues of certificates were used to spoof a legitimate website or to install malicious software etc.

Case Study: The DigiNotar was a Dutch certificate authority that was compromised, and the adversary used the CAs system to issue 500 fake SSL certificates. In the investigation, it was discovered that the adversary issued the wildcard certificate for google.com. Which gave the adversary the ability to impersonate Google. This was widely used by the adversary to attack against Gmail users in Iran.

How does Certificate Transparency help in OSINT?

Now we know that certificate transparency logs all the entries of the issued certificates in an inventory. This includes domain names, sub-domain names, and email addresses. This is publicly available to everyone. By using CT (Certificate Transparency) logs an adversary can gather basic information about the organization’s infrastructure in a passive way.

CT logs search engines:

Note: It may be possible that the domain/sub-domain during recon may not resolve because the domain/sub-domain names may not exist. Certificate Transparency logs are only appended and there is no way to delete these domains from CT logs.

By using Search Engines:

Search engines are one of the best techniques to find subdomains in a passive way. During my research, I found that the “Site:” operator which was used to search domain and subdomains was working in the below-mentioned search engines:

Example: site:example.com

By using Online DNS Tools:

During my research I found 9 sub-domain enumeration services:

Comparison of online available subdomains enumeration services:

The subdomains number mentioned in pie-chart were not validated, so it may be possible that these subdomains are false positive, but you can validate by using “massdns” or any other tool that resolves subdomains.

By using ASN (Autonomous System Number) number

An autonomous system is a set of interconnected networks under single administrative control. In simple term, ISP (Internet Service Provider) is an autonomous system where one or more intradomain routing protocols are used to handle the networking inside the autonomous system and only one inter-domain routing protocol which is BGP (Border Gateway Protocol) is used to handle the routing between different autonomous systems.

Autonomous System Number

An autonomous system number is a unique number that is given to an Autonomous system and which is assigned by IANA (Internet Assigned Numbers Authority). Autonomous system numbers can be public or private.

Private ASN: A private ASN is a unique identifier for an AS that is used for communicating an AS to one entity.

Public ASN: A public ASN is a unique identifier that is advertised to the public internet. This is also used to communicate between ASNs.

Every ISP buys the IP address pool and ASN number from IANA. The IP address pool and ANS number are unique. It helps to differentiate the network from other networks. For BGP connection with multiple networks, ASN is needed and for an internal autonomous system (AS) IP address is needed. BGP protocol works on the ASN number.

In the above image, when BOB sends a message to Marley from Reliance Jio ISP to BSNL ISP below are the steps that are performed for sending messages.

Note: Many large network infrastructure organizations also have the ASN numbers, so it’s not necessary that only ISP has the ASN number. It usually depends on the network infrastructure in use.

How is ASN number used in domain enumeration?

Online tools to find ASN number

Online tools to find IP pool from ASN number

We can use WHOIS queries and Nmap script to find all the IP addresses that belong to the ASN

Example:

whois -h whois.radb.net -- '-i origin AS36459' | grep -Eo "([0-9.]+){4}/[0-9]+" | uniqnmap --script targets-asn --script-args targets-asn.asn=17012

By using Subject Alternate Name (SAN)

What is Subject Alternate Name (SAN)

It is an extension to the X.509 specification that allows us to secure additional domains or subdomains by a single SSL certificate. It is also called a “multi-domain SSL certificate”.

So, now the question is what is the difference between a multi-domain SSL certificate and a wildcard SSL certificate?

Wildcard Certificate

Wildcard certificate only secures the multiple subdomains under the main domain.

Example: If issued a wildcard certificate on *.example.com then this certificate secures only subdomains which ends with “example.com”.

Multi-Domain SSL Certificate

The multi-domain SSL certificate secures up to 250 unique domain names or subdomains and that domain/subdomains names mentioned in the Subject Alternative Names (SAN) field in the certificate.

Example: If issued a multi-domain SSL certificate on “example.com” then we can add multiple other domains and subdomains in SAN to protect them by SSL.

Tools to extract domain names from SAN

OpenSSL

true | openssl s_client -connect wikimedia.org:443 2>/dev/null \
| openssl x509 -noout -text \
| perl -l -0777 -ne '@names=/\bDNS:([^\s,]+)/g; print join("\n", sort @names);'

By Python Script

https://github.com/appsecco/the-art-of-subdomain-enumeration/blob/master/san_subdomain_enum.py

Command: python san_subdomain_enum.py wikimedia.org

By using Public Dataset (Rapid7)

Project Sonar is an initiative by Rapid7. Rapid7 initiates the project Sonar, where Rapid7 performs Internet scanning to collect Internet-wide scan data and then publish the results publicly for free and some data is paid. Datasets cover areas like:

These datasets can be useful during an offensive engagement like in this case for subdomain enumeration. Although finding data in large datasets is very time-consuming.

Rapid7 Datasets Link: https://opendata.rapid7.com/

By using Cloudflare

Cloudflare is the network of servers that are spread all over the world to create CDN (Content delivery network) like technology. This network act as a reverse proxy server between users and the server that holds the website.

Difference between reverse proxy and forward proxy

Reverse Proxy

Forward Proxy

Content Delivery Network (CDN):

It is a network of distributed caching servers with a web application firewall, which are used for delivering the content in the most optimal way and improve end user’s performance.

Steps for subdomain enumeration by Cloudflare:

Active sub-domain enumeration techniques

Brute force or Dictionary Attacks

Brute force means guessing possible combinations of the target until the expected output is discovered. So, in the subdomain context, the brute-forcing is to try the possible combination of words, alphabets, and numbers before the main domain in order to get a subdomain that is resolved to IP address. Sometimes subdomains are not indexed on search engines and are not available on online DNS aggregators sites in that case brute forcing is the best way to find out the subdomains which may have been forgotten by the organization. It is like a treasure for an adversary.

Two techniques for subdomain enumeration using brute force:

Dictionary brute force:

In the dictionary brute force, we directly use the wordlist to the brute force domain name to find valid subdomains.

Tools:

Permutation brute force:

In permutation brute force, we create a new resolved subdomain list from already known subdomains/domains by using permutation, mutation, and alteration with a wordlist.

Tool:

By Zone Transfer

DNS zone transfer is the process of replication DNS database or DNS records from the primary name server to the secondary name server.

The DNS zone transfer functionality used by an adversary only when the primary name server is configured to replicate the zone information to any server. An adversary acts as a slave and asks the master for a copy of the zone records.

Why need DNS zone transfer:

Availability

If the primary name server goes down, then the secondary server handles all DNS requests.

Load Balancing

If many DNS name resolution requests occurred, then the secondary name server load balances these requests.

Faster Resolution

If the primary name server is located on the slow WAN link, then the secondary name server handles the name resolution requests.

Types of information gathered by zone transfer

Command to check zone transfer

dig ns zonetransfer.me +noall +answer
dig axfr @nsztm1.digi.ninja zonetransfer.me
host -t ns zonetransfer.mehost -t ns zonetransfer.me
host -t axfr zonetransfer.me intns1.zonetransfer.me
nslookup -type=ns zonetransfer.me
nslookup -query=AXFR zonetransfer.me nsztm2.digi.ninja

By Domain Name System Security Extensions (DNSSEC)

Domain Name System Security Extensions (DNSSEC) is used to protect the integrity and authenticity of the data in DNS by establishing a chain of trust.

Before an understanding of DNSSEC, we first need to understand the basics of DNS:

What are the DNS functionalities?

www.example.com → 192.168.1.10

192.168.1.10 ← www.example.com

Why DNS?

Domain names are alphabet and they are easier to remember that is why we use domain names.

Before DNS the host.txt file was required to be regularly updated, which was distributed to all hosts on the Internet.

Issues with this are:

How does it work?

When you enter a domain name in the browser (www.google.com), it first tries to resolve from the system host file, then passes the request to the ISP DNS server, then passes to another DNS server and this process continues until the request is not resolved. When the request resolves the response flow back to the DNS servers to the original requester that is the browser and then the requested website becomes accessible.

Basic DNS terminologies:

Name servers

DNS Query

Name servers

Authoritative Server

Non-Authoritative/Caching Server

The non-Authoritative name server is the cache server, which cache the domain information for faster response to the domain you are querying.

DNS Query

DNS query is configured on the DNS server and accordingly, the DNS server works.

Recursive DNS Query

When the recursive query is configured on the DNS server then the DNS server does all the job on behalf of you to fetch the answer of your query. During this process, the DNS server might query other DNS servers on the internet for the answer.

Non-Recursive or Iterative DNS Query

When a non-recursive query is configured on the DNS server then the DNS server do not fetch the complete answer of the query on behalf of you but will give the address of other DNS server, which might have the answer of the query. During this process, the DNS server might provide the IP address of other DNS servers on the internet for the answer and the operating system resolver will query until the domain name does not resolve.

Inverse Query

Inverse DNS query works opposite to normal DNS query. Inverse DNS query or reverse DNS query is used when the user wants to resolve the IP address to a fully qualified domain name (FQDN).

DNS Protocol Vulnerability

The original DNS protocol was not designed with security in mind, as the Internet grows, it becomes less trustworthy. There was no DNS data protection mechanism available which caused below vulnerabilities:

Basic DNS Security Practices

Two protection mechanism “Transaction Signature (TSIG) and DNS Security Extension (DNSSEC)” use for DNS data protection

Transaction Signature (TSIG)

Command: dig @<server> <zone> AXFR -k <TSIG key file>

dig @localhost example.com AXFR -k Key-ns1.net.+200+1337.key

DNS Security Extension (DNSSEC)

For signature validation DNSSEC add new DNS record types:

RRset and RRSIG

The first step for securing the zone with DNSSEC is to group all the same types of records which is called resource record set (RRset).

RRset Example:

Bundle into a single A RRset

www.example.com. 7200 IN A 192.168.1.10

blog.example.net. 7200IN A 10.0.1.1

admin.example.net. 7200IN A 172.16.2.20

RRsets are digitally signed with the private key of the zone server and signature are published in DNS as RRSIG. Public DNSKEY is also published to verify the RRSIG signature.

RRSIG Example:

Zone Signing Key (ZSK)

Key Signing Key (KSK)

Delegation Signer (DS) Record

Why do we use two keys?

So, now the quest is what is the purpose of using two keys when both keys are from the zone itself.

We used two keys to protect the zone itself and to provide the parent-child trust, as making any modification in key signing key (KSK) is a difficult process because we have to make a hash and change that hash into the parent delegation signer (DS) record. Changing the DS record is a multi-step process that can end up breaking the zone if it’s performed incorrectly. On the other side, changing the zone signing key (ZSK) is much easier. We just need a modification within a zone.

What if the attacker compromised the child and parent server then all the DS information would match. So, to trust the DS record first understand the “The chain of trust”.

The Chain of Trust

Till now, example.com and .com check out, but what about root? There is no parent DS record to validate against.

For completing the chain of trust cycle, we must trust the security procedure done by a human.

For signing the root, the Root Signing Ceremony is performed every quarter in verify the public and highly audited way. In this ceremony, some selected individuals from the international community agree and sign the root DNS zone’s RRset for the DNSKEY records. The ceremony produces an RRSIG record that can be used to verify the root name servers public KSK and ZSK. Instead of trusting the public KSK because of the parent’s DS record, DNS resolver assumes that it’s valid because we trust the security process in which the Root Signing Ceremony was performed.

NSEC/NSEC3 (Explicit Denial of Existence)

In traditional DNS setup, when we ask DNS for the IP address of a domain that doesn’t exist, it returns an empty answer. So, how we authenticate that domain doesn’t exist since there is no message to sign. The fix for this issue was to add NSEC and NSEC3 record types that explicitly tell a DNS resolver that a given zone does not exist.

The NSEC record is used to prove that something really does not exist, by providing the name before it, and the name after it.

NSEC works by returning the “next secure” record. The NSEC record allows for proof of non-existence for record types. For every existing name, there is a corresponding NSEC record. This NSEC record states what types are available for the current name, as well as the next valid name, is in the sorted zone file. This brings us to the downside of NSEC: Since the NSEC records essentially chain through the complete zone, it’s possible to do “zone walking”.

Example:

example.com

api.example.com

beta.example.com

Let’s assume a request is made for example.com

Request:

dig NSEC example.com

Response:

ANSWER SECTION:

example.com. 3600 IN NSEC api.example.com. A NS SOA MX TXT AAAA RRSIG NSEC DNSKEY

Request:

dig NSEC api.example.com

Response:

ANSWER SECTION:

api.example.com. 3600 IN NSEC beta.example.com. A NS SOA MX TXT AAAA RRSIG NSEC DNSKEY

In the same way, attackers use it for zone walking and enumerate all subdomains in the zone.

NSEC3 uses a hashing algorithm to list the next available domain in the “hashed” format. It is still possible for an attacker to do zone walking, although at a higher computation cost.

DNSSEC Process

Tools for NSEC subdomain enumeration

Subdomain Enumeration by DNS Records

CNAME Record

In DNS zone file CNAME stands for a canonical name. CNAME is used in the domain name system (DNS) to create an alias or pointer for one domain name to another domain name.

For example:

example.com                 A             192.168.1.1documents.example.com       CNAME         example.com

Let say, example.com is the domain name that connects with the file server, where you store all the personal files, but you want to access the same server with documents.example.com for better understanding. So, for accessing the domain with documents.example.com you have to create an alias or pointer that points to the example.com.

Sometimes CNAME reveals an organization subdomain or reveals information about third-party services like Amazon.

SPF Record

SPF is the Sender Policy Framework as the name suggests. It is a policy that is used for whitelisting the IP address and domain names which are authorized to send an email on behalf of your domain. SPF is the list of IP addresses and domains that are allowed to send an email on behalf of your domain.

So, when you send an email let say from the example.com domain to the gmail.com domain. The Gmail that receives the email will check if the IP address of the mail server that sent the email is included in the SPF record of the example.com zone file.

If the sending IP address is in the list, the email will pass the SPF check and mail will likely be delivered.

If the sending IP is not on the SPF record, it likely not to be delivered.

An adversary uses this information to understand the organization’s infrastructure, IP addresses, third-party email services, internal netblocks, and subdomains. The SPF record is defined using the TXT record. So, whenever one searches for the SPF record, it should be searched by the TXT record.

Tool

Netblocks, ASN, and Domain names extractor from SPF Record

https://github.com/0xbharath/assets-from-spf

Subdomain Enumeration by HTTP Header

Content Security Policy (CSP) is a response header that tells the browser from what sources it is allowed to include and execute resources from.

It is like a filter where sources are mentioned and in sources, the domains and subdomains are mentioned. An adversary may use this information to enumerate more subdomains and other domains that are allowed by the organization.

Tool:

https://github.com/0xbharath/domains-from-csp

Conver Domain to IP Address

Tool

https://gist.github.com/xdavidhu/07457247b9087dea4ddaf52858500cce

#!/bin/bash# Converter.sh by @xdavidhu# This is a script inspired by the Bug Hunter's Methodology 3 by @Jhaddix# With this script, you can convert domain lists to resolved IP lists without duplicates.# Usage: ./converter.sh [domain-list-file] [output-file]echo -e "[+] Converter.sh by @xdavidhu\n"if [ -z "$1" ] || [ -z "$2" ]; then  echo "[!] Usage: ./converter.sh [domain-list-file] [output-file]"  exit 1fiecho "[+] Resolving domains to IPs..."while read d || [[ -n $d ]]; do  ip=$(dig +short $d|grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"|head -1)  if [ -n "$ip" ]; then    echo "[+] '$d' => $ip"    echo $ip >> $2  else    echo "[!] '$d' => [RESOLVE ERROR]"  fidone < $1echo -e "\n[+] Removing duplicates..."sort $2 | uniq > $2.newmv $2.new $2echo -e "\n[+] Done, IPs saved to '$2'."

--

--

Check This: https://lazyhacker22.blogspot.com/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store