Hidden information

This article explores common sources of hidden information during web penetration testing, including files and folders, HTML comments, and cookies, along with tools and techniques to discover them.

Files and Folders

Web servers often contain files and directories that are not linked from the main website but are accessible if their paths are known. These hidden resources can include configuration files, backup files, or administrative interfaces, which may expose sensitive information or provide an entry point for attackers.

Nmap Scripting Engine (NSE)

Use Nmap's http-enum script to enumerate common files and directories on a web server.

nmap -sV -p 80 --script=http-enum <domain>

This script scans the target server for well-known paths and directories, helping identify hidden resources.

Robots.txt:

The robots.txt file is used to instruct web crawlers about which parts of the site should not be indexed. However, it can also reveal hidden directories.

curl http://domain/robots.txt

Review this file to discover directories that the site administrator may want to keep hidden.

sitemap.xml:

The sitemap.xml file provides a hierarchical overview of the website's structure. It can be used to identify important pages and directories.

curl http://domain/sitemap.xml

Website Mirroring

Tools like HTTrack can be used to create a local copy of the entire website, making it easier to analyze its structure and content.

httrack http://domain/ -O <folder>

This command downloads the website to the specified folder, allowing offline inspection of its files and directories.

HTML Comments

Developers often leave comments in the HTML source code for documentation or debugging purposes. While these comments are not visible to end-users, they can be accessed by viewing the page source. Sensitive information, such as internal links, credentials, or development notes, may be inadvertently exposed in these comments.

Example:

<!-- Admin login: [e-mail], Password: P@ssw0rd123 -->

Manually inspect the HTML source code of each web page or use automated tools to extract comments. Look for keywords like "TODO," "FIXME," or "DEBUG," which may indicate sensitive information.

Cookies

Cookies are small pieces of data stored on the client side that are used to track user sessions, store preferences, or manage authentication. While cookies are essential for web functionality, they can also contain sensitive information or be misconfigured, leading to security vulnerabilities.

Key Points to Analyze:

• Session Cookies: These cookies are used to maintain user sessions. If they are not properly secured (e.g., lacking the HttpOnly or Secure flags), they can be stolen via cross-site scripting (XSS) attacks.

• Privilege Information: Some cookies may store user roles or privileges, which can be manipulated to escalate privileges or impersonate other users.

• Expiration Time: Cookies with long expiration times increase the risk of session hijacking if they are intercepted.

Use browser developer tools or extensions like EditThisCookie to inspect and analyze cookies. Look for sensitive data, weak security flags, or improper configurations.