dirsearch: How Smart Extension Handling Makes Path Discovery Actually Useful
Hook
Most directory brute-forcers append extensions blindly to every path, creating thousands of useless requests like ‘admin/.php’ or ‘backup.tar.php’. dirsearch solved this with a simple keyword replacement system that nobody talks about.
Context
Web application enumeration has always involved discovering hidden endpoints—admin panels tucked away at /wp-admin, forgotten backup files at /backup.sql, or configuration files at /config.php that leak credentials. Traditional tools like DirBuster used append-only extension logic: take a wordlist, add .php/.asp/.html to everything, and fire requests. This created massive noise and false positives. If your wordlist contained ‘database.sql’, you’d get requests for database.sql.php, database.sql.asp, and database.sql.html—none of which make sense.
dirsearch emerged from the penetration testing community’s frustration with this approach. Being actively developed by maurosoria and shelld3v, it introduced a wordlist templating system where you control exactly where extensions appear. Instead of blind appending, you use the %EXT% keyword as a placeholder. The tool replaces it with your target extensions, giving you surgical precision over what gets tested. For bug bounty hunters and red teamers scanning large applications, this difference between 10,000 smart requests and 50,000 garbage requests is the difference between finding vulnerabilities and getting blocked by rate limiters.
Technical Insight
The core architectural decision in dirsearch is separating extension logic from wordlist content. When you provide a wordlist like this:
admin
login.%EXT%
backupAnd run with -e php,asp,aspx, dirsearch generates:
admin
login.php
login.asp
login.aspx
backupNotice what didn’t happen: ‘admin’ didn’t become ‘admin.php.asp.aspx’, and ‘backup’ stayed untouched. Only entries with %EXT% got replacements. This maps to how real web servers actually work—you’re not guessing where extensions go, you’re explicitly defining path structures.
For legacy wordlists without %EXT% placeholders (like Daniel Miessler’s SecLists), dirsearch provides the --force-extensions flag:
python3 dirsearch.py -u https://target.com \
-e php,html \
-w /usr/share/seclists/Discovery/Web-Content/common.txt \
--force-extensionsThis appends extensions to every entry AND adds a trailing slash variant, turning ‘admin’ into ‘admin’, ‘admin.php’, ‘admin.html’, and ‘admin/’. It’s the append-everything behavior you’d expect from other tools, but opt-in rather than forced.
The --overwrite-extensions flag handles a trickier scenario. If your wordlist has ‘config.bak’ but you want to test ‘config.php’ and ‘config.asp’ instead, overwrite mode replaces existing extensions:
python3 dirsearch.py -u https://target.com \
-e php,asp \
-w wordlist.txt \
-OHowever, dirsearch intelligently excludes certain extensions from overwriting—.log, .json, .xml, and media files like .jpg or .png. This prevents nonsensical transformations like turning ‘error.log’ into ‘error.php’.
Session management addresses a practical problem in long-running scans. When enumerating a large application with 100k+ wordlist entries, scans can take hours. If your connection drops or you hit Ctrl+C, traditional tools lose all progress. dirsearch writes session state to ~/.dirsearch/sessions/ (or $HOME/.dirsearch/sessions/ for standalone binaries according to the documentation):
# Start scan
python3 dirsearch.py -u https://target.com -e php -w large.txt
# Interrupted? Resume with:
python3 dirsearch.py --session [session-file]Note that legacy .pickle/.pkl session formats are no longer supported—a breaking change that affected users upgrading from pre-0.4.x versions.
The bundled wordlist categories in db/categories/ are underrated. Instead of hunting for specialized wordlists, you can target specific attack surfaces:
# Scan for VCS exposure (.git, .svn)
python3 dirsearch.py -u https://target.com --wordlist-categories vcs
# Scan for database files, configs, and logs
python3 dirsearch.py -u https://target.com --wordlist-categories db,conf,logs
# Everything
python3 dirsearch.py -u https://target.com --wordlist-categories allCategories include: extensions, conf, vcs, backups, db, logs, keys, web, and common. This categorical approach mirrors how penetration testers actually think during enumeration—you’re not scanning for random paths, you’re hunting for specific file types that represent risk.
Gotcha
dirsearch is a brute-force tool at its core, which means performance and detection are inherent limitations. Aggressive request patterns will trigger rate limiting on most modern web applications, and intrusion prevention systems will flag the repetitive request patterns. There’s no documented built-in WAF evasion—no user-agent rotation, no request timing randomization, no header obfuscation. If the target has even basic security controls, you’ll get blocked quickly. Tools like ffuf or wfuzz offer more sophisticated filtering and matching logic for navigating around defensive systems.
The session file format breaking between versions is a painful gotcha. If you have saved sessions from dirsearch 0.3.x using .pickle files, they’re unusable in current versions. There’s no migration path—you have to re-run scans from scratch. For penetration testers with archived session data from old engagements, this is a documentation and reproducibility problem.
Wordlist dependency is also a constraint. dirsearch doesn’t generate paths intelligently based on observed application behavior—it’s entirely wordlist-driven. If your wordlist doesn’t contain ‘/api/v2/users’, you won’t find it, even if the tool discovered ‘/api/v1/users’. You’re bounded by what your wordlists cover. Competitors like feroxbuster have better recursive discovery with automatic depth limiting.
Verdict
Use dirsearch if you need a reliable, actively maintained directory scanner for penetration testing or bug bounty reconnaissance where you value control over extension handling and need session persistence for long scans. The %EXT% templating system and categorized wordlists make it faster to configure correctly than fumbling with SecLists paths in ffuf. The cross-platform standalone binaries eliminate dependency hell when deploying on client systems during engagements. It’s the right choice for methodical enumeration where you’re working through systematic wordlists against targets without aggressive WAFs. Skip it if you’re facing heavily defended applications where detection avoidance matters—dirsearch has no documented evasion capabilities and will get you blocked. Also skip it if you need parameter fuzzing, custom insertion points, or advanced matching logic beyond what the tool provides. For those scenarios, wfuzz or ffuf give you more flexibility despite steeper learning curves. And if raw speed is your priority over features, gobuster’s Golang implementation will likely outperform dirsearch’s Python implementation on large wordlists.
