CewlAI: Teaching Language Models to Think Like a Pentester
Hook
While most subdomain enumeration tools brute-force their way through DNS with massive wordlists, CewlAI asks a different question: what if AI could learn your target’s naming conventions and generate plausible domains on the fly?
Context
Traditional subdomain discovery follows a predictable pattern: throw massive wordlists at DNS resolvers and hope something sticks. Tools like subfinder and amass excel at passive enumeration through certificate transparency and search engines, while permutation tools like altdns mechanically swap characters and append common prefixes. But these approaches share a fundamental limitation—they’re pattern-matching against pre-existing knowledge, not learning from the target itself.
CewlAI, inspired by the classic CeWL wordlist generator and its DNS-focused cousin dnscewl, takes a different approach. Instead of algorithmic permutations, it feeds your seed domains to large language models—Gemini, OpenAI, White Rabbit Neo, or local Ollama instances—and asks them to recognize organizational naming patterns. If your target uses ‘prod-api-v2.example.com’ and ‘staging-db-primary.example.com’, an LLM can infer that they might also have ‘dev-api-v2.example.com’ or ‘prod-db-primary.example.com’. It’s pattern recognition rather than brute force, and it’s particularly valuable when attacking organizations with unique or creative naming schemes that don’t appear in public wordlists.
Technical Insight
The architecture follows Unix philosophy with support for three input methods: command-line arguments (-t example.com), file input (-tL domains.txt), or stdin piping. When you pipe data in, the tool automatically skips confirmation prompts, making it trivial to chain with other reconnaissance tools:
cat discovered_subdomains.txt | python main.py --limit 500 -o ai_generated.txtThe core loop includes practical safeguards. Before sending anything to the LLM, CewlAI estimates token usage and automatically manages input to stay within API limits—critical when you’re feeding it the output of a passive DNS scrape that returned thousands of historical subdomains. With the -v flag, you’ll see when truncation occurs to stay under token limits. The tool supports iterative generation via --loop, where each iteration feeds previous results back as new seeds, and --no-repeats ensures you don’t generate the same domain twice across iterations.
The model abstraction layer deserves attention. While Gemini is the default (requiring GEMINI_API_KEY), you can swap to OpenAI (-m openai with OPENAI_API_KEY), White Rabbit Neo (-m whiterabbitneo with KINDO_API_KEY from Kindo AI), or completely eliminate API costs by running local models through Ollama (-m ollama). This flexibility matters in operational contexts—cloud APIs offer better quality but cost money, while Ollama keeps everything local.
# Cloud-based with Gemini
export GEMINI_API_KEY='your-key'
python main.py -tL seeds.txt --loop 3 --limit 1000
# Local with Ollama
python main.py -m ollama -tL seeds.txt --loop 3 --limit 1000The output filtering is straightforward. CewlAI maintains a set of seen domains across iterations, excludes the original seed domains from output (you already have those), and writes results line-by-line to stdout or a file. The --limit parameter caps total output, which is useful for controlling generation volume. Verbose mode reveals the internal mechanics with debug output showing how many domains were suggested by the LLM versus how many were added after deduplication.
What’s not here is equally telling. There’s no DNS resolution built-in—CewlAI generates candidates, and you’re expected to pipe them to massdns, puredns, or similar tools for validation. There’s no complexity around rate limiting or retries because the LLM APIs handle that. This focused scope makes CewlAI a Unix-style component rather than an all-in-one solution, which is the right architectural choice for a reconnaissance tool.
Gotcha
The main consideration is that LLMs are probabilistic pattern matchers, not deterministic algorithms. The quality and relevance of generated domains will vary based on the model used and the patterns present in your seed data. Feed CewlAI a handful of seed domains and you’ll get back variations that may range from plausible to less useful.
Token usage is a practical concern when using cloud API models. CewlAI manages token limits automatically by truncating input when needed and showing estimates before processing begins. Use the --force flag to skip confirmation prompts, but be mindful of API usage with large inputs or multiple loop iterations. The Ollama option eliminates API costs entirely by running locally, though you need a working Ollama installation with appropriate models. For teams with operational security requirements, note that cloud-based models require sending domain data to third-party APIs (Google, OpenAI, or Kindo AI).
The tool’s output requires validation—there’s no built-in DNS resolution, so you’ll need to pipe results through resolution tools to determine which generated domains actually exist. The --limit parameter helps control output volume, and --no-repeats prevents duplicate generation across iterations.
Verdict
Use CewlAI if you’re conducting targeted reconnaissance against organizations with distinctive naming conventions and already have a seed list of discovered domains to teach the AI. It works well in bug bounty and red team scenarios where you’ve exhausted traditional enumeration but suspect additional infrastructure exists based on observed patterns—think companies that use environment indicators, version numbers, or team names in their subdomains. The tool is particularly valuable for organizations with unique naming schemes that don’t appear in public wordlists.
Consider alternatives if you’re doing initial broad discovery (passive tools like subfinder may be more appropriate), need deterministic reproducible results, or are targeting organizations with generic naming schemes where traditional wordlist-based tools already provide comprehensive coverage. The sweet spot is middle-stage reconnaissance where AI-based pattern recognition can identify variations that algorithmic permutation tools would miss. For cost-conscious scenarios or sensitive targets, the Ollama integration provides a local alternative to cloud APIs.
