CeWL

CeWL Package Description

CeWL is a ruby app which spiders a given url to a specified depth, optionally following external links, and returns a list of words which can then be used for password crackers such as John the Ripper.

CeWL also has an associated command line app, FAB (Files Already Bagged) which uses the same meta data extraction techniques to create author/creator lists from already downloaded.

tools included in the cewl package

cewl – Custom wordlist generator

root@kali:~# cewl --help
CeWL 5.4.3 (Arkanoid) Robin Wood (robin@digi.ninja) (https://digi.ninja/)
Usage: cewl [OPTIONS] ... <url>

    OPTIONS:
    -h, --help: Show help.
    -k, --keep: Keep the downloaded file.
    -d <x>,--depth <x>: Depth to spider to, default 2.
    -m, --min_word_length: Minimum word length, default 3.
    -o, --offsite: Let the spider visit other sites.
    -w, --write: Write the output to the file.
    -u, --ua <agent>: User agent to send.
    -n, --no-words: Don't output the wordlist.
    --with-numbers: Accept words with numbers in as well as just letters
    -a, --meta: include meta data.
    --meta_file file: Output file for meta data.
    -e, --email: Include email addresses.
    --email_file <file>: Output file for email addresses.
    --meta-temp-dir <dir>: The temporary directory used by exiftool when parsing files, default /tmp.
    -c, --count: Show the count for each word found.
    -v, --verbose: Verbose.
    --debug: Extra debug information.
     
    Authentication
    --auth_type: Digest or basic.
    --auth_user: Authentication username.
    --auth_pass: Authentication password.
     
    Proxy Support
    --proxy_host: Proxy host.
    --proxy_port: Proxy port, default 8080.
    --proxy_username: Username for proxy, if required.
    --proxy_password: Password for proxy, if required.
     
    Headers
    --header, -H: In format name:value - can pass multiple.
     
    <url>: The site to spider.

fab – Files Already Bagged

root@kali:~# fab-cewl --help
fab-cewl

Usage: fab-cewl [OPTION] ... filename/list
    -h, --help: show help
    -v: verbose
   
    filename/list: the file or list of files to check

cewl Usage Example

Scan to a depth of 2 (-d 2) and use a minimum word length of 5 (-m 5), save the words to a file (-w docswords.txt), targeting the given URL (https://example.com):

root@kali:~# cewl -d 2 -m 5 -w docswords.txt https://example.com
CeWL 5.4.3 (Arkanoid) Robin Wood (robin@digi.ninja) (https://digi.ninja/)
root@kali:~# wc -l docswords.txt
13 docswords.txt