> For the complete documentation index, see [llms.txt](https://f1shh.gitbook.io/pentest-tips/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://f1shh.gitbook.io/pentest-tips/web/content.md).

# Content Discovery

**Gobuster:**

```bash
# Discover Pages
gobuster dir -u http://<ip>/ -w /usr/share/wordlists/dirb/big.txt -x php,html,xml,txt

# Follow paths recursivly
gobuster dir -u http://<ip>/ -w /usr/share/wordlists/dirb/big.txt -e -r -f

# Fast discovery
gobuster dir -u http://<ip>/ -w /usr/share/wordlists/SecLists/Discovery/Web-Content/directory-list-2.3-medium.txt -x php,html,xml,txt
```

**GoSpider:**

```bash
gospider -S domain.txt -o output -c 10 -d 1
```

**Recursive download found sites:**

```bash
wget -r -i sitelist.txt

## Create a list with all files 
find . -type f -exec echo "{}" \; | sort -u > allfiles.txt
```