> For the complete documentation index, see [llms.txt](https://developers.chathive.app/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.chathive.app/crawler.md).

# Crawler

- [Overview](https://developers.chathive.app/crawler/overview.md)
- [Crawler config](https://developers.chathive.app/crawler/crawler-config.md)
- [Login](https://developers.chathive.app/crawler/crawler-config/login.md): This property defines how the crawler can authenticate it self to protected websites. Currently only the playbook is supported our must versatile and flexible authentication option.
- [Attributes](https://developers.chathive.app/crawler/attributes.md): This page outlines the attributes that can be used to configure your crawler.