OwlHoundBot: how OwlHound reads public web pages

What it is

OwlHoundBot is the name OwlHound uses when it reads public web pages for its own research and free tools, for example to check whether the links in an article still work. It is not a search engine and it does not copy your content to publish elsewhere.

How it behaves

• It obeys robots.txt. It reads your robots.txt first, at most once a day, and skips every page you disallow.
• It is slow on purpose. At most one request a second to any one site.
• It remembers what it read. Each page is kept for 24 hours, so reading it again in that time does not reach your server.
• It reads public pages only. No logins, no forms, and never private or internal network addresses.
• It always says who it is, with a link back to this page.

How to block it

Add this to your robots.txt. Rules you have already written for OwlHound apply too.

User-agent: OwlHoundBot
Disallow: /

Or email [email protected] with your domain and we will stop reading it.

Other OwlHound requests

Monitoring checks are different. When an OwlHound customer sets up monitoring, OwlHound-Monitor reads the addresses that customer gave us every few minutes, including whether the links on those pages still work. If you see checks you did not expect, email us and we will look into it.