FactDriftBot, our crawler

What it is

FactDriftBot is the crawler behind FactDriftbot, a service that checks a website for facts that contradict each other, out-of-date numbers and dates, and broken links. If you see it in your logs, it is reading your public pages to prepare such a check, because you or someone at your organisation asked for one, or because we are preparing a sample report to send you.

How to recognise it

Every page it reads is requested with a user agent that starts with FactDriftBot/ and names this page:

FactDriftBot/1.0 (+https://factdriftbot.com/bot)

The version number changes, and a contact address may follow in brackets.

Its requests come from these addresses: 188.245.65.112, 2a01:4f8:1c1e:cf7f::1 (also as a list for firewalls).

What it reads

Public HTML pages only: the ones in your sitemap, or linked from your home page. It never logs in, fills in forms, runs scripts or reads anything your robots.txt rules out. A visit reads a limited number of pages, usually 100 or fewer, and most sites see one visit.

It also checks that the links on those pages still work, with one or two requests per link, always under its own name.

The text of the pages is read by an AI model to find the facts in them. We don't republish your pages.

How fast

At most two requests at a time, with at least half a second between them. If your robots.txt sets a Crawl-delay, it waits that long instead, up to 10 seconds. If your server answers 429, it slows down and leaves the rest for another day.

Blocking it

Add this to your robots.txt and it reads nothing:

User-agent: FactDriftBot
Disallow: /

Rules for User-agent: * apply to it too when you have none for FactDriftBot. Or write to hello@factdriftbot.com and we stop reading your site and delete what we have.

Letting it in

If you asked for a check and your firewall turned the crawler away, allow requests whose user agent contains FactDriftBot from the addresses above. In Cloudflare that is a WAF custom rule with the action Skip; other firewalls have an allow list that does the same.

Who runs it

FactDriftbot is a product of VTSOL INC. Questions and complaints go to hello@factdriftbot.com.