TermSureBot
The crawler that reads school websites for TermSure.
What it does
TermSure shows each school's term dates, half terms and INSET days, with a link back to the school's own page. TermSureBot finds those dates: it visits a school's home page, follows links like "Term dates" or "Calendar", and reads those pages and the documents they link to.
It fetches about ten pages a visit, waits at least two seconds between requests to a site, and checks a site again about once a month.
How to check it's really TermSureBot
Every request TermSureBot makes is signed with Web Bot Auth: it carries a Signature-Agent header naming https://termsure.co.uk, and a signature you can check against the key at /.well-known/http-message-signatures-directory. A request that uses the name without a valid signature didn't come from us.
How to stop it
TermSureBot follows robots.txt. To keep it off your whole site, add this:
User-agent: TermSureBot
Disallow: /It also follows rules written for all crawlers (User-agent: *). When a site asks crawlers to stay away, TermSure links to the school's own website instead of showing dates.