TermSure

TermSureBot

The crawler that reads school websites for TermSure.

What it does

TermSure shows each school's term dates, half terms and INSET days, with a link back to the school's own page. TermSureBot finds those dates: it visits a school's home page, follows links like "Term dates" or "Calendar", and reads those pages and the documents they link to.

It fetches about ten pages a visit, waits at least two seconds between requests to a site, and checks a site again about once a month.

How to check it's really TermSureBot

Every request TermSureBot makes is signed with Web Bot Auth: it carries a Signature-Agent header naming https://termsure.co.uk, and a signature you can check against the key at /.well-known/http-message-signatures-directory. A request that uses the name without a valid signature didn't come from us.

How to stop it

TermSureBot follows robots.txt. To keep it off your whole site, add this:

User-agent: TermSureBot
Disallow: /

It also follows rules written for all crawlers (User-agent: *). When a site asks crawlers to stay away, TermSure links to the school's own website instead of showing dates.