← Government hub

PuchoPkBot

PuchoPkBot collects public pages and PDFs from official Pakistani government websites so that people can search them in one place. It always links back to the original page.

  • It identifies itself as PuchoPkBot/1.0 (+https://pucho.pk/government/about-crawler).
  • It reads /robots.txt first and obeys Disallow and Crawl-delay. If robots.txt cannot be read, the site is not crawled.
  • It waits at least five seconds between requests to one website, and stops for the day if a site answers with “too many requests” or refuses access. It never uses logins, never solves CAPTCHAs and never works around a block.
  • It only requests public pages and PDFs on the same website, and re-checks unchanged pages with conditional requests.
  • Pages marked noindex are not stored.

A government body that wants a site removed, slowed down or corrected can write to [email protected]. Removal takes effect immediately in the index.