# ================================================================= # ===> agent <=== # ================================================================= User-agent: * Sitemap: https://www.pricewise.nl/sitemap.xml # ================================================================= # ===> Excludes for bots <=== # ================================================================= Disallow: /accounts Disallow: /iframes Disallow: /login_register Disallow: /login.aspx Disallow: /purchase/ Disallow: /rdr Disallow: /vergelijken/index Disallow: /angular-templates/ User-agent: rogerbot Disallow: / User-agent: Ezooms Disallow: / User-agent: webcrawler Disallow: / User-agent: BUbiNG Disallow: / User-agent: proximic Disallow: / User-Agent: trendictionbot Disallow: / User-Agent: Vagabondo Disallow: / User-Agent: Bilbo Disallow: / User-Agent: linkdexbot Disallow: / User-Agent: Exabot Disallow: / User-Agent: Slurp Disallow: / User-Agent: niki-bot Disallow: / User-Agent: CompSpyBot Disallow: / User-Agent: trendictionbot Disallow: / User-Agent: SEMrushBot Disallow: / User-Agent: BeetleBot Disallow: / User-Agent: Mail.RU_Bot Disallow: / User-Agent: 360Spider Disallow: / User-Agent: IstellaBot Disallow: / User-Agent: Plukkie Disallow: / User-Agent: melodibot Disallow: / User-Agent: spbot Disallow: / User-Agent: QuerySeekerSpider Disallow: / User-Agent: BuddhaBot Disallow: / User-Agent: imrbot Disallow: / User-Agent: AcoonBot Disallow: / User-Agent: yacybot Disallow: / User-Agent: GufyBot Disallow: / User-Agent: Nesotebot Disallow: / User-Agent: Sogou web spider Disallow: / User-Agent: SEOkicks-Robot Disallow: / User-Agent: SolomonoBot Disallow: / User-Agent: WBSearchBot Disallow: / User-Agent: NoyaBot Disallow: / User-Agent: FlappyBot Disallow: / User-Agent: SiteExplorer Disallow: / User-Agent: proximic Disallow: / User-agent: 008 Disallow: / User-agent: BLEXBot Disallow: / User-agent: crawler4j Disallow: / User-agent: Nutch Disallow: / User-agent: berlin-fu-cow Disallow: / User-agent: Riddler Disallow: / User-agent: python-requests Disallow: / User-agent: python-requests/2.8.1 Disallow: / # ================================================================= # ===> Allow: all others <=== # ================================================================= Allow: / # ================================================================= # ===> end <=== # =================================================================