Determine Whether the Origin of the Request is a Crawler May 9, 2017 A situation may arise in which you would need to hide certain sections of your page from crawlers or determine what actions to take based on the referrer of the