devxlogo

Determine Whether the Origin of the Request is a Crawler

Determine Whether the Origin of the Request is a Crawler

Why Trust Us

A situation may arise in which you would need to hide certain sections of your page from crawlers or determine what actions to take based on the referrer of the Request. You can use the HttpCapabilitiesBase.Crawler property to determine this.

In the sample below, the Request.Browser object represents HttpCapabilitiesBase object.

if (Request.Browser.Crawler){   HideCertainSections(); }
devxblackblue

About Our Editorial Process

At DevX, we’re dedicated to tech entrepreneurship. Our team closely follows industry shifts, new products, AI breakthroughs, technology trends, and funding announcements. Articles undergo thorough editing to ensure accuracy and clarity, reflecting DevX’s style and supporting entrepreneurs in the tech sphere.

See our full editorial policy.

About Our Journalist

Charlie Frank

Charlie has over a decade of experience in website administration and technology management. As the site admin, he oversees all technical aspects of running a high-traffic online platform, ensuring optimal performance, security, and user experience.
View Author
Show More