What are all of the possible Hubspot User Agents?

#1

We are blocking Hubspot requests for gated application URL's that Hubspot is trying to call, but want to perfect the regex in blocking them.

HubSpot Connect 2.0 (http://dev.hubspot.com/) - UrlBlacklistsKafkaWorkers-urlchecksworker
HubSpot Connect 2.0 (http://dev.hubspot.com/) - WebhooksPlatformService-web
HubSpot Links Crawler 2.0 (http://www.hubspot.com/)

Any more?

#3

Hi @terrance! Happy to help in any way I can. My team and I are wondering, though, what exactly are you trying to do? And why exactly would you need the URLs from us?

We'd love to do anything we can to help but just need a bit more info. Would you also be able to respond with your HubID and if you're creating a HubSpot application, your AppID? Thanks so much

#4

Hey @Connor_Barley, I took this convo offline last year with @Derek_Gervais : HubSpot Links Crawler 2.0 user agent hitting protected pages

Essentially Hubspot isn't respecting the subdomain's robots file (it only looks at root level domain robots) and Hubspot is trying to crawl gated application pages. This is flooding error logs with 403's so we block them at layer 5.