I noticed our servers are getting
HEAD requests from the following UA “HubSpot Connect 2.0 (http://dev.hubspot.com/) - UrlBlacklistsKafkaWorkers-urlchecksworker”. Hubspot seems to be crawling links in emails sent through the transactional API.
Why is Hubspot crawling links sent in email? Will a standard
HTTP 405 response stop this behavior? Or will the crawler see a
HTTP 405 response, from a
HEAD request, and start attempting