APIs & Integrations

tburger
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

below is what I started to get. Not sure if there is something we did to trip this though built a Proof of Concept that I’m trying to make changes to and then put into production. The API is on a shared hosted plan and could be part of it but no idea how to get past this as there is no active user to perform a captcha

Need help and quick if possible

Thanks

14 Replies 14
grover-results
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

I'm also getting 403 errors being thrown from our AZURE deployed production site. We have no issues from our offices, only once deployed.

I'm guessing this might be a similar issue. The azure server is 23.100.46.198

0 Upvotes
grover
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Any update on this issue. It's been two months now with our automated systems offline.

0 Upvotes
Derek_Gervais
HubSpot Alumni
HubSpot Alumni

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @techrivy,

Based on my investigation that IP address (207.174.215.236) did have a bad IP reputation that was triggering our CDN blocks. However this reputations has expired on Feb. 5th, and all now looks healthy. Can you monitor it going forward and let me know if you continue to see issues with reputation errors?

0 Upvotes
techreach
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @Derek_Gervais

Thank you very much for your response and help. I did pause all the
requests to the API on 3/4 Feb. And it did start to work on 5 Feb. Not sure
what resolved it. Did the pause expire the bad reputation?
If yes, then that could be noted for future. Thanks once again for your
help. Appreciate it.

Derek_Gervais
HubSpot Alumni
HubSpot Alumni

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @techrivy,

We don’t whitelist IPs from shared services, since it could allow bad actors to take advantage of the IP you’re using and potentially compromise the security of our system. You’re correct though that these security features regularly update based on the activity coming from a particular IP, so if the reputation of this IP changes these blocks are likely to clear. Are you seeing all of your requests fail due to these errors, or are you partially successful? Many API consumers find that pausing for a while (~ 1 minute at most) and retrying the request can help alleviate these issues.

0 Upvotes
techreach
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

@Derek_Gervais https://integrate.hubspot.com/u/derek_gervais

  1. All of my requests are failing.
  2. We do not hit this API 24 hours. It just gets executed sporadically
    about 25-50 times in a day.
0 Upvotes
Derek_Gervais
HubSpot Alumni
HubSpot Alumni

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @techrivy,

Is that IP address part of a shared IP service? Is it possible there are/were bad actors using that IP? There are a number of security features that might trigger this type of error. Can you rotate your IP address? Additionally, can you give me some idea for your request volume overall?

0 Upvotes
techreach
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @Derek_Gervais https://integrate.hubspot.com/u/derek_gervais,

Yes that IP address is a shared one and it is not possible to rotate /
change the IP address.
We do not do bulk transactions via the API at a single instance. But over
the day we might send 25-50 transactions on an average.
Is there any other way to white list my IP? I have read on other forums
that the rescan happens after 2 weeks and if no bad actor was found, it
would be white listed. Is this true?

0 Upvotes
Derek_Gervais
HubSpot Alumni
HubSpot Alumni

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @techrivy,

If you’re seeing this error, I would recommend scanning your devices/network for malware, as that’s the most common reason this page appears.

0 Upvotes
techreach
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi Derek,

Thank you for your response. I have scanned my shared server for malware
twice already and have not found any malware.
This is leading to a lot of manual uploads and overheads in my task since
we are not able to update the contacts via the API.

Can you please help in this regard.

Thanks,

0 Upvotes
techreach
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @Derek_Gervais.
We are also receiving the same error. Our server IP is 207.174.215.236
Can you please check if it is blocked?

0 Upvotes
Derek_Gervais
HubSpot Alumni
HubSpot Alumni

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @tburger,

The IP reputation system is not handled by HubSpot directly, but instead by our CDN provider. We can look into situations with extenuating circumstances, but on the whole we will not make exceptions to our security systems as this can potentially compromise the security of the entire platform. This is particularly true when using shared hosting; making exceptions for traffic coming from a shared IP address can allow bad actors to abuse the exception.

In the majority or cases, IP reputation blocks (or other CDN based security features) will automatically clear after a certain amount of time without any malicious looking behavior. The best solution in this particular case is to transition to a dedicated IP (like you’re already doing) and wait a few seconds (a minute at most) if you’re running into security blocks.

If you find that you continue to consistently run into security blocks after transitioning to a dedicated IP, feel free to reach back out here and we can continue to troubleshoot.

0 Upvotes
tburger
Member

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

The resolution that the CDN offers with a captcha doesn’t work for an API so there is not “self healing” approach as there is with a user in a browser. The CDN indicates that your company can make changes to allow activity from that IP should an issue arise, are you saying that you are unable to if need be?

Today, I will be putting up a new service on a dedicated IP that will both call the HubSpot APIs and also be the service endpoint for WebHooks. I’ll be calling a process that will rebuild a cached company table in another system. Its not huge and only 10k companies. It calls the “companies/v2/companies/paged” endpoint and should be processing 250 at a time. As well, we have reworked a few things on the WebHook end that should reduce calls back to HubSpot. Why am I saying all this? API calls could be getting caught up in general user rules/config of the CDN. What we are doing is not a huge load and always trying to always make things better. If we get identified as a issue from your CDN for some reason, we need assurance that HubSpot is going to do right and work with us as we have nothing but the purest of intention and use. Is this a fair request?

Thanks for looking into this and look forward to your response.

0 Upvotes
Derek_Gervais
HubSpot Alumni
HubSpot Alumni

Forbidden returned from API and Cloudflare picking calls up as a bot or threat

Hi @tburger,

These errors are a result of our CDN provider’s IP reputation system, and are triggered automatically by a number of their security features. If the IP address being used to access our API has a negative reputation, the CDN will return a captcha. This generally means the computer and/or network in question needs to be checked for malware. If you’re on a shared IP system, you should also inquire about rotating/changing the IP you’re using.

0 Upvotes