Webhook Workflow exceeding external server's rate limit

workflows

#1

We have set up a workflow which triggers a web hook to an external api service.

The external service is returning 429's because the workflow is exceeding it's rate limit.

According to the web site:

  1. Workflows will not retry after receiving 4XX or 5XX series response status codes.
  2. Workflows will issue webhook attempts at up to 20 requests per second.

Is it possible to change the "20 requests per second" to another value, say 5 per second? Since 429's are not retried, we are losing valuable data.

Please advise. Is there any way to throttle back these outbound web hook requests?

Thank you,
Jay


#2

Hi @jwright,

It's not currently possible to throttle workflow webhook requests; they're processed as quickly as the workflows tool processes contacts, which is 20 per second. If you require some form of queuing/throttling, your best bet would be to build some middleware to accept the webhook requests and queue them up, sending them to your external service at an appropriate rate. You might also consider looking into a 3rd party service that can handle this queuing/rate limiting for you.

Also if you have the inclination, I would strongly encourage you to check out the Ideas Forum on the HubSpot Community. There, you can create a post including your use case that product and other customers can see. I think this is valuable feedback, so I would definitely encourage you to create an idea. I'll also pass this along internally as well.


#3

A recent update to the "Using webhooks in workflows" page says that 429's will be retried. They are the exception to the 4xx and 5xx responses which are typically not retried. Not sure if this is a new feature or if it just wasn't documented before. But that really helps.

https://developers.hubspot.com/docs/methods/workflows/webhook_information


#4

Hi @jwright

Apologies for not updating this thread; that was a small update made recently that we didn't create an announcement for. Happy to hear it helps.