Webhooks to https fails without details


#1

EDIT - a summary for the future reader. Original post below.

This applies when you get the message Something's not quite right. Are you sure this is the correct URL? while testing the webhooks API from the developers panel, get nothing in your application logs but can make it work using cURL.
It is probably an SSL issue.

In our case, we used to have a certificate delivered by StartSSL.
As of October 2016, those certificate are not to be trusted, so Hubspot rightfully don’t support them.

A good, trusted, and free alternative is Let’s Encrypt.
As of the 26th of October, Hubspot supports these certificate.


Hello!

I can’t seem to make the webhooks send messages to my server via https.
It works with http, but that request won’t be managed in my case, as all http requests get redirected to the homepage with https.
So it shouldn’t be a DNS issue, nor an IPTables filter.
It works with a service like ngrok, and https queries are routed correctly (in this case, to my local machine).

But it doesn’t work with the url to my server. I suspect an SSL issue, but no browser gives any SSL error message.
The application logs on my server are silent (error and access).
So it doesn’t even reach the web server level.
If I use the test tool in the edit modal of a subscription, I get Something's not quite right. Are you sure this is the correct URL?

Can someone from hubspot acknowledge if it’s an SSL issue or not? And if it is, what’s wrong?
I don’t want to leave the URL here but I’ll be happy to send it privately.

PS: here’s a link to a similar issue


Webhooks with HTTPS/SSL not working
#2

Hi @fonji

Sounds like this is happening with the new Webhooks API (and not a workflow)? If you’re still seeing this, can you message me directly with the URL you’re testing?


#3

Hi @dadams, thanks for your answer!
Sadly, it’s still not working.
I’ll send you a PM.


#4

Hi @dadams, I’m working with @fonji,

To complete on what we tried to receive webhooks events on our server:

  • Declaration of a new application
  • Associate portal to application via oauth
  • Configuration of a webhook event on Contact firstname
  • Edition of a Contact firstname in Hubspot CRM

We can see the webhooks on Hubspot side sent something (Total events: 1):
<img src="/uploads/default/original/1X/ff32ed8d6b62623d54e7ae6a4eb7b7d8ee261e27.png" width=“690” height=“121”

When going to Monitoring > Webhook Logging
<img src="/uploads/default/original/1X/519d06407774f115664c6f6a9cd25c4e4cf18ecb.png" width=“690” height=“392”

On our side we use:

  • Apache / Passenger as web server for a Ruby On Rails application
  • The website is only accessible in SSL, HTTP is redirected to https
  • Our staging server prompt with an htpasswd, but we allowed the webhook URI and confirmed the autorization worked
  • We use startssl as our SSL certificates provider ( startssl.com )
  • Our SSL VHost configuration is the following (Is the handshake ok ?):
SSLEngine On
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder On
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS

In Hubspot interface for webhooks
In Test subscription on Subscription details for an event:

  • The real https url return an error in a flash message. We don’t get any log in Apache logs:
  • The same url in http, which is redirected to the https (redirect 301) return a green flash message. We get a line in our logs
    <img src="/uploads/default/original/1X/680ebf2fe5d210b045d8456c850d58d4bd88f398.png" width=“690” height=“360”
    <img src="/uploads/default/original/1X/63abee2417760defcf17b2c2f55c2fffba367106.png" width=“690” height=“29”
  • When using the service http://requestb.in in the field as HTTP or HTTPS both works and we can see the details of the request.

Hope that can help debug this issue !

We are on webhooks since 3 weeks and have to present an MVP of the integration on monday, it become pressing for us to find solutions.

Cheers !


#5

Update on our side, we tried to change the SSL limitations and we also tried to change our code in case it was breaking Apache logs, without success for both.


#6

Answer from Hubspot:

Apologies for delay with this, our product manager has been discussing this issue with developers.

They have found that the issue lies with your CA StartCom, which is not listed as one of our trusted CAs.
We would add it but it seems that we are weary doing so based on security concerns raised in places such as https://news.ycombinator.com/item?id=12582534.
This is not to say that StartCom will not become a trusted CA but for the moment we are hesitant to add it.
Would changing your CA be an option at this point.

Kind regards,

Brian

StartSSL CA provider is not trusted and webhooks are not forwarded

So,
We also tried Let’s encrypt this morning. It’s not working either, despite being trusted by Mozilla.

Any advice on working CA certs with which level of validation ?


#7

Any advice on working CA certs with which level of validation ?

@dadams any idea?


#8

@fonji @pifleo are you still testing the same URL that you were previously having problems with? We just tested this again this afternoon and the test is showing success at this point.


#9

We tried many SSL providers and the url seem to work with a Comodo free certificate valid for 90 days ( https://ssl.comodo.com/free-ssl-certificate.php ), we saw the Success message on Hubspot interface, but with all the tests we did today our staging server is broken and can’t display some assets with our new SSL configuration and AWS Cloudfront. I’m on it.

Again this is a temporary solution before buying a real certificate. We wanted to confirm it is going to work with this provider.

From the previous link I would have assumed the webhooks can work with Letsencrypt certificates can you confirm this ? That would be a good solution for us and many people.


#10

@dadams we can’t make our CDN work with this certificate (more on that below for the interested).
We would really appreciate that you add support for Let’s Encrypt certificates ASAP.

They have a huge support from big players including mozilla, CISCO, the EFF, facebook, OVH, google chrome and HP.

About our problem with our CDN:
The certificate we generated with comodo uses EC instead of RSA and that is not supported by Amazon AWS’s CDN (cloudfront).


#11

It’s very likely that we will support Letsencrypt, if we don’t currently. I’m working with our platform team now to determine the status of that, or the timing if it’s not currently supported.


#12

Thanks for your quick response @dadams!


#13

I just got confirmation that we should support Let’s Encrypt certificates. Would you be able to switch back to that certificate, and let me know if you’re still having issues with the webhook test?


#14

Thanks! I’ll try to find some time today to test that and I’ll let you know ASAP.


#15

@dadams it didn’t work :frowning:


#16

@fonji are you still testing with the same URL as before? Testing that URL now shows success, and it doesn’t show a Let’s Encrypt/IdenTrust certificate.


#17

@dadams Yes I am using the same URL. I just changed the certificate and rolled back to the working one after my tests. So people here can test the integration. That’s why you don’t see the Let’s Encrypt certificate.
I’m not leaving an incompatible certificate for hours.

Here’s a detail of what I did, just to be clear:

  • Changed the conf to use the certificate from let’s encrypt
  • Restarted the server
  • Checked using my browser that the correct certificate (from let’s encrypt) was indeed loaded and deployed
  • Clicked “test” in the webhooks subscriptions interface
  • Got a “something’s not quite right. Are you sure it’s the correct URL” message, which is the one I get with an incompatible certificate (became sad)
  • Rolled back the server configuration to restore the working certificate
  • Restarted the server
  • Browser check
  • Clicked test
  • Got a success
  • Came here to report

#18

@dadams it still doesn’t work with Let’s Encrypt!


#19

@dadams any progress has been made? It’s been almost two weeks since your last answer!


#20

We’re planning on support Let’s Encrypt, but it’s being rolled out manually to our services, so it’s possible that webhooks don’t support this yet. I’m verifying this with our platform team, but there’s not a set ETA for when webhooks will support this yet.