Own your mistakes. That’s a core tenet of the Customer Code that HubSpot announced on stage at INBOUND this week. We just didn’t know we’d have to put it to use so quickly. On the morning of September 6, an internal issue caused Marketing Hub enterprise customers to lose features and caused their websites to fail. Any outage is bad, but an outage that affects your website — your company’s digital front door to the world — and key tools like workflows is worse. For those affected, we are so sorry our mistake disrupted your business.
What Happened
At 04:38 EDT our team identified the issue, and at 5:45 EDT we identified the cause. The cause was a bug in the product configuration code that had ripple effects across the product. The most straightforward way to say it is that the bug caused affected HubSpot portals to temporarily forget they were enterprise and lose access to those features.
At 7:06 EDT we fixed the bug, and resolved the underlying issue. We are currently working to resolve issues that were a direct result of this outage. Here's a link to our status page.
This was a mistake and it was ours alone. We know what it feels like to be waiting for a resolution to a problem, and we’re sorry we put you in that situation. Thank you for hanging with us as we worked to fix the error and bring features and websites back up. We are lucky to have you as HubSpot customers and on this morning, we didn’t live up to our end of the relationship.
The copy above is pulled from a post to our customer blog written by HubSpot's COO, JD Sherman. You can find that blog post here.
Wanted to add some integrator-specific color here: This issue will have affected any access tokens created/refreshed during the relevant timeframe. If an installation failed during this time (likely with a BAD_SCOPES error) the installation will need to be retried.
For any integrations that successfully refreshed tokens during the outage, they might have gotten a token with a lower level of access. These tokens will gain the expected level of access on the next token refresh.
Please feel free to reach out below with any questions/concerns!
.png "HubSpot Alumni"){kind=icon}
