Should 404 also return Response content or error message?


#1

Hello,

We've been having issues with the API on our production server since approx 2/25. It worked fine prior to that. Now...
It does not matter what we attempt to do (add/ update company, contact, deal). When the code is executed from website running on production server, always get back 404 with nothing in either Response.Content or Response.ErrorMessage. Should either of these fields have data if it's a legit 404 and it authenticated okay?

Same exact code and test scenarios run from local dev computer (localhost) with same database connection, returns 200 for exact same calls against exact same records.

I added logging to better track what's happening but all calls to API are returning the 404 and no helpful info. Wondering if it should at least be returning the the portalId in response content if successful connect. If not authenticating, would expect a different code or error message.

I'm perplexed. Any assistance would be appreciated.

C# Code...

// API Endpoint Paramaters
        private static string ApiEndpoint = "https://api.hubapi.com/";
        private static string ApiUserAgent = "FMP Hubspot API Connector v1.0";


        private static JObject CallAPI(Method oMethod, string oUrl, string oQueryString, JObject oContent)
        {
             ApiKey = ApiKeyLIVE;

            string lRequestContent = "";

            if (String.IsNullOrEmpty(ApiKey))
            {
                throw new Exception("Hubspot API Key is not defined");
            }

            string url = string.Format("{0}{1}", ApiEndpoint, oUrl);

            RestClient client = new RestClient(ApiEndpoint);
            RestRequest request = new RestRequest("{url}", oMethod);
            request.AddUrlSegment("url", oUrl);
            request.AddQueryParameter("hapikey", ApiKey);

            request.RequestFormat = DataFormat.Json;
            request.AddHeader("Content-Type", "application/json");
            request.AddHeader("User-Agent", ApiUserAgent);
            if (oContent != null)
            {
// NOTE: tried both way - specifying Content-Type param and not in the AddParameter call
                //request.AddParameter("application/json; charset=utf-8", oContent.ToString(Formatting.None), ParameterType.RequestBody);
                request.AddParameter("application/json; charset=utf-8", oContent.ToString(Formatting.None), "application/json", ParameterType.RequestBody);

                lRequestContent = oContent.ToString(Formatting.None);
            }

            IRestResponse response = client.Execute(request);
            // @TODO - Add check to make sure our response is an object and not an array (wrap in {"result":response.Content} if not)
            setStatusCode(response);
            JObject result = String.IsNullOrEmpty(response.Content) ? new JObject() : JObject.Parse(response.Content);

            LogIt(ApiMode, ApiKey, ApiUserAgent, oMethod.ToString(), url, oQueryString, (int)response.StatusCode, response.Content.ToString(), response.ErrorMessage, lRequestContent);

            return result;
        }

Here are a couple examples of what's being sent. EVERY time returns 404 with no other info.

GET https://api.hubapi.com/contacts/v1/contact/email/Name@server.com/profile
POST https://api.hubapi.com/contacts/v1/contact


#2

@BWATKINS2000 Can you show me an example of when you are hitting the create a contact endpoint and receiving a 404? It should only be returning a 409 if there is a contact already made and maybe a different 400 level error if you formatted the call incorrectly.

On the get contact by email. The output is

{
  "status": "error",
  "message": "contact does not exist",
  "correlationId": "d9f2a0a9-d91d-4b12-a997-09054274ff11",
  "requestId": "4bafc4c8e0248f5adf1cf1a3bfafd367"
}

As you see the info would be in response.message. There is no Content or ErrorMessage Property in the object that gets returned.


#3

@pmanca -- Hi. Sorry I missed your reply. New to the forum and guess I thought I'd get an email notification. Here is an example...
POST
https://api.hubapi.com/contacts/v1/contact

RequestContent...
{"properties":[{"property":"fmp_last_activity","value":"1524658084305"},{"property":"fmp_company_contact_id","value":"10124"},{"property":"fmp_company_id","value":"2168"},{"property":"fmp_email_address","value":"email@server.com"},{"property":"fmp_comment","value":""},{"property":"fmp_contact_roles","value":""},{"property":"fmp_is_active","value":"true"},{"property":"fmp_referral_source","value":""},{"property":"fmp_referral_source_comments","value":""},{"property":"fmp_created_date","value":"1524653756927"},{"property":"fmp_created_by","value":"First C. Last"},{"property":"fmp_modified_date","value":""},{"property":"fmp_modified_by","value":""},{"property":"fmp_deleted_date","value":""},{"property":"associatedcompanyid","value":"82764387"},{"property":"hubspot_owner_id","value":"6935811"},{"property":"email","value":"email@server.com"},{"property":"salutation","value":""},{"property":"firstname","value":"FName"},{"property":"lastname","value":"LName"},{"property":"company","value":"Uncover"},{"property":"jobtitle","value":""},{"property":"linkedinbio","value":""},{"property":"city","value":""},{"property":"zip","value":""},{"property":"state","value":""},{"property":"address","value":""},{"property":"address2","value":""},{"property":"address3","value":""},{"property":"phone","value":"111-222-3333"},{"property":"mobilephone","value":""},{"property":"fax","value":""},{"property":"website","value":""}]}

Response code is 404 EVERY TIME for every call (Deals, Contacts, etc)
Error message is null
Response content is blank

It behaves as if it is not actually connecting to the Hubspot server.

I did a test on another account / web site and forced a legit 404. That actually returned Response Content

It would be great if there was a way to check logs on the Hubspot side to see if the request is actually getting to Hubspot. Any ideas as to website config that could impede traffic to Hubspot via the API would be greatly appreciated.

To add to the weirdness, this same exact code will work fine if I connect to the production database from my local dev computer (localhost). If executed via the production website, it returns 404 every time with NO response content. It appears to be WEBSITE specific. The same exact POST directly from the server (not the website) will go thru.

Thanks for any help you can provide. Really stumped here.


#4

Hi @BWATKINS2000,

Is it possible that the Content-type header is being excluded? It has to be set to application/json, and sometimes weird things happen when people leave that out. Also, if you use OAuth2 for authentication you'll have access to API Call Logging in your developer portal.


#5

Content-Type is specified in the code. Is there somewhere else it should be specified?


#6

Hi @BWATKINS2000,

Can you give me your Hub ID?


#7

The hub Id for the account having issues is... 720952


#8

Hi @Derek_Gervais,

In addition to providing Hub ID 72092, wanted to mention that the recently added / updated records were done so most recently on 5/6 outside the normal production server. I was using my work around -- using my local dev environment as the website but connected to the production database.

Appreciate any help you can provide.


#9

I typo-ed the Hub ID in the last post. Wanted to resubmit it. It is actually 720952


#10

Hi @BWATKINS2000,

Thanks for your patience here. So looking at some high-level request traffic for your portal (Hub ID 720952), I'm not seeing a particularly large percentage of errors. I'm only seeing ~22 404 errors over the past few days, and they all appear to be coming from the 'Get contact by email' or 'Update contact by vid' endpoints. Are you still running into issues with requests coming from your production server?


#11

Hi. Yes, we are still having issues with this. I have been manually doing a sync by connecting my local dev environment to the live DB and then can do a push -- calling same exact code -- and get the deals, contacts, etc to push. I am noticing that occasionally a transaction here or there will go thru via the live site.

I did a manual push 5/29 around 10 am. Will do another one shortly. If you look at dates from 5/29 post 10 am = 6/1, you will see lots of 404's -- At least I do!


#12

Here's a bit more narrow date range...
Between 5/30/2018 and 6/1/2018 11:59 PM
1247 transactions attempts
15 successful Deal updates (PUT) - status 200
1232 failed - 404 status code and nothing in the ResponseContent field

Are you able to confirm if those transactions are even making it to the Hubspot server ?


#13

Hi @BWATKINS2000,

I'm not seeing any of these 404s, but the logs only go back 7 days so I can't see anything from 5/29. Some next steps:

  1. Are you using a hapieky or OAuth for authentication? If you're using OAuth, can you post your appId?
  2. Is there a way for you to run a test of this code that can quickly reproduce these errors? I know turnaround here has been fairly long given the current forum volume, but if you can run a test and create a number of these errors I should be able to check our logs. It's suspicious that I'm not seeing any trace of these errors, but I want to completely eliminate the possibility that it's because the logs only go back 7 days.

#14

We are using ApiKey.

If you look at today - there are already 135+ transaction attempts. Only 2 were successful and those were a test from running the web app from my local DEV computer connected to the LIVE database.

I think there has to be something different about the environment on our LIVE web server vs my dev environment but I'm not sure what to look for. What minimum version of TLS do you guys require? Any minimum version of .NET at play?


#15

Hi @BWATKINS2000,

We support TLS 1.0/1.1/1.2, but not 1.3, and we don't have any explicit requirements on .NET but I would imagine newer versions would be better supported. I'm not seeing any of these errors in out error logging, but we don't store logs for request that don't make it past the TCP handshake stage, which leads me to believe these aren't showing up in the logs for that reason.

I'm thinking this might be an issue with the root CA or the supported ciphers of your production server. Can you make sure that your server and root CA is up to date? If you're updating based on the Mozilla Root CA Store you shouldn't have issues; also here are the ciphers we use:
https://www.ssllabs.com/ssltest/analyze.html?d=api.hubapi.com&s=2400%3Acb00%3A2048%3A1%3A0%3A0%3A6811%3Ac9cc&latest


#16

Hi @Derek_Gervais,

I'm now logging Response Headers among many other fields. We are getting a response Header value from cloudflare. Among other things in there is "Content-Length=0" which corresponds to empty Response Content. But response status code = 404 and no error message.

Are you able to examine traffic there to see if you can detect anything? If so, I can provide more info.


#17

Here is an example of a response that returns with status 404 but nothing in the Error Message. Nor did it raise any errors. The PUT request in this case looks just like one that works when I run the same code from my local dev environment connected to the production server, so I don't think it is formatting issue or else it would have more consistent results rather than works from one website but not another.

Response Headers:

Connection=keep-alive
Access-Control-Allow-Credentials=false
X-Trace=2B230815D26D5F6F362E34AA3325307FFB16903998000000000000000000
Strict-Transport-Security=max-age=31536000; includeSubDomains; preload
Expect-CT=max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
CF-RAY=43cf822fffbc56ff-IAD
Content-Length=0
Date=Thu, 19 Jul 2018 19:14:00 GMT
Set-Cookie=__cfduid=abc123; expires=Fri, 19-Jul-19 19:14:00 GMT; path=/; domain=.hubapi.com; HttpOnly
Server=cloudflare

I'm hoping that you can look at this from the cloudflare side to see if there's a way to tell if cloudflare is failing to pass it on for some reason. Or if there is an error of some sort that cloudflare is not passing back.

Appreciate your help in getting this figured out.

Thanks,
Belinda


#18

Hi @Derek_Gervais,

Hope you are doing well. Was hoping you could give a little more help / guidance on the issue we are experiencing. Seems our messages make it to cloudflare but are returning 404 with empty response / error message / etc. I can pull more current example if you can take a look. If I should be reaching out to someone else, please advise.

Thanks!


#19

Hi @BWATKINS2000,

Thanks for your patience here; this fell off my radar since it'd been a while between updates. I'm going to take this to my team to do some deeper digging; this seems to be occurring a layer deeper than I thought, so I'll need to dig in with them. If you have any other recent examples of this that'd be extremely helpful, but otherwise I'll post updates in this topic as I have them.


#20

I can provide all kinds of examples. Let me know specifics that are most helpful to you. I'm logging all info on request and response side.