Email Association Overwrite from Contacts API


#1

Around 11/18 (also when the multiple email address associated with one contact launched), Intrigma had a lot of contacts in the database that their email address property for many contacts is being overwritten from the Contacts API they each night around midnight. It’s a one-way integration from Intrigma to HubSpot and it can create new users (free or paid) and update existing user details. Example of contacts below:

https://app.hubspot.com/contacts/488501/contact/5345/?interaction=note
https://app.hubspot.com/contacts/488501/contact/3312/?interaction=note
https://app.hubspot.com/contacts/488501/contact/1614634/?interaction=note
https://app.hubspot.com/contacts/488501/contact/6384/?interaction=note
https://app.hubspot.com/contacts/488501/contact/3821/?interaction=note
https://app.hubspot.com/contacts/488501/contact/4064/?interaction=note
https://app.hubspot.com/contacts/488501/contact/1044384/?interaction=note
https://app.hubspot.com/contacts/488501/contact/19993/?interaction=note (edited)


#2

@Derek_Gervais - looping you into this


#3

@lhoyland, thanks for creating this post and putting us in touch with @Derek_Gervais.

I’ll be pulling Marlen Pak, one of our engineers, into this conversation as well.

@Derek_Gervais, what is the next step here?

-Alex


#4

@awhite - can you reply/post any and all API methods they use on that topic? Not just contacts API but the specific endpoints, including any use of the forms api, embeded forms, tracking code customization, etc.


#5

@Derek_Gervais, forgive me for having to piece this together. The engineer who configured the integration for us originally is no longer with Intrigma.

I have determined that we are using the Forms API for our login page (form name: Logins). We disabled the integration on our end yesterday (12/12/2017) around 7:20pm. I’m not seeing any new form submissions from the Forms API since that time.

Marlen will add the specifics of the API configuration and endpoints before the beginning of the workday tomorrow.

Let me know if there is anything else that we should look into.


#6

We are using Contacts API for synchronization:

https://api.hubapi.com/contacts/v1/contact/createOrUpdate/email/[email]?hapikey=[apikey]

  • [email] and [apikey] are just placeholders

and Forms API each time user signs in (to Intrigma):

https://forms.hubspot.com/uploads/form/v2/[portalId]/[formGuid]

  • [portalId] and [formGuid] are just placeholders

#7

Hi @m-intrigma and @awhite,

Thank you both for getting this information all in one place, and posting here to the forum where I’m able to communicate with you directly. Also, thank you for your patience on this. This issue was notoriously tricky to pin down, with a lot of information from a lot of different sources. I’ve outlined my findings below.

In all of these cases, multiple email identities were merged into a single contact record. Because each record has multiple email identities, the update requests coming in from each email all change the same record in HubSpot. I can disassociate the email identities from each of these (and any other) records, which will result in a new contact record being created for each email address when you run your usual create/update job.

I’ve outlined a few steps you all can take to help mitigate this issue going forward:

  1. The forms API shouldn’t be used to track logins. Contacts that share a device (i.e. form submissions with different information) can overwrite existing contact information. Because contacts are limited to 1000 form submissions each, using them to track logins can make it difficult (or if you’ve hit the 1000 submission limit, impossible) to track real conversions. If you’re looking to track login information, you should use the HTTP events API (doc below).
  2. There are a huge number of associated usertokens/vids for each of these contacts. Contacts might have multiple vids if they have multiple devices or they’re clearing cookies from their browser, but the number that exist for these contacts seems to point to something else going on. Are you clearing visitor’s hubspotutk cookies, or perhaps loading the tracking code in an app that doesn’t persist cookie information? This type of behavior resulting in huge numbers of associated usertokens/vids can lead to association issues and lost analytics data.
  3. When the one-way sync is made from your system to HubSpot, it doesn’t appear you’re using the batch APIs. Wherever possible, you should batch your contact updates to reduce request volume (doc below).

I’m still working on figuring out exactly how all of these contacts got into this state. It appears that at least a few of these email identities were merged erroneously due to an internal job around July 4th, 2017. In other cases, it appears there was a manual merge of two contact records in the UI. Still others I’m not so sure of. I’ll continue digging into these examples to find where these merged identities came from, and update this thread with more information as I find it.



#8

@Derek_Gervais, thanks for the detailed response. @m-intrigma is going to estimate and plan the API changes you outlined. He’ll post again here on Monday if he has any further questions for you.

We will have to investigate the tracking code question further. We’ll let you know what we find.

The list of affected contacts in the first post was just a small subset of all those affected. Can you help us identify all affected contacts? I assume it would be anyone with multiple e-mail addresses and also anyone with lots of e-mail update events.


#9

Hi @awhite,

The easiest way to identify these contacts would likely be to add some logging to the update job you run every night. The response to the create or update endpoint includes a contact’s vid, which is unique for different contact records in HubSpot. Contacts affected by this (i.e. contacts with one record but multiple email identities) will return the same vid when you attempt to update both (or more) email addresses, so you can collect a list of vids that are associated with multiple email addresses in your system.

For example: The first example contact has two email identities (kadominguez and crang4) but one vid (5345). If you could log a list of these 'repeat vids’ and send that over to me, I could go about disassociating the email identities for you.


#10

@Derek_Gervais, thanks for that suggestion!

We think it might be easier to extract all e-mail addresses for all contacts from HubSpot adn then compare that to our database. Is that possible?

I tried the standard export all contacts method and it only seems to include the primary e-mail address…


#11

Hi @awhite,

You’d have to use the Contacts API to pull all contacts, since that response will include the identity-profiles information. You’d be interested in the contacts with multiple "EMAIL" identity profiles, though that’s not necessarily an indicator that they need to be ‘fixed’ since there are valid reasons to merge contact identities (like a merge ).


#12

@Derek_Gervais, thanks for that suggestion. Based on our latest planning session, I think we are going to proceed with updating nightly synchronization job to log the contact’s vid.

We’ll let you know once we have a list of Intrigma users that returned the same vid from HubSpot.


#13

Hi @awhite,

Sounds like a plan. If you could direct message me the list when you have it (ideally with the ‘real’ email address that a vid should be associated with) I can run a job to disassociate the other email identities from the contact records. After that, the next sync should create new contact records for the newly disassociated emails instead of updating the existing vids.


#14

@awhite - Since its been a week since you last commented, I wanted to reach out to close the loop and see if everything is working as designed.


#15

@lhoyland, thanks for checking in. We’re still working on this. Things are just moving a bit slower than normal because of the holidays.

@Derek_Gervais, I will have the list of affected contact vids and e-mail addresses for you tomorrow morning.

Talk soon!


#16

@lhoyland and @Derek_Gervais, I just sent the list of affected contacts over via direct message.