Search results include menu (html list)

[KenW]

Some of the results on the search page include a dropdown menu list items rather than content excerpt. The issue appears to be new. I contacted tech support but because I wrote the template they won't help me.

Any idea what the issue could be?

[mcoley]

@cadriel We use two types of extraction so we can build good snippets without missing results. The first kind analyzes the content on a page and extracts out the most relevant text (attempting to strip out things like menus, footers, etc.). The second kind does not filter any html out and is deemed less important to the search scoring algorithm. We use the first kind to build the result snippets to reduce noise from irrelevant HTML text. The text extraction is not perfect for all pages, and in the above case it was pulling in non-relevant data leading to bad snippets. I changed some of the filtering to be more strict and rebuilt the index for their portal, and all is well again.

[Cadriel]

@Matthew_Coley

Sweet, thanks for the explanation..

Secondly, I tried hunting for documentation surrounding your suggestion of adding the &length property to the API call - but didn't come up with much. Is this documented somewhere? And if not - could it be? Specifically the part about by using this parameter, the description would come from the pages meta description - I feel like this might have helped us out a bit.

Cheers!

[cbarley]

hi @cadriel, yup it's here: https://developers.hubspot.com/docs/methods/content/search-for-content. Command + F for "length"

[Cadriel]

@Connor_Barley

Out of interest - what was your fix? How does HubSpot index pages and produce the description for search results?

[cbarley]

Hey @KenW, I'm still working on this and digging in with my team here. I'll need to update you when I am able to find a solution. It's 5:30 here in Cambridge so I'll be digging into this tomorrow morning.

[KenW]

Sounds good, Connor. Thank you for your time.

For what it is worth, periodically, we record anonymized screens for user research. I found evidence of queries that worked as expected Oct 1 and the same queries now return the nav and menu lists (mentioned above).

All the best,
Ken

[cbarley]

@KenW, interesting, thanks for the extra info. I'll be sure to consider this with any investigating I do. I don't think we made updates to the way search works internally recently, but I could be wrong. I'll be sure to check with our CMS developers.

[cbarley · Accepted solution]

Hi @KenW, my apologies on the delay here. Just reached out to my team about this and they pushed out a fix so that the search results won't index your nav menu. If you check your oracle query now, you'll see the right results are pulling through.

As another option if you're looking to customize your code, you can clone your results module, and add a parameter to your search URL on line 49 by including the parameter &length=SHORT which will pull the description from the meta description of your blog posts instead, but this would require upkeep and making sure all your meta descriptions are up to snuff. This is documented here: https://developers.hubspot.com/docs/methods/content/search-for-content. Again, though, you should be good to go

[KenW]

Hi Connor,

I just checked the search results and they look good. I appreciate your attention to this issue and will communicate with my team.

All the best,
Ken

[cbarley]

Hi @KenW, I'm not aware of a common reason why this might happen, but would be happy to take a closer look if you send over an example of a search result page where there are results returned with the menu items showing up rather than an excerpt. Your HubID would be great to have as well! Thanks

[KenW]

Seems that now all of the results contain the "View by Product" menu item as the excerpt.
https://blog.toadworld.com/hs-search-results?term=oracle

[cbarley]

Hi @KenW, thanks for the page link! I'll take a look at this and get back to you when I have a solution