“Are my GSA search results good?” Well, if you don’t know, then I don’t know. This is obviously a subjective question, and you likely know your own content much better than I do. While I may not be able to answer the question for you, I would like to offer some tips to help objectively measure the quality of your GSA search results.
Step 1: What are we measuring?
In order to make any conclusions about search result quality, we must first establish a set of queries to evaluate. There are infinite possibilities, but we can use some smarts to pick a good list of queries to measure. Here are a few suggestions:
Most popular queries
Worst-performing queries
Queries that returned no results
Queries where the user paginated very deep into the results
Queries where the user did not open any results
Queries where the user opened a bunch of different results in quick succession
Trending queries
Queries with a significant increase over a short period of time
Seasonal or event-related queries
Select a representative set of queries using these techniques. Keep the list manageable (50-100 total would be a good goal) because you are going to have to do some work for each individual query in the next step. You will also need to repeat this selection process periodically; automating the log file analysis could be a worthwhile effort.
Step 2: Establish a baseline
Once you have a list of queries to measure, we need to establish a manual baseline for each one. For each query, document one *ideal* search result (URL) that you would expect to be near the top of the search results. For example:
| Query | Desired Result | |
|---|---|---|
| contact us | https://www.perficient.com/About/Locations | |
| stock | https://www.perficient.com/Investor-Relations | |
| blogs | http://www.perficient.com/Thought-Leadership/Social-Media/Blogs | |
| ... | ... |
You do not have to identify *the* #1 result. We are going to use averages and trends over time to judge the quality of the results. If a desired result consistently comes back in the #2 spot, week in and week out, that could be an acceptable result.
Run a search for each of your baseline queries and document the actual position of the desired result. (Hint: run a GSA query with num=1000 and remove the proxystylesheet parameter. Use Control-F to search the XML for your desired URL. Look for <R N=”x”>, where x is the search result’s position). You might assign a 0 if the result comes back as a KeyMatch. This scoring process is another good candidate for an automated script. (FYI: We have a reusable search quality toolkit in the works…).
For example:
| Query | Desired Result | Actual Result | ||
|---|---|---|---|---|
| contact us | https://www.perficient.com/About/Locations | 2 | ||
| stock | https://www.perficient.com/Investor-Relations | 63 | ||
| blogs | http://www.perficient.com/Thought-Leadership/Social-Media/Blogs | 1 | ||
| ... | ... |
Step 3: Lather, Rinse and Repeat
Doing this for the first time might yield some interesting results, or it might not. It might be necessary to implement a few corrective measures and run the same test queries over again next week.
Corrective measure might include:
Implementing KeyMatches to promote “best bets” for the worst queries in your list
Implementing biasing policies if results from a certain content source are consistently low in the rankings
Adjusting the content of underperforming pages to better match how people are trying to find it
This analysis is not the Holy Grail of search result quality, but it might be the Rosetta Stone. It can definitely help you spot egregious problems. But more importantly, by moving the problem from something subjective to something objective, you can detect statistical changes in the quality of the results after each adjustment you make. If a certain change does not improve your overall ‘score’, you can roll it back and try something else.
| Query | Desired Result | Actual Result (WK1) | Actual Result (WK2) | ||
|---|---|---|---|---|---|
| contact us | https://www.perficient.com/About/Locations | 2 | 2 | ||
| stock | https://www.perficient.com/Investor-Relations | 63 | 0 | ||
| blogs | http://www.perficient.com/Thought-Leadership/Social-Media/Blogs | 1 | 1 | ||
| ... | ... | ||||
| AVERAGE SCORE | 2.4 | 1.8 | |||
You should also periodically rerun your log file analysis to update the list of poorly performing queries or trending queries. Ideally, you should see queries fall off of the worst-performing query list, making room for other poorly performing queries that need help. I’m not sure you will ever reach the bottom of that barrel, but you should see the average hit count for the worst-performing queries drop, meaning you are moving away from the left-edge of the graph, so to speak, and into the long tail of queries that will have less impact on overall result quality.
And finally, when your boss asks you if your GSA search results are good, you can now say yes, and show them why.