Machine Learning Pt. 3: We Need to Build a Computer/Brain Thingy

Chad Johnson August 19, 2015

In previous posts I have discussed the algorithms and applications of machine learning. One of the most sophisticated forms of machine learning is building a neural network that mimics the flesh-and-blood brain – with software- or hardware-based versions of neurons and synapses. But silicon operates differently than the stuff in our brains. In the past, researchers have suspected an entirely new type of computer chip would be needed – something radically different from our current silicon- and transistor-based CPUs and GPUs.

But, leave it to creative engineers to find a way. IBM has invented a new type of computer chip, dubbed TrueNorth, that could get us closer to the sci-fi dream of a silicon brain. Using traditional semiconductor manufacturing techniques (instead of trying to mimic the actual physics and chemistry of the brain), IBM has gotten as close as they can to modeling the way a real brain works.

Machine Learning Pt. 2: Applications in Enterprise Search

Chad Johnson August 5, 2015

In last week’s article, What is Machine Learning?, I gave an overview of Machine Learning and provided a link to a very informative visual guide that describes how the technology works. I also hinted at some useful applications within Enterprise Search. I would like to cover a few of those that I have seen in recent months.

Query Analysis with Clustering

There are often many ways to search for the same concept or idea. When we are reviewing search logs and trying to draw conclusions from what people are searching for, slight variations from query to query can make it difficult to do a comprehensive analysis. For example, we recently worked with a major airline and one of their top queries was “bereavement fares”, i.e. people looking for discounted fares when attending a sudden funeral. We knew that this particular query was popular, and that it needed some tuning to best serve the customers, but further analysis reveled other similar but different queries suffering from the same problem. The other forms of the query were less common, and if treated individually, they would have flown under the radar (no pun intended) in our analysis.

What is Machine Learning?

Chad Johnson July 30, 2015

What is Machine Learning? Well, I have a general idea of how to explain Machine Learning – it is the process of training a computer using historical data and decisions, such that the computer can then make its own decisions in the future using all new data that it has never seen before. For example, machine learning is used to train computers to read handwriting by studying countless examples of known writing samples. It is also used to study market data and make predictions about when to buy or sell.

This has given me a pretty good “black-box” understanding of machine learning algorithms. I understand that you feed in a bunch of data along with historical decisions and outcomes. Then a miracle occurs. And now the machine can make its own decisions in the future. If I feed the black box a million different handwritten letter A’s, and I tell it that all of them are the letter A, it should now be able to see a letter that it has never seen before and tell me with some confidence whether or not it is a letter A.

Spam is dead! Well, at least for Google Mail users

Chad Johnson July 20, 2015

Last week, Google announced some astounding statistics about the success rate of their spam filtering technology in Google Mail. Google says that less than 0.1% of email in the average user’s Gmail inbox is spam, and the rate for legitimate email ending up in the spam folder is even lower at less than %0.05.

Despite these superb results, Google is continuing to innovate and find new ways of detecting and blocking spam. This year, Google has announced several innovations across their product line that utilize their machine learning technology, affectionately referred to as Google Brain. Using their artificial neural network software, Google Photos can identify objects in your photographs, like dogs or cats or bridges or birthday cakes. Google Maps can automatically detect new businesses or speed limit signs in the imagery collected by their entire fleet of Street View cars. Google Earth Engine can detect new refuge camps or deforestation boundaries by scanning for visual patterns in petabytes of satellite images.

Putting your Google Search Appliance index on a diet

Chad Johnson July 16, 2015

Setting up a web crawler in the Google Search Appliance is a piece of cake — you enter a starting URL and some boundaries and let it rip. The GSA will spider its way around until it finds every reachable page in the site. For a well-structure site, this usually produces very good results, but not all sites are created equally. While the GSA does have features for detecting cyclical loops and excessively redundant pages, it will often find significantly more pages than you expect. This can cause a reduction in the quality of your search results. The extra pages can dilute the relevancy of higher-quality pages, making it difficult to find the desired results.

In the worst-case scenario, the GSA index size will reach the licensed limit, resulting in only part of your site being indexed and searchable. In this situation, the GSA starts evicting pages to make room for presumably better pages, but in practice, the eviction algorithm is not perfect and can result in essentially random pages being removed. Regardless, eviction or truncation is not a happy thing and you will want to take action to fix the problem.

What do Google’s I/O announcements mean for the enterprise?

Chad Johnson May 29, 2015

Google announced a wide variety of new products and features during the Google I/O 2015 Keynote yesterday. At first glance, most of them are consumer focused. Google Photos, Brillo for smart home automation, virtual reality headsets, etc. However, several of the announcements or themes have an impact on the enterprise world.

Globalization and Mobile First

Google emphasized many times during the keynote how little of the world is currently covered with fast broadband. They are now deploying special low-bandwidth versions of their most important sites like google.com, Maps and YouTube to help the rest of the world access them reliably. Those emerging markets are almost entirely “mobile first”, or really “mobile only”. Most new customers in India and China will *never* own a desktop computer. They rely exclusively on mobile computing devices.

When building enterprise applications that could possibly be accessed outside of the US or a few other select countries, we should follow Google’s lead. Don’t overlook the need for low-bandwidth, offline-capable, mobile-ready configurations.

Ascending the Enterprise Search Value Map

Chad Johnson May 5, 2015

When I wake up in the morning I have a couple of choices for what to do first. I could brush my teeth. I could take a shower. Or I could fix that broken shelf in the garage.

Come again? Two of those sound very typical and easy. But the last one is less obvious, much harder, and not what most people would do first in the morning. That is, until I tell you the story about how a hammer fell off the shelf last night and nearly crashed through the windshield of my car. It sounds more important now, doesn’t it?

All work has value, but not all work has the same value. I could spend an hour with my kids drawing pictures on the sidewalk with chalk, or I could spend an hour with them sorting a box of beads and explaining how they can be groups by size or color or shape. Both tasks would be fun (theoretically), but working on basic math skills is arguably more valuable than drawing with chalk. I’m not saying they don’t both have value; one just happens to be more valuable than the other. Since I have limited free time, I should try to maximize the value of the work I do with the kids. The challenge is that drawing with sidewalk chalk is so much fun!

Much-maligned Google Glass will get a second chance

Chad Johnson March 24, 2015

I tried on Google Glass a few times. I wasn’t impressed. Does that mean it’s an awful product? No. Does that mean it is not useful? No. So what happened?

This week the Wall St. Journal reported that Google is not giving up on Google Glass. Google acknowledges that it was a first generation product with some rough edges, but they still see a future for wearable computing.

[Eric Schmitt, Google Executive Chairman] said Glass, like Google’s self-driving car, is a long-term project. “That’s like saying the self-driving car is a disappointment because it’s not driving me around now,” he said. “These things take time.”

I kept my initial impressions in check. I was certainly impressed with the execution of such a radical new product, but I knew the first version was not meant to be the Nintendo 3DS or Sony Walkman – some wildly popular consumer electronics device that everyone will own. It was a proof of concept that would allow Google to learn what worked and what didn’t work in the wearable computing space.

Google Search Appliance Connector Development: Discovery Phase

Chad Johnson March 10, 2015

Sorry… this post is not going to teach you how to build a connector. Maybe next time.

Instead, I am going back a little earlier in the process. I want to describe the process for qualifying our ability to build a new connector in the first place. How do we estimate the level of effort? How do we ensure that we will have sufficient access to the content or that the necessary APIs exist?

First, not all integrations or connections are alike – there can be wide variations in complexity or level of effort. When we run through this qualification process, we always evaluate our options in order, from easiest to most difficult. For each option, there is a set of criteria that must be met in order to be successful. If any criteria cannot be met, we move on to the next-most-complex option.

Zen and the Art of Search Statistics

Chad Johnson October 10, 2014

Yesterday a client asked me what Key Performance Indicators (KPIs) they should be tracking on their public site search. At first I was afraid I would not be able to answer this question – I don’t know their business or website nearly as well as they do. I know the wide variety of statistics that are possible to capture, but what should I recommend to this particular client?

I thought for a moment and realized they needed to know their goals for search before diving headfirst into any analysis. There is no point capturing and analyzing search statistics unless you need them. They should have a clear reason or objective behind each KPI they track. Second, they needed to think in terms of deltas or comparisons instead of raw numbers or simple orders of magnitude. Statistics like quantity of searches or frequency of keywords are unlikely to reveal anything by themselves, but observing something increasing or decreasing over time can tell a very interesting story. Likewise, comparing any certain statistic between users that ran a search vs. users that did not run a search can be very revealing.

Is your Google Search Appliance platform secure?

Chad Johnson August 11, 2014

If you have read Google’s product literature, you know that the Google Search Appliance is a very secure device. The bright yellow appliance runs a hardened version of CentOS, and the inner-workings are safely hidden behind root login.

So, assuming we are dealing with an appliance with Fort Knox-level protection, what risks remain? Below are several potential vulnerabilities that could jeopardize the security of your GSA platform. Some of these risks can be mitigated easily, while others (particularly those involving human beings) may never be 100% avoidable. I am not trying to cause panic. I only hope to better educate the community so that simple risks can be avoided, and more complex risks can be appropriately understood and mitigated.

Administrator access

The GSA admin console offers two levels of accounts, Administrator and Manager. In a perfect world, we would issue most accounts at the Manager-level. But in practice, Manager-level accounts are not typically powerful enough to satisfy the needs of most users. Manager accounts cannot adjust crawl URL patterns, manage query expansion files, or even set up dynamic navigation facets. Because of this, I find that a most accounts in the GSA admin console are created at the Administrator level.

Are my GSA search results good?

Chad Johnson July 28, 2014

“Are my GSA search results good?” Well, if you don’t know, then I don’t know. This is obviously a subjective question, and you likely know your own content much better than I do. While I may not be able to answer the question for you, I would like to offer some tips to help objectively measure the quality of your GSA search results.

Step 1: What are we measuring?

In order to make any conclusions about search result quality, we must first establish a set of queries to evaluate. There are infinite possibilities, but we can use some smarts to pick a good list of queries to measure. Here are a few suggestions:

Most popular queries
Worst-performing queries
- Queries that returned no results
- Queries where the user paginated very deep into the results
- Queries where the user did not open any results
- Queries where the user opened a bunch of different results in quick succession
Trending queries
- Queries with a significant increase over a short period of time
- Seasonal or event-related queries

Reverse engineering the GSA Query Suggestions feature

Chad Johnson July 7, 2014

Google often quotes that the average query submitted to their website is 1.7 words long. That means that most queries contain only two words, and a fair number contain only one word. Google can be magical at times, but using one or two words to search through trillions of web pages seems at best hit or miss. In practice, longer and more specific queries tend to produce better results.

Query Suggestions to the Rescue

How do we encourage users to use more than 1.7 words in their query? The Query Suggestions feature is one popular and effective solution. By presenting the user with a list of suggested queries below the search box, they only have to type a few letters before seeing a list of queries to pick from.

Plexi (Part 2) – Adaptors are evil!

Chad Johnson June 25, 2014

Previously I discussed the Plexi Adaptor framework for the Google Search Appliance. Adaptors can provide a simple and elegant way to index a content repository. An Adaptor sits in front of a repository, making it behave like a web site from the GSA’s perspective. This off-loads the state-management and queue-management to the GSA’s built-in web crawler, simplifying the implementation.

But I mentioned that I was leery of Adaptors. I have written dozens of connectors and I like the control that the Connector Manager framework affords. I have written Connectors that do not seem transferable to the Adaptor methodology. I would like to discuss a few reasons that I am not completely letting go of Connectors.

Change Detection

When I implement connectors with complex hierarchical ACL’s or that require joining multiple database tables, I often have to track the state of multiple objects to do change detection. For example, the ACLs in our Atlassian JIRA Connector take into consideration various objects, including the Project, Permission Schemes, Issue Schemes and Custom Attributes, to compute the ACL for a single Issue. Changes could occur in any of these objects, with ripple effects throughout the entire repository. Implementing stateless change-detection with a Retriever would be very difficult because these permission objects, and the complex interactions between them, do not have timestamps to reveal modifications. Instead, we store a snapshot of the permissions for each object in our Connector, and that allows us to quickly check for even subtle changes to the permissions. There is nothing that prevents an Adaptor from storing this kind of state information, but it goes against the recommended design.

Sequential Iteration

Google’s attention to detail – down to a single pixel

Chad Johnson May 30, 2014

I have always been impressed with Google’s attention to detail. The custom yellow shell around the Google Search Appliance is completely useless, but I suspect it has enhanced the value of their trademark. GMail’s reminder when you mention an attachment in your message but forget to actually attach a document is downright useful.

Two recent examples caught my attention, and even made me chuckle a little.

Moving Shadows

Last year Google upgraded the desktop version of Google Maps to use vector rendering instead of static image tiles. It improved the performance but also allowed for some cool 3D rendering tricks. Have you noticed that the 3D shadows cast by buildings and tall monuments change throughout the day according to the sun’s position? Static shadows look static. Dynamic shadows look real. Or just cool, like that magical map in Harry Potter. (via. http://kottke.org/14/05/google-maps-impressive-attention-to-detail)

Plexi: GSA adaptors for non-HTTP content repositories

Chad Johnson May 21, 2014

The Google Search Appliance has historically required Connectors to facilitate the indexing of non-HTTP content repositories. A Connector is a program that sits between the GSA and a repository, and pushes the GSA everything it needed to know to index the repository (including the content, metadata, and security information). A Connector can use any technique to interrogate the repository (APIs, database queries, etc.), and the Connector is wholly responsible for deciding what to send, and when.

But a new methodology is emerging. Google is transitioning from Connectors to Adaptors. You can read Google’s description of an Adaptor here. I’ll admit, I have read this a few times, and talked to Google engineers and my team members about it, but I didn’t immediately grasp the value. I am comfortable developing Connectors, so why change?

Last week I was speaking to a group of prospective GSA customers at Google HQ. While I was taking some Q&A from the audience, it clicked. Someone asked me how connectors keeps track of new, updated, or deleted items to ensure the GSA is in sync with a repository. I mentioned that our Connectors either rely on an API that will stream us those events, or the Connector maintains a snapshot of the repository’s current state and does a periodic comparison to figure out if anything has changed.

The GSA’s ‘Out of the Box’ XSL template, with several features enabled.

GSA UI: Should I use the on-board XSLT engine?

Chad Johnson May 13, 2014

The Google Search Appliance ships with a full-function — but no thrills — XSL template (XSLT) that we refer to as the ‘out of the box’ user interface. It looks a little bit like google.com did a few years ago – lots of blue and white, and very few graphics. But it does support all of the enterprise features that the GSA includes, such as Dynamic Navigation, Document Preview, Expert Search, and more.

You can think of the XSLT as a skin on top of the XML results that the GSA produces. When you run a search, the GSA generates an XML document with your results, and then optionally applies an XSL template of your choice to that XML. The XSLT converts the XML into HTML, and then the GSA’s web server delivers the HTML to your browser.

Staffing a successful Google Search Appliance implementation

Chad Johnson May 5, 2014

Congratulations, you have either purchased or are interested in purchasing the Google Search Appliance. You will soon be able to search until your heart is content. Your friends will be jealous.

Now might be a good time to think about the staff required to deploy and maintain your cool new toy investment.

The Google Search Appliance, being an appliance, does not have exactly the same staffing profile as other enterprise search engines. Some people will go so far as to claim it requires virtually no staff to deploy or maintain. You just plug it in and it works, right?

Sort of.

I would like to discuss three critical people (or roles) you will need for your implementation, and roughly when they will need to engage throughout your implementation. These may be applicable to other vendor’s solutions, but I am going to specifically address the types of maintenance the GSA requires.

Where does metadata come from?

Chad Johnson April 24, 2014

In my last post, I discussed several ways that metadata can be used in a Google Search Appliance solution. But where does metadata come from?

Let’s hold on to that question for just a second, and start with an even simpler one.

What is metadata?

Every record in a Google Search Appliance (such a document, a web page) can have an arbitrary set of name / value pairs called metadata. The name and value can have pretty much any value, and you can include as many different metadata fields as you want. You do not have to define any schema ahead of time. Metadata values are interpreted as strings, but the GSA is also smart enough to recognize dates and numbers automatically. Metadata fields can include multiple values by either repeating the same field several times, or by using a multi-value separator, like a comma or pipe.

Why do I need metadata?

Chad Johnson April 15, 2014

When architecting Google Search Appliance implementations, I am often asked if additional metadata needs to be loaded into the GSA for the project to be successful. The client is usually considering the cost and benefit of manually or automatically tagging additional metadata for every document.

I typically address this question by pointing out precisely how metadata is used by the Google Search Appliance. If the additional metadata is not needed for one of the reasons listed below, it is not needed. Metadata is not like chocolate – more is not always better.

Why do I need metadata?

Search: Metadata is included as part of the searchable text of an item in the GSA. If the user searches for a word or phase or number that appears in the metadata, it can cause that item to be returned in the search results. Metadata is not included as part of the title or snippet, so it provides a way to make certain text searchable without necessarily showing up in the title or snippet.
Display: Metadata can be returned with each search result in the XML output, and can be used when constructing the search results display page. This could include thumbnail URLs, authors, dates, price, description, etc. Other than the Title, URL and Snippet, anything else shown on the search results page usually comes from metadata.