Three Disadvantages of Cloud-Based Search Engines

Chad Johnson May 9, 2016

Last week I wrote about three advantages of ououd-based search engines. While the future looks bright for these tools, there are still potential downsides. This week I would like to discuss three disadvantages of cloud-based search engines, as well as potential mitigations. I am not saying that you should not consider cloud-based search engines because of these reasons. I am simply pointing out some reasonable concerns in the hope that you can make the most educated decision for your company.

1. Lack of Control

Not having to worry about the physical architecture of a search engine means that certain aspects of the implementation will be out of your control. If a cloud-based search engine is not performing fast enough, you cannot simply add another server instance or increase the RAM. If your users are geographically distributed, you cannot simply place a replica server in another location. Cloud services should perform well and they should offer geographic dispersion, but if they don’t, you are unlikely to be able to do anything about it yourself. You are at the mercy of the cloud for those types of configuration changes or improvements.

Three Advantage of Cloud-Based Search Engines

Chad Johnson May 2, 2016

Traditionally, enterprise search engines have been firmly planted on the ground. Now, along with almost everything else in enterprise IT, search engines are quickly moving into the sky. But why? What factors are prompting the growth of cloud-based search engines? Given the immense IT security (and mindset) challenges associated with indexing sensitive content in the cloud, why are vendors continuing to march full-speed in this direction?

1. Follow the Content

Almost all of my enterprise search clients are indexing at least one cloud-based content repository, such as Box, Salesforce, Google Drive, Office 365, etc. This historically involved writing connectors that download all of the information back to an on-premise search engine. As more content sources move to the cloud, it is becoming increasingly inefficient (and absurd?) to download all of the content back to the ground so it can be indexed.

Introducing the Application Indexing Protocol (AIP)

Chad Johnson April 22, 2016

I am excited to introduce the Application Indexing Protocol! AIP, for short, allows any application to proactively make itself indexable by a variety of third-party enterprise search engines. Unlike traditional connectors (created by search engine vendors or systems integration companies), application developers can now choose exactly what they want indexed and the best way to do it. Applications can choose the most appropriate indexing mechanism, either push (event-driven) or pull (polling), as both are supported by the Application Indexing Protocol. And because AIP is an open standard, applications only need to develop a single, canonical implementation in order to get their content indexed by numerous enterprise search engines. Gone are the days of every search engine vendor having to write its own version of every connector to every application. Now, the work can be done once, by the people who know the application the best.

[Chad wakes up from his dream…]

Drat! The Application Indexing Protocol does not actually exist [yet]. But I want it to so badly. I am tired of writing connectors for every new application, and for every new search engine. I am tired of trying to figure out how to efficiently and completely extract content and security trimming information from both on-premise and cloud-based applications, that may or may not provide appropriate APIs for this purpose.

Can Search Get Too Personal?

Chad Johnson April 19, 2016

The following article caught my attention this afternoon:

The UX Of Ethics: Should Google Tell You If You Have Cancer?

“If I’m on a park bench, and I’m next to someone, and I hear them talking about symptoms of cancer, am I obligated to turn around and tell them they might have cancer?”

… artificial intelligence products are rapidly approaching the same diagnostic power. Google Search can already predict coming flu trends with some level of success. It’s not hard to imagine a system that can track my searches over a year—an ache, a cough, a rash—recognizing a cascade of symptoms that point to a disease with surprising accuracy.

source: http://www.fastcodesign.com/3058943/the-ux-of-ethics-should-google-tell-you-if-you-have-cancer

I have not often considered the ethical implications of enterprise search. The article above reminded me that the information people type into those small search boxes can be extremely sensitive and revealing. Do you recall the 2006 release of anonymized search queries by AOL? Some very simple detective work made it possible to identify real people and real situations, even with usernames removed. For example, a vanity search for someone’s own name followed later in the day by a search for a rare medical condition or a financial problem. And it got more personal and embarrassing from there.

(Re)defining Enterprise Search

Chad Johnson April 15, 2016

Enterprise Search is changing. It used to be defined simply as finding documents in a large corporate repository or looking up specific information from some sea of data. But Enterprise Search is transitioning to something more broad and powerful. Enterprise Search now means reducing friction in your business operations. It means finding information you didn't know you were looking for. It works hands in hand with digital marketing to promote and sell goods and services. It provides unparalleled insights into what people are doing or looking for on a website or in an application.

I think it is time for an updated definition of Enterprise Search, and I could use your help to create one. If you have 30 seconds to spare, provide your own definition of Enterprise Search below (all data is collected anonymously). I am also curious if the definition is influenced by your job description or industry, so provide those if you like. Next week I will analyze the responses and report back on our modern definition of Enterprise Search.

Google Releases Cloud Vision API. I Eat Crow.

Chad Johnson March 3, 2016

Last year Google introduced groundbreaking image recognition technology as part of their release of Google Photos. Deep neural networks could identify objects, people and places inside of photographs with astonishing accuracy, completely autonomously.

As fascinating as this technology is, I was certain that Google would keep it locked away inside their own products like Google Photos and Google Drive for the foreseeable future. I did not anticipate it being sold standalone as an API.

I was wrong. Two weeks ago, Google announced a beta version of their Cloud Vision API, allowing anyone to process images and identify things like everyday objects, faces (w/sentiment) and text or logos.

For example, processing the picture below with the Cloud Vision API that identifies common objects …

One Size Doesn’t Fit All with Enterprise Search

Chad Johnson March 2, 2016

Last week I was starting to design a new demonstration environment for our Search and Knowledge Discovery practice. It would simulate a search application for a high tech call center, searching across a customer support case repository. I was having difficultly coming up with a scenario that would be applicable to a wide audience and include all of the diverse features that I wanted to show off. I realized I was falling into a trap just like many of my own clients.

Ask most people what an enterprise search application looks like and they will probably describe something very traditional – a search box across the top with 10 search results down the middle of the page. The results will come from predictable sources such as the corporate intranet or knowledge management repositories. And the experience will be the same for all users, except for security trimming that might hide certain results based on document permissions.

This kind of enterprise search application can satisfy some employees some of the time, but it is far from perfect. It doesn’t take into account the diverse job functions within a company and the different search needs of each. It doesn’t learn from experience and get better over time. And it doesn’t promote the discovery of information by exposing the relationships between different content sources.

Google Search Appliance Sunsetting: Week One Reactions

Chad Johnson February 11, 2016

It has been exactly one week since Google announced that it would be sunsetting the Google Search Appliance over the next three years. And what a week it has been. I have had the opportunity to speak with media representatives at Fortune and CIODive, I have spoken with several current and prospective customers of the Google Search Appliance, I have spoken with numerous other search companies, and I have chatted informally with our consulting team and friends in the enterprise search ecosystem.

I would like to share some of my initial thoughts to the news, and the rollercoaster of reactions that I, and others, have had.

Did the news surprise me?

The most common question I have been asked is if the news surprised me. Was I surprised that Google would focus more energy on a cloud-based service than a physical yellow box? Not at all. It makes good sense for Google to develop a technology that does not require shipping GSAs around the world and installing them in data centers. I have been a partner with Google for 7 years and I have come to expect periodic hard-left turns from them. They usually turn out to change an industry in retrospect.

Can You Quantify a Cold?

Chad Johnson February 2, 2016

How do computers see the world? Can they tell if we are happy? Can they predict what we are going to have for dinner next Tuesday? Can they recognize when we are sick?

Well, my iPhone can. Sort of.

I was (finally) recovering from a nasty cold last week and I opened the Health app on my phone and noticed that the step-counter had fairly accurately captured my cold, from start to finish. My normally active body had flatlined for over three weeks, except for one day where I thought I was getting better and went snowshoeing for 8 miles in the Mount Evans Wilderness. Bad idea.

So why didn’t my phone send me a sympathy card? Well, that’s the problem with data. It’s just data. It takes skillful, and contextual, interpretation to make sense of the data and to draw useful conclusions. With the step data, a computer might notice a trend or anomaly, but it doesn’t understand that it was caused by working out more or being sick in bed. That is the art of data science and knowledge discovery. A graph is just a graph until someone interprets it.

What’s New in Search Solutions – People Search 2.0

Chad Johnson January 20, 2016

For many years, People Search with the Google Search Appliance was essentially a glorified phonebook lookup. Our customers could search for colleagues by name or title, and retrieve basic information like phone numbers or addresses. Even the advent of ‘Expert Search’ did not tangibly change the situation — it just made the phonebook-style lookup easier to implement.

The Problem: In 2015, several of my clients came to us demanding more from People Search. We realized that these companies did not have just a single source for information about people, and that by investing in additional data aggregation and pre-processing, we could deliver more sophisticated and useful people search results.

The Solution: Instead of indexing only first-order fields about people (name, title, location, etc.), we started adding data like skills, project experience, previous jobs, and recently authored content. This expanded the number of ways that a person can be found, and it also increased the number of ways that the results can be sorted, ranked or biased. You can now search for people with similar skills, or who worked together on past projects, or who recently collaborated on a document about a certain topic.

Machine Learning Pt. 2: Applications in Enterprise Search

Chad Johnson August 5, 2015

In last week’s article, What is Machine Learning?, I gave an overview of Machine Learning and provided a link to a very informative visual guide that describes how the technology works. I also hinted at some useful applications within Enterprise Search. I would like to cover a few of those that I have seen in recent months.

Query Analysis with Clustering

There are often many ways to search for the same concept or idea. When we are reviewing search logs and trying to draw conclusions from what people are searching for, slight variations from query to query can make it difficult to do a comprehensive analysis. For example, we recently worked with a major airline and one of their top queries was “bereavement fares”, i.e. people looking for discounted fares when attending a sudden funeral. We knew that this particular query was popular, and that it needed some tuning to best serve the customers, but further analysis reveled other similar but different queries suffering from the same problem. The other forms of the query were less common, and if treated individually, they would have flown under the radar (no pun intended) in our analysis.

Ascending the Enterprise Search Value Map

Chad Johnson May 5, 2015

When I wake up in the morning I have a couple of choices for what to do first. I could brush my teeth. I could take a shower. Or I could fix that broken shelf in the garage.

Come again? Two of those sound very typical and easy. But the last one is less obvious, much harder, and not what most people would do first in the morning. That is, until I tell you the story about how a hammer fell off the shelf last night and nearly crashed through the windshield of my car. It sounds more important now, doesn’t it?

All work has value, but not all work has the same value. I could spend an hour with my kids drawing pictures on the sidewalk with chalk, or I could spend an hour with them sorting a box of beads and explaining how they can be groups by size or color or shape. Both tasks would be fun (theoretically), but working on basic math skills is arguably more valuable than drawing with chalk. I’m not saying they don’t both have value; one just happens to be more valuable than the other. Since I have limited free time, I should try to maximize the value of the work I do with the kids. The challenge is that drawing with sidewalk chalk is so much fun!