Google Cloud Search: First Impressions

Chad Johnson February 13, 2017

On February 14th, Google G Suite Business and Enterprise edition customers will begin to have access to Google Cloud Search - the first step in Google’s journey to replace the Google Search Appliance. You can read the official announcement here: https://blog.google/products/g-suite/introducing-google-cloud-search-g-suite/

SADA Systems has activated Google Cloud Search on our domain, so I would like to show you some details of what is included in the first release of Google Cloud Search. I have run a series of tests to show you how it works on our own content.

Artificial Intelligence Could Replace Traditional Search

Chad Johnson June 24, 2016

Go grab a stack of papers from the “TO-DO” pile on your desk – it’s OK, I know you have one – mine is full of utility statements to be filed away, drawing from my kids, and coupons for things I will probably never buy. Now find the cable TV bill from April 2016.

Not very difficult, huh? We (humans) are remarkably good at classifying information. We can very quickly tell the difference between a cable TV bill and a water bill. We can quickly decide if something is “What we are looking for” or “Not what we are looking for”. We don’t even break a sweat while finding a single, precise document out of hundreds of random papers. We have learned how to do this intuitively – all the way back to sorting leaves and sticks and acorns into piles or selecting the “best” slice of cake.

Artificial Intelligence is very good at similar types of problems – deciding if something is or is not what the program has been trained to recognize, or sorting data into arbitrary piles based on distinguishing characteristics. In machine learning, this is called either classification (supervised learning) or unsupervised learning. These techniques are some of the fundamental building blocks of intelligence, and they are being exploited by computer scientists to solve many real-world problems, from recognizing objects in pictures to recommending books you might like based on your reading habits.

Are You Tired of Hearing About Artificial Intelligence Yet?

Chad Johnson June 23, 2016

Are you tired of hearing about artificial intelligence yet? Well, I have some bad news. It’s only going to become the most important thing in our lives. Every once in a while you read an article that completely changes and overwhelms the way you think about something. Today, for me, it was The Artificial Intelligence Revolution: Part 1 and Part 2.

These are very long articles, so I have taken the liberty of extracting the best parts for a quick skim. If these excerpts get your gears moving, please, read the full articles.

Here is the best of Part 1:

First, the article describes the concept of accelerating progress, and why it is hard for us to accurately predict how much things will advance in the future:

Three Disadvantages of Cloud-Based Search Engines

Chad Johnson May 9, 2016

Last week I wrote about three advantages of ououd-based search engines. While the future looks bright for these tools, there are still potential downsides. This week I would like to discuss three disadvantages of cloud-based search engines, as well as potential mitigations. I am not saying that you should not consider cloud-based search engines because of these reasons. I am simply pointing out some reasonable concerns in the hope that you can make the most educated decision for your company.

1. Lack of Control

Not having to worry about the physical architecture of a search engine means that certain aspects of the implementation will be out of your control. If a cloud-based search engine is not performing fast enough, you cannot simply add another server instance or increase the RAM. If your users are geographically distributed, you cannot simply place a replica server in another location. Cloud services should perform well and they should offer geographic dispersion, but if they don’t, you are unlikely to be able to do anything about it yourself. You are at the mercy of the cloud for those types of configuration changes or improvements.

Three Advantage of Cloud-Based Search Engines

Chad Johnson May 2, 2016

Traditionally, enterprise search engines have been firmly planted on the ground. Now, along with almost everything else in enterprise IT, search engines are quickly moving into the sky. But why? What factors are prompting the growth of cloud-based search engines? Given the immense IT security (and mindset) challenges associated with indexing sensitive content in the cloud, why are vendors continuing to march full-speed in this direction?

1. Follow the Content

Almost all of my enterprise search clients are indexing at least one cloud-based content repository, such as Box, Salesforce, Google Drive, Office 365, etc. This historically involved writing connectors that download all of the information back to an on-premise search engine. As more content sources move to the cloud, it is becoming increasingly inefficient (and absurd?) to download all of the content back to the ground so it can be indexed.

Introducing the Application Indexing Protocol (AIP)

Chad Johnson April 22, 2016

I am excited to introduce the Application Indexing Protocol! AIP, for short, allows any application to proactively make itself indexable by a variety of third-party enterprise search engines. Unlike traditional connectors (created by search engine vendors or systems integration companies), application developers can now choose exactly what they want indexed and the best way to do it. Applications can choose the most appropriate indexing mechanism, either push (event-driven) or pull (polling), as both are supported by the Application Indexing Protocol. And because AIP is an open standard, applications only need to develop a single, canonical implementation in order to get their content indexed by numerous enterprise search engines. Gone are the days of every search engine vendor having to write its own version of every connector to every application. Now, the work can be done once, by the people who know the application the best.

[Chad wakes up from his dream…]

Drat! The Application Indexing Protocol does not actually exist [yet]. But I want it to so badly. I am tired of writing connectors for every new application, and for every new search engine. I am tired of trying to figure out how to efficiently and completely extract content and security trimming information from both on-premise and cloud-based applications, that may or may not provide appropriate APIs for this purpose.

Can Search Get Too Personal?

Chad Johnson April 19, 2016

The following article caught my attention this afternoon:

The UX Of Ethics: Should Google Tell You If You Have Cancer?

“If I’m on a park bench, and I’m next to someone, and I hear them talking about symptoms of cancer, am I obligated to turn around and tell them they might have cancer?”

… artificial intelligence products are rapidly approaching the same diagnostic power. Google Search can already predict coming flu trends with some level of success. It’s not hard to imagine a system that can track my searches over a year—an ache, a cough, a rash—recognizing a cascade of symptoms that point to a disease with surprising accuracy.

source: http://www.fastcodesign.com/3058943/the-ux-of-ethics-should-google-tell-you-if-you-have-cancer

I have not often considered the ethical implications of enterprise search. The article above reminded me that the information people type into those small search boxes can be extremely sensitive and revealing. Do you recall the 2006 release of anonymized search queries by AOL? Some very simple detective work made it possible to identify real people and real situations, even with usernames removed. For example, a vanity search for someone’s own name followed later in the day by a search for a rare medical condition or a financial problem. And it got more personal and embarrassing from there.

(Re)defining Enterprise Search

Chad Johnson April 15, 2016

Enterprise Search is changing. It used to be defined simply as finding documents in a large corporate repository or looking up specific information from some sea of data. But Enterprise Search is transitioning to something more broad and powerful. Enterprise Search now means reducing friction in your business operations. It means finding information you didn't know you were looking for. It works hands in hand with digital marketing to promote and sell goods and services. It provides unparalleled insights into what people are doing or looking for on a website or in an application.

I think it is time for an updated definition of Enterprise Search, and I could use your help to create one. If you have 30 seconds to spare, provide your own definition of Enterprise Search below (all data is collected anonymously). I am also curious if the definition is influenced by your job description or industry, so provide those if you like. Next week I will analyze the responses and report back on our modern definition of Enterprise Search.

My Initial Impressions of Google’s Cloud Vision API

Chad Johnson April 6, 2016

When Google released the Cloud Vision API last month I started to think of the interesting ways that artificially intelligent image recognition could be used. One idea that jumped to mind was enhancing the ability to search for marketing images and stock photography in a digital asset management system.

As much as companies encourage individuals to tag and annotate images accurately, invariably the metadata is poor or missing altogether. What if we could use artificial intelligence to add metadata to images automatically at the time they are uploaded to the system? Could this new metadata make it easier or faster to find images?

Google’s Cloud Vision API and IBM Watson’s AlchemyVision API both provide this service. And Adobe is incorporating this features natively into their AEM DAM, calling it Smart Tags. I have tested the Google and IBM services and initially found that the Google Cloud Vision API returned more metadata for each image, and was slightly more precise with the labels it suggested. I will continue to test these services as they advance, but for this post, I am going to focus on giving my impressions of the Google Cloud Vision API.

Google’s DeepMind Artificial Intelligence Wins!

Chad Johnson March 9, 2016

For the first time, a computer has beaten a world champion in the game of Go. Google’s DeepMind program defeated grandmaster Lee Sedol after a three and a half hour battle. This accomplishment was once considered impossible, given the complexity of the game and the sheer number of permutations possible in the game.

What I find most interesting is the change in approach from IBM’s Deep Blue computer that beat chess champion Gary Kasparov in 1997. Deep Blue could analyze billions of potential future scenarios and pick the most logical and advantageous move at each step of the game. It essentially used brute force and statistics to pick the best path and stay ahead of Kasparov.

Google Releases Cloud Vision API. I Eat Crow.

Chad Johnson March 3, 2016

Last year Google introduced groundbreaking image recognition technology as part of their release of Google Photos. Deep neural networks could identify objects, people and places inside of photographs with astonishing accuracy, completely autonomously.

As fascinating as this technology is, I was certain that Google would keep it locked away inside their own products like Google Photos and Google Drive for the foreseeable future. I did not anticipate it being sold standalone as an API.

I was wrong. Two weeks ago, Google announced a beta version of their Cloud Vision API, allowing anyone to process images and identify things like everyday objects, faces (w/sentiment) and text or logos.

For example, processing the picture below with the Cloud Vision API that identifies common objects …

One Size Doesn’t Fit All with Enterprise Search

Chad Johnson March 2, 2016

Last week I was starting to design a new demonstration environment for our Search and Knowledge Discovery practice. It would simulate a search application for a high tech call center, searching across a customer support case repository. I was having difficultly coming up with a scenario that would be applicable to a wide audience and include all of the diverse features that I wanted to show off. I realized I was falling into a trap just like many of my own clients.

Ask most people what an enterprise search application looks like and they will probably describe something very traditional – a search box across the top with 10 search results down the middle of the page. The results will come from predictable sources such as the corporate intranet or knowledge management repositories. And the experience will be the same for all users, except for security trimming that might hide certain results based on document permissions.

This kind of enterprise search application can satisfy some employees some of the time, but it is far from perfect. It doesn’t take into account the diverse job functions within a company and the different search needs of each. It doesn’t learn from experience and get better over time. And it doesn’t promote the discovery of information by exposing the relationships between different content sources.

3 Predictions for the Future of Search and Knowledge Discovery

Chad Johnson February 17, 2016

Warning: The future of search is going to get very personal. Hold on to your tin foil hats.

Let’s begin with a brief history of the search and knowledge discovery field.

Search + Analytics

In 1997, Google launched the Google.com search engine. 4.3 seconds later, Google started looking at everyone’s search queries. Just kidding. Sort of. Search and analytics have been tied at the hip since the very beginning. Companies learned that what people are searching for is useful and valuable information. Analytics can be used to evaluate the performance and effectiveness of a search solution, and they can be used to improve it going forward. By looking at what people are searching for and how they are interacting with the search results, the experience can be continuously modified to yield the best results.

Google Search Appliance Sunsetting: Week One Reactions

Chad Johnson February 11, 2016

It has been exactly one week since Google announced that it would be sunsetting the Google Search Appliance over the next three years. And what a week it has been. I have had the opportunity to speak with media representatives at Fortune and CIODive, I have spoken with several current and prospective customers of the Google Search Appliance, I have spoken with numerous other search companies, and I have chatted informally with our consulting team and friends in the enterprise search ecosystem.

I would like to share some of my initial thoughts to the news, and the rollercoaster of reactions that I, and others, have had.

Did the news surprise me?

The most common question I have been asked is if the news surprised me. Was I surprised that Google would focus more energy on a cloud-based service than a physical yellow box? Not at all. It makes good sense for Google to develop a technology that does not require shipping GSAs around the world and installing them in data centers. I have been a partner with Google for 7 years and I have come to expect periodic hard-left turns from them. They usually turn out to change an industry in retrospect.

Can You Quantify a Cold?

Chad Johnson February 2, 2016

How do computers see the world? Can they tell if we are happy? Can they predict what we are going to have for dinner next Tuesday? Can they recognize when we are sick?

Well, my iPhone can. Sort of.

I was (finally) recovering from a nasty cold last week and I opened the Health app on my phone and noticed that the step-counter had fairly accurately captured my cold, from start to finish. My normally active body had flatlined for over three weeks, except for one day where I thought I was getting better and went snowshoeing for 8 miles in the Mount Evans Wilderness. Bad idea.

So why didn’t my phone send me a sympathy card? Well, that’s the problem with data. It’s just data. It takes skillful, and contextual, interpretation to make sense of the data and to draw useful conclusions. With the step data, a computer might notice a trend or anomaly, but it doesn’t understand that it was caused by working out more or being sick in bed. That is the art of data science and knowledge discovery. A graph is just a graph until someone interprets it.

What’s New in Search Solutions – People Search 2.0

Chad Johnson January 20, 2016

For many years, People Search with the Google Search Appliance was essentially a glorified phonebook lookup. Our customers could search for colleagues by name or title, and retrieve basic information like phone numbers or addresses. Even the advent of ‘Expert Search’ did not tangibly change the situation — it just made the phonebook-style lookup easier to implement.

The Problem: In 2015, several of my clients came to us demanding more from People Search. We realized that these companies did not have just a single source for information about people, and that by investing in additional data aggregation and pre-processing, we could deliver more sophisticated and useful people search results.

The Solution: Instead of indexing only first-order fields about people (name, title, location, etc.), we started adding data like skills, project experience, previous jobs, and recently authored content. This expanded the number of ways that a person can be found, and it also increased the number of ways that the results can be sorted, ranked or biased. You can now search for people with similar skills, or who worked together on past projects, or who recently collaborated on a document about a certain topic.

How Big is 1.5 Billion?

Chad Johnson January 14, 2016

Sadly, I did not win the Powerball Lottery (I admit it is hard to win when you don’t play) so I am still here writing blog posts. Oh well, I kind of like my job.

The frenzy did get me thinking about how big is a billion. A little back of the napkin math suggests that my Google Search Appliance (GSA) implementation team has indexed over a quarter of a billion documents and records for our customers. How much content is that?

If each document was an average of 5 pages long, that would amount to a stack of paper 98.6 miles tall, or roughly the beginning of Low Earth Orbit for satellites. If the documents were about 100KB in size, it would take 23 one terabyte hard drives to store them.

Surprisingly, it would only take thirteen G100 Google Search Appliances, or three G500’s, to index this many documents. So everybody, get out there and fill up your GSAs!

What is the Timeline for a Typical GSA Implementation?

Chad Johnson September 4, 2015

I’ll get this out of the way quickly — there is no such thing as a typical GSA implementation.

With that said, I have been involved in scoping and delivering around 250 GSA implementations, and some averages and trends have emerged. Most projects involve the same five core phases: Design and Requirements, Hardware and Security, Content Acquisition, User Interface, and Testing and Deployment. I will walk through how we estimate the duration of these phases, with the understanding that we always do a thorough scoping workshop with a customer to refine this timeline. Thanks to our large volume of projects, our estimates have gotten very accurate. While certain tasks might not always be completed in the order we predict, the overall effort is usually correct.