Artificial Intelligence Could Replace Traditional Search

Chad Johnson June 24, 2016

Go grab a stack of papers from the “TO-DO” pile on your desk – it’s OK, I know you have one – mine is full of utility statements to be filed away, drawing from my kids, and coupons for things I will probably never buy. Now find the cable TV bill from April 2016.

Not very difficult, huh? We (humans) are remarkably good at classifying information. We can very quickly tell the difference between a cable TV bill and a water bill. We can quickly decide if something is “What we are looking for” or “Not what we are looking for”. We don’t even break a sweat while finding a single, precise document out of hundreds of random papers. We have learned how to do this intuitively – all the way back to sorting leaves and sticks and acorns into piles or selecting the “best” slice of cake.

Artificial Intelligence is very good at similar types of problems – deciding if something is or is not what the program has been trained to recognize, or sorting data into arbitrary piles based on distinguishing characteristics. In machine learning, this is called either classification (supervised learning) or unsupervised learning. These techniques are some of the fundamental building blocks of intelligence, and they are being exploited by computer scientists to solve many real-world problems, from recognizing objects in pictures to recommending books you might like based on your reading habits.

Are You Tired of Hearing About Artificial Intelligence Yet?

Chad Johnson June 23, 2016

Are you tired of hearing about artificial intelligence yet? Well, I have some bad news. It’s only going to become the most important thing in our lives. Every once in a while you read an article that completely changes and overwhelms the way you think about something. Today, for me, it was The Artificial Intelligence Revolution: Part 1 and Part 2.

These are very long articles, so I have taken the liberty of extracting the best parts for a quick skim. If these excerpts get your gears moving, please, read the full articles.

Here is the best of Part 1:

First, the article describes the concept of accelerating progress, and why it is hard for us to accurately predict how much things will advance in the future:

Can Search Get Too Personal?

Chad Johnson April 19, 2016

The following article caught my attention this afternoon:

The UX Of Ethics: Should Google Tell You If You Have Cancer?

“If I’m on a park bench, and I’m next to someone, and I hear them talking about symptoms of cancer, am I obligated to turn around and tell them they might have cancer?”

… artificial intelligence products are rapidly approaching the same diagnostic power. Google Search can already predict coming flu trends with some level of success. It’s not hard to imagine a system that can track my searches over a year—an ache, a cough, a rash—recognizing a cascade of symptoms that point to a disease with surprising accuracy.

source: http://www.fastcodesign.com/3058943/the-ux-of-ethics-should-google-tell-you-if-you-have-cancer

I have not often considered the ethical implications of enterprise search. The article above reminded me that the information people type into those small search boxes can be extremely sensitive and revealing. Do you recall the 2006 release of anonymized search queries by AOL? Some very simple detective work made it possible to identify real people and real situations, even with usernames removed. For example, a vanity search for someone’s own name followed later in the day by a search for a rare medical condition or a financial problem. And it got more personal and embarrassing from there.

My Initial Impressions of Google’s Cloud Vision API

Chad Johnson April 6, 2016

When Google released the Cloud Vision API last month I started to think of the interesting ways that artificially intelligent image recognition could be used. One idea that jumped to mind was enhancing the ability to search for marketing images and stock photography in a digital asset management system.

As much as companies encourage individuals to tag and annotate images accurately, invariably the metadata is poor or missing altogether. What if we could use artificial intelligence to add metadata to images automatically at the time they are uploaded to the system? Could this new metadata make it easier or faster to find images?

Google’s Cloud Vision API and IBM Watson’s AlchemyVision API both provide this service. And Adobe is incorporating this features natively into their AEM DAM, calling it Smart Tags. I have tested the Google and IBM services and initially found that the Google Cloud Vision API returned more metadata for each image, and was slightly more precise with the labels it suggested. I will continue to test these services as they advance, but for this post, I am going to focus on giving my impressions of the Google Cloud Vision API.

Google’s DeepMind Artificial Intelligence Wins!

Chad Johnson March 9, 2016

For the first time, a computer has beaten a world champion in the game of Go. Google’s DeepMind program defeated grandmaster Lee Sedol after a three and a half hour battle. This accomplishment was once considered impossible, given the complexity of the game and the sheer number of permutations possible in the game.

What I find most interesting is the change in approach from IBM’s Deep Blue computer that beat chess champion Gary Kasparov in 1997. Deep Blue could analyze billions of potential future scenarios and pick the most logical and advantageous move at each step of the game. It essentially used brute force and statistics to pick the best path and stay ahead of Kasparov.

Google Releases Cloud Vision API. I Eat Crow.

Chad Johnson March 3, 2016

Last year Google introduced groundbreaking image recognition technology as part of their release of Google Photos. Deep neural networks could identify objects, people and places inside of photographs with astonishing accuracy, completely autonomously.

As fascinating as this technology is, I was certain that Google would keep it locked away inside their own products like Google Photos and Google Drive for the foreseeable future. I did not anticipate it being sold standalone as an API.

I was wrong. Two weeks ago, Google announced a beta version of their Cloud Vision API, allowing anyone to process images and identify things like everyday objects, faces (w/sentiment) and text or logos.

For example, processing the picture below with the Cloud Vision API that identifies common objects …

3 Predictions for the Future of Search and Knowledge Discovery

Chad Johnson February 17, 2016

Warning: The future of search is going to get very personal. Hold on to your tin foil hats.

Let’s begin with a brief history of the search and knowledge discovery field.

Search + Analytics

In 1997, Google launched the Google.com search engine. 4.3 seconds later, Google started looking at everyone’s search queries. Just kidding. Sort of. Search and analytics have been tied at the hip since the very beginning. Companies learned that what people are searching for is useful and valuable information. Analytics can be used to evaluate the performance and effectiveness of a search solution, and they can be used to improve it going forward. By looking at what people are searching for and how they are interacting with the search results, the experience can be continuously modified to yield the best results.

Machine Learning Pt. 3: We Need to Build a Computer/Brain Thingy

Chad Johnson August 19, 2015

In previous posts I have discussed the algorithms and applications of machine learning. One of the most sophisticated forms of machine learning is building a neural network that mimics the flesh-and-blood brain – with software- or hardware-based versions of neurons and synapses. But silicon operates differently than the stuff in our brains. In the past, researchers have suspected an entirely new type of computer chip would be needed – something radically different from our current silicon- and transistor-based CPUs and GPUs.

But, leave it to creative engineers to find a way. IBM has invented a new type of computer chip, dubbed TrueNorth, that could get us closer to the sci-fi dream of a silicon brain. Using traditional semiconductor manufacturing techniques (instead of trying to mimic the actual physics and chemistry of the brain), IBM has gotten as close as they can to modeling the way a real brain works.

Machine Learning Pt. 2: Applications in Enterprise Search

Chad Johnson August 5, 2015

In last week’s article, What is Machine Learning?, I gave an overview of Machine Learning and provided a link to a very informative visual guide that describes how the technology works. I also hinted at some useful applications within Enterprise Search. I would like to cover a few of those that I have seen in recent months.

Query Analysis with Clustering

There are often many ways to search for the same concept or idea. When we are reviewing search logs and trying to draw conclusions from what people are searching for, slight variations from query to query can make it difficult to do a comprehensive analysis. For example, we recently worked with a major airline and one of their top queries was “bereavement fares”, i.e. people looking for discounted fares when attending a sudden funeral. We knew that this particular query was popular, and that it needed some tuning to best serve the customers, but further analysis reveled other similar but different queries suffering from the same problem. The other forms of the query were less common, and if treated individually, they would have flown under the radar (no pun intended) in our analysis.

What is Machine Learning?

Chad Johnson July 30, 2015

What is Machine Learning? Well, I have a general idea of how to explain Machine Learning – it is the process of training a computer using historical data and decisions, such that the computer can then make its own decisions in the future using all new data that it has never seen before. For example, machine learning is used to train computers to read handwriting by studying countless examples of known writing samples. It is also used to study market data and make predictions about when to buy or sell.

This has given me a pretty good “black-box” understanding of machine learning algorithms. I understand that you feed in a bunch of data along with historical decisions and outcomes. Then a miracle occurs. And now the machine can make its own decisions in the future. If I feed the black box a million different handwritten letter A’s, and I tell it that all of them are the letter A, it should now be able to see a letter that it has never seen before and tell me with some confidence whether or not it is a letter A.

Spam is dead! Well, at least for Google Mail users

Chad Johnson July 20, 2015

Last week, Google announced some astounding statistics about the success rate of their spam filtering technology in Google Mail. Google says that less than 0.1% of email in the average user’s Gmail inbox is spam, and the rate for legitimate email ending up in the spam folder is even lower at less than %0.05.

Despite these superb results, Google is continuing to innovate and find new ways of detecting and blocking spam. This year, Google has announced several innovations across their product line that utilize their machine learning technology, affectionately referred to as Google Brain. Using their artificial neural network software, Google Photos can identify objects in your photographs, like dogs or cats or bridges or birthday cakes. Google Maps can automatically detect new businesses or speed limit signs in the imagery collected by their entire fleet of Street View cars. Google Earth Engine can detect new refuge camps or deforestation boundaries by scanning for visual patterns in petabytes of satellite images.