AI Guide · Updated 14 November 2026
History
The machine learning era, 1993-2011
By drayz.ai · Updated · 9 min read

From the early 1990s to 2011, artificial intelligence quietly reinvented itself. Instead of trying to write down human knowledge as rules, researchers let computers learn patterns from data using statistics and probability. The internet supplied the data, faster chips supplied the power, and AI crept into search engines, spam filters, shopping sites and phones, often without being called AI at all. This chapter covers the machine learning era, from Deep Blue's chess victory to IBM Watson and Siri.
From rules to statistics
The expert systems of the 1980s tried to capture what experts knew as hand-written rules. That proved slow, expensive and brittle. The alternative was to let the machine work out the rules for itself by finding patterns in examples. This approach, machine learning, became the dominant way of doing AI during the 1990s and 2000s.
A famous example of the shift came from speech recognition. At IBM in the 1970s and 1980s, Frederick Jelinek's team replaced linguistic rules with statistical models trained on large amounts of recorded speech and text, and accuracy improved dramatically. Jelinek is often quoted as joking that every time he fired a linguist, the performance of the speech recogniser went up. Whether or not he said exactly that, the point stuck: with enough data, statistics often beat hand-crafted expertise.
The key tools of the era
Machine learning in this period was built on a set of mathematical tools that are still widely used:
- Bayesian networks. In 1988 Judea Pearl published Probabilistic Reasoning in Intelligent Systems, showing how to represent uncertain knowledge as networks of probabilities and reason with it efficiently. Bayesian methods were used in medical diagnosis, fault-finding and spam filters. Pearl received the Turing Award in 2011.
- Hidden Markov models powered speech recognition and early handwriting and gene-sequence analysis.
- Decision trees, such as Ross Quinlan's ID3 and C4.5, learned flowchart-like rules from data that people could read.
- Support vector machines (SVMs), developed by Vladimir Vapnik and colleagues (the modern soft-margin form by Corinna Cortes and Vapnik in 1995), found the best boundary between classes of data and were often the most accurate method available for a decade.
- Ensemble methods combined many simple models into a strong one. Boosting (AdaBoost, 1995, by Yoav Freund and Robert Schapire) and random forests (Leo Breiman, 2001) were especially successful and are still workhorses of business analytics.
- Reinforcement learning was formalised in Richard Sutton and Andrew Barto's 1998 textbook. Sutton and Barto received the Turing Award in 2025.
- Kernel methods, clustering, and dimensionality reduction helped find structure in unlabelled data.
These methods had something in common: they needed people to design good features, the measurements fed into the model. A spam filter might use features such as "contains the word 'free'" or "sent from an unknown address". Good feature engineering often mattered more than the choice of algorithm. Deep learning would later remove much of this manual step.
Deep Blue beats Kasparov, 1997
The most famous AI moment of the 1990s was a chess match. IBM's Deep Blue had lost to world champion Garry Kasparov 4-2 in Philadelphia in 1996. In a rematch in New York in May 1997, an upgraded Deep Blue won 3½-2½. It was the first time a computer had beaten a reigning world champion in a match under standard tournament conditions.
Deep Blue was not a learning system in the modern sense. It used custom chips to search around 200 million chess positions per second, combined with an evaluation function tuned with input from grandmasters and an opening book. It showed what raw computing power and clever search could do, but it could do nothing except play chess. Kasparov, who suspected human intervention at the time, later wrote that he had been beaten by a machine that was "intelligent" only in the way a programmable alarm clock is. Even so, the match captured the public imagination and fulfilled, almost forty years late, the 1958 prediction that a computer would become world chess champion.
The web changes everything
The growth of the internet from the mid-1990s transformed AI in two ways: it created huge amounts of data to learn from, and it created businesses that needed AI to function.
Search
Search engines needed to work out which of billions of web pages best answered a query. Google, founded by Larry Page and Sergey Brin in 1998, used the PageRank algorithm, which ranked pages by the links pointing to them, alongside many other signals that were increasingly learned from user behaviour. Search became one of the largest applications of machine learning in the world.
Spam filtering
By the early 2000s email spam was a flood. In 2002 the programmer Paul Graham's essay "A Plan for Spam" popularised the use of naive Bayes classifiers, which learned the probability that an email was spam from the words it contained. Statistical spam filters became standard and remain a classic example of everyday machine learning.
Recommendations
Amazon introduced item-to-item collaborative filtering ("customers who bought this item also bought...") in the late 1990s. In 2006 Netflix launched the Netflix Prize, offering $1 million to anyone who could improve its film recommendation accuracy by 10%. Thousands of teams competed. The prize was won in 2009 by a team called BellKor's Pragmatic Chaos, using a blend of many models. The competition popularised techniques such as matrix factorisation and ensembles, and showed the power of open data challenges, an idea that would later drive the deep learning revolution.
Online advertising and fraud
Ad networks used machine learning to predict which adverts people would click. Banks and card companies used it to spot fraudulent transactions in real time. These systems handled billions of decisions a day, largely invisibly.
Driverless cars: the DARPA challenges
In 2004 DARPA held the Grand Challenge, offering $1 million to any self-driving vehicle that could cross about 142 miles of the Mojave Desert. None finished; the best managed about seven miles. In 2005, five vehicles completed a 132-mile course. The winner was Stanley, built by Stanford University's team led by Sebastian Thrun, which used machine learning to interpret laser and camera data and tell drivable road from rough terrain. In the 2007 Urban Challenge, vehicles had to navigate a mock town with other traffic; Carnegie Mellon's "Boss" won.
Many of the engineers involved went on to lead self-driving projects at Google (later Waymo), Uber and car makers. Waymo began offering fully driverless paid rides to the public in Phoenix in 2020, and by the mid-2020s robotaxis were operating in several US cities. In the UK, the Automated Vehicles Act 2024 created a legal framework for self-driving vehicles, with pilots of self-driving taxi services planned from 2026.
Robots in homes, factories and space
- Roomba (2002), from iRobot, brought simple behaviour-based robotics into millions of homes.
- NASA's Mars rovers Spirit and Opportunity (landed 2004) used autonomous navigation to drive between commands from Earth. Opportunity was designed to last 90 Martian days and kept working for over 14 years.
- Honda's ASIMO humanoid (2000) could walk, climb stairs and run, a showcase for robotics that would inspire later humanoid programmes.
- Kiva Systems robots moved shelves around warehouses; Amazon bought the company in 2012 for $775 million.
Data, chips and the seeds of deep learning
Three developments in the 2000s set the stage for the next revolution.
1. Big data
Smartphones, social media, digital cameras and online services generated data on a scale never seen before. In 2009 Google researchers Alon Halevy, Peter Norvig and Fernando Pereira published "The Unreasonable Effectiveness of Data", arguing that simple models trained on huge datasets often beat sophisticated models trained on small ones.
2. ImageNet
At Princeton and then Stanford, Fei-Fei Li led the creation of ImageNet, a database of millions of photographs labelled with thousands of object categories, using the crowdsourcing platform Amazon Mechanical Turk to label them. ImageNet was presented in 2009. From 2010, the annual ImageNet Large Scale Visual Recognition Challenge invited researchers to compete at classifying 1.2 million images into 1,000 categories. It became the proving ground on which deep learning would make its breakthrough.
3. GPUs
Graphics processing units (GPUs) were designed to render video game graphics, which requires doing huge numbers of simple calculations at the same time. In 2007 Nvidia released CUDA, which made it much easier to use GPUs for general computing. Researchers including Andrew Ng's group at Stanford showed in 2009 that GPUs could train neural networks many times faster than ordinary processors. Nvidia, then known mainly to gamers, would become one of the most valuable companies in the world.
Neural networks persist
Through the 2000s a small community kept working on neural networks, notably Geoffrey Hinton (Toronto), Yann LeCun (New York University) and Yoshua Bengio (Montreal), with support from the Canadian Institute for Advanced Research. In 2006 Hinton and colleagues showed that "deep belief networks" could be trained effectively one layer at a time, reviving interest in deep networks, a term that began to catch on. In 1997 Sepp Hochreiter and Jürgen Schmidhuber had invented long short-term memory (LSTM) networks, which could learn from sequences such as speech and handwriting; by 2009 LSTMs were winning handwriting recognition competitions. Hinton, LeCun and Bengio shared the 2018 Turing Award for their work on deep learning.
Watson and Siri, 2011
Two events in 2011 brought AI back into the headlines.
In February 2011 IBM's Watson beat the two greatest champions of the American quiz show Jeopardy!, Ken Jennings and Brad Rutter. Watson combined hundreds of language-processing and machine learning techniques to interpret cryptic clues and search millions of documents, all within a few seconds. It was a remarkable achievement. IBM then tried to turn Watson into a business, especially in healthcare, with mixed results: several high-profile hospital projects were scaled back, and IBM sold off much of Watson Health in 2022. The gap between a brilliant demonstration and a reliable product was a lesson the AI winters had already taught.
In October 2011 Apple launched Siri on the iPhone 4S. Siri had grown out of CALO, a DARPA-funded research project at SRI International, the same institute that had built Shakey forty years earlier. Siri was limited, but it put a conversational AI assistant into millions of pockets. Google Now (2012), Microsoft's Cortana (2014) and Amazon's Alexa (2014) followed.
Why this era matters
The machine learning era transformed AI from an academic field with a troubled reputation into an essential part of the world's biggest technology companies. It established:
- data-driven AI as the dominant approach;
- the importance of benchmarks and open competitions such as the Netflix Prize and ImageNet;
- the practical value of AI in search, advertising, recommendations, fraud detection, speech and translation;
- the infrastructure, in data and in GPUs, that made deep learning possible.
By 2011 everything was in place for a breakthrough. It came the following year.
Key takeaways
- In the 1990s and 2000s AI shifted from hand-written rules to machine learning from data.
- Key tools included Bayesian networks, SVMs, decision trees, random forests and boosting.
- Deep Blue beat Garry Kasparov at chess in 1997 using brute-force search.
- The web created vast data and businesses built on machine learning: search, spam filters, recommendations and ads.
- Stanford's Stanley won the 2005 DARPA Grand Challenge, launching the self-driving industry.
- ImageNet (2009), GPUs and persistent neural network research set the stage for deep learning.
- IBM Watson won Jeopardy! and Apple launched Siri in 2011.
Previous chapter: AI winters and expert systems · Next chapter: The deep learning revolution, 2012-2017