What is Artificial Intelligence?

Start with the word that means the least

"Artificial intelligence" was coined in 1956 for a summer workshop, and it has always described an ambition rather than a method. Any system that does something we would call intelligent if a person did it qualifies. A chess engine that searches millions of positions per second qualifies. So does a medical expert system from 1980 built entirely from hand-written IF-THEN rules, with no learning of any kind.

This matters because the term is now used almost exclusively to mean one particular technology from one particular year, and the substitution hides the actual question. When someone says a product "uses AI", they have told you the goal and nothing about the method.

Machine learning: where did the rule come from?

Machine learning narrows things usefully. A system is doing machine learning when the rule it applies was fitted to examples rather than written by a person.

The distinction is about provenance, not sophistication. Consider two spam filters. The first checks incoming mail against a list of banned words that an engineer maintains by hand. The second was shown a hundred thousand messages labelled spam or not-spam and worked out its own weighting over words. The second is machine learning; the first is not, however well it performs.

The consequence follows immediately and is the single most useful thing to understand about the field: a machine-learning system inherits the properties of its training data, including the ones nobody intended. If the labelled examples came from one country, one demographic or one decade, the fitted rule encodes that. There is no separate step where general knowledge gets added.

Deep learning: who chose the features?

Classical machine learning has a step that is easy to overlook. Before you can fit a rule, someone has to decide what the model gets to look at. For a spam filter that might be word frequencies, the presence of links, the sender's history. This is feature engineering, and until about 2012 it was where most of the skill and most of the labour in the field lived.

Deep learning removed that step. A deep network is built from many layers, and each layer transforms the output of the one below it. Trained end to end, the early layers converge on simple detectors — edges in an image, character patterns in text — and later layers combine those into progressively more abstract things. Nobody specifies this. The structure appears because it is the arrangement that best reduced the training error.

Two conditions had to be met before this worked, and both arrived at roughly the same time: enough labelled data to constrain millions of parameters, and enough parallel compute to train them in finite time. The core ideas are considerably older than their success.

Generative models: constructing rather than choosing

A classifier picks from a fixed set of answers. A generative model constructs its output one piece at a time, each piece conditioned on everything before it. That is the difference between "is this email spam?" and "write a reply to this email".

The architecture that made this practical at scale is the transformer, introduced in 2017. Its central mechanism, attention, lets every position in a sequence draw on every earlier position in a single step — and, critically, lets all positions be computed in parallel. That parallelism is not a detail. It is the reason training at the current scale is affordable at all.

It is worth being precise about what such a model is optimising. It is trained to predict likely continuations of text. Nothing in that objective distinguishes true from likely, which is exactly why hallucination is normal behaviour rather than a bug: a plausible-sounding citation is a likely continuation whether or not the paper exists.

Where the layers actually sit in practice

Because public attention landed on the innermost ring, it is easy to assume the outer ones were superseded. They were not, and the gap between what is discussed and what is deployed is wide.

• Rule systems still run payroll, tax, and most of what a bank does. They are auditable and they do not drift.
• Classical machine learning — gradient-boosted trees, logistic regression — still wins on tabular data, which is most of the data an organisation actually has. It also trains in minutes on a laptop.
• Deep learning owns perception: images, audio, and any signal where the useful features are not obvious in advance.
• Generative models own open-ended language and code, and are moving into scientific domains where the output is a structure rather than a sentence.

Choosing the innermost ring for a problem the outer ones solve is a common and expensive mistake. A gradient-boosted tree that trains in ninety seconds and can be explained to a regulator is frequently the correct answer.

What to learn, and in what order

The ladder has a natural sequence, and skipping rungs tends to produce someone who can call an API but cannot tell when its output is wrong.

Start with linear algebra and probability — not as prerequisites to be endured, but because every subsequent idea is expressed in them. Then fit a linear model by hand, on real data, and sit with the fact that most of the work is preparing the data rather than choosing the model. Then classification metrics, and specifically why accuracy is misleading on imbalanced classes. Only then a neural network, and only then a transformer.

At each rung the question to keep asking is the one this article started with: where did the rule come from, and what was in the data that produced it?