Through Time

AI: from symbols to agents

The whole history of artificial intelligence: rules, chatbots, neural networks, GPT, the labs, AGI and ASI, and what follows tomorrow.

A course for people who use AI every day and want to understand what is inside it. You will walk through seventy years of ideas, dead ends and breakthroughs, learn how a language model works and why it makes mistakes, and be able to judge bold claims for yourself. No programming needed.

So far there is only this table of contents. Lessons appear one at a time, and we promise no dates. Feed of new lessons (Atom)

41 lessons, 9 parts

Part 1 Thinking as rules

The first big idea: if a mind works with symbols, it can be written down as a program.

  1. Can a machine think

    Turing’s 1950 paper, the imitation game, and why he replaced a question about thinking with a question about behavior.

    coming soon

  2. Dartmouth, 1956: a science gets its name

    The proposal by McCarthy, Minsky, Rochester and Shannon, the summer workshop, and the promise of real progress in a single summer.

    coming soon

  3. Logic Theorist and the symbol system hypothesis

    How the program by Newell, Simon and Shaw proved theorems, and how symbolic AI’s central bet grew out of it.

    coming soon

  4. SHRDLU and the blocks world

    Winograd’s program of about 1970 understood orders about blocks, which showed why that success does not scale.

    coming soon

  5. Expert systems: knowledge as a product

    MYCIN in medicine, XCON at DEC and the knowledge engineer: the first time AI made money.

    coming soon

  6. Two AI winters

    The Lighthill report of 1973, the funding cuts, the collapse of the Lisp machine market, and what inflated promises end in.

    coming soon

Part 2 Conversation partners

Machines that understood nothing, and the people who trusted them anyway.

  1. ELIZA: a therapist made of patterns

    Weizenbaum’s program of 1966, the DOCTOR script, and how people’s readiness to confide in it frightened its author.

    coming soon

  2. The ELIZA effect today

    Why we attribute understanding to whatever answers smoothly, and how that habit works on modern models.

    coming soon

  3. A.L.I.C.E.: tens of thousands of rules

    Richard Wallace’s chatbot of 1995, the AIML language, and three Loebner Prizes won with no learning from data at all.

    coming soon

  4. The Turing test as a contest

    What the Loebner Prize actually measured, and why fooling a judge is not the same as thinking.

    coming soon

  5. Siri, Alexa and Yandex’s Alice

    The voice assistants of 2011, 2014 and 2017: speech recognition on neural networks, and scripts written by people.

    coming soon

Part 3 What counts as intelligence

An argument about words on which research goals and today’s headlines both depend.

  1. Weak and strong AI

    The distinction from Searle’s 1980 paper, his Chinese room, and why the argument over it is not finished.

    coming soon

  2. Cognitive architectures

    Soar and ACT-R: the attempt to build a model of a whole mind, with memory, attention and learning, rather than solve separate tasks.

    coming soon

  3. Common sense by hand: the Cyc project

    From 1984 Douglas Lenat and his team wrote knowledge of the world down as rules, which shows how much of it there is.

    coming soon

Part 4 Generations of neural networks

The second big idea, which ran beside the first for seventy years: do not write the rules, learn from examples.

  1. The artificial neuron and the perceptron

    The McCulloch and Pitts model of 1943, Rosenblatt’s perceptron of 1958, and the first loud promises of machine learning.

    coming soon

  2. The book that stopped perceptrons

    In 1969 Minsky and Papert showed the limits of a single-layer network, and the XOR problem became the symbol of that dispute.

    coming soon

  3. Backpropagation

    The 1986 paper by Rumelhart, Hinton and Williams, and the simple idea of how a network finds out which weight is to blame for an error.

    coming soon

  4. Networks for images and for sequences

    LeCun’s convolutional network for zip codes and the 1997 LSTM by Hochreiter and Schmidhuber: two architectures waiting for their data.

    coming soon

  5. The quiet victory of statistics

    Spam filters, search and the machine translation of the 1990s: when data began to beat rules without any fuss.

    coming soon

Part 5 Deep learning

Big data, graphics cards and a few old ideas came together, and within five years everything had changed.

  1. ImageNet and AlexNet

    Fei-Fei Li’s dataset of 2009, the 2012 network by Krizhevsky, Sutskever and Hinton, and the part graphics cards played in that turn.

    coming soon

  2. Meaning as coordinates

    Word2vec of 2013 and the idea all language models rest on: a word can be represented as a point in space.

    coming soon

  3. Deep Blue and AlphaGo: two victories

    Chess in 1997 was won by search and Go in 2016 by learning, and the difference between them matters more than the wins.

    coming soon

  4. Machines that draw

    Goodfellow’s GAN of 2014, diffusion models and Stable Diffusion of 2022: how a network learns to create and not only to recognize.

    coming soon

  5. AlphaFold: AI in science

    How the problem of protein shape got a practical solution in 2020 and led to a Nobel Prize in 2024.

    coming soon

Part 6 Attention and GPT

One paper from 2017 and what grew out of it in five years.

  1. The idea of attention

    Machine translation, the 2014 work by Bahdanau, Cho and Bengio, and the simple idea that a network can decide for itself where to look.

    coming soon

  2. Attention Is All You Need

    The 2017 paper by Vaswani et al.: the transformer architecture, in plain words and drawn step by step.

    coming soon

  3. GPT, GPT-2, GPT-3

    OpenAI’s models of 2018, 2019 and 2020: what changed from version to version, and where the arguments about publishing began.

    coming soon

  4. How a language model works

    Tokens, probabilities, context and temperature: what happens between your question and the answer, and why a model invents plausible things.

    coming soon

  5. Scaling laws and their limits

    The 2020 work by Kaplan et al., growth through data and compute, and the honest question of how long that will keep working.

    coming soon

  6. ChatGPT: training on people

    InstructGPT, reinforcement learning from human feedback, and November 30, 2022, the day a model began talking to everyone.

    coming soon

  7. Models that reason

    Chains of reasoning, the models of 2024 and 2025 such as o1 and DeepSeek-R1, and the argument over what exactly they are doing.

    coming soon

Part 7 Agents

A model gets tools and memory, and turns from a conversation partner into a doer.

  1. Tools and the agent loop

    How a model calls tools, reads the result and decides what to do next, and the part the MCP protocol plays in that.

    coming soon

  2. Agent memory

    Why a model remembers nothing between conversations, how memory is built on top of it, and what NoeticEcho’s open library Reminisce shows.

    coming soon

  3. How agents fail

    Swapping the task, prompt injection and a confident report of work not done: typical failures that follow from how the model is built.

    coming soon

Part 8 Who makes it

Companies, chips and money: without them the history of ideas does not explain why things turned out this way.

  1. The AI labs

    DeepMind (2010), OpenAI (2015), Anthropic (2021), xAI, Mistral and DeepSeek (2023), along with Meta, Yandex and Sber: where each came from and how it is set up.

    coming soon

  2. Open and closed models

    What open weights mean, how that differs from open source, and why the argument became a practical one after LLaMA in 2023.

    coming soon

  3. Chips, data centers and energy

    Where compute comes from, why a few companies in the supply chain matter so much, and how much electricity it takes.

    coming soon

Part 9 AGI, ASI and tomorrow

The loudest words in the field: what they mean, what is known about them, and what nobody knows.

  1. AGI: general intelligence

    Where the term came from, why it has no single definition, and how to treat claims that it is about to arrive.

    coming soon

  2. ASI: superintelligence

    From Good’s “ultraintelligent machine” of 1965 to Bostrom’s book of 2014: the logic of the argument and its weak points.

    coming soon

  3. Alignment: how to set a goal

    Why a machine may do what was said rather than what was meant, and what the builders of these systems themselves warn about.

    coming soon

  4. A look into the near future

    What may change in the next few years, what is only promised, and how to check such forecasts yourself when they appear in the news.

    coming soon