Acting Rationally, or Thinking Like a Human
Chapter Two
Syllabus topic Module 1, "Rational agents vs human thinking"
Pages 6 to 9 of 591
In one line
A rational agent does the thing that is expected to work out best, judged by a stated measure; a machine that thinks like a human reproduces the steps a person's mind actually takes, whether or not those steps work out best.
In the wording a student can write in an examination: the human-thinking approach aims to build systems whose internal reasoning matches human cognition, and is validated against evidence from psychology. The rational-agent approach aims to build systems that, given what they have perceived, select the action expected to maximise their performance measure. The two differ in their standard of success: resemblance to a person, against doing well.
Why the distinction matters before anything else is taught
Everything in this paper is an answer to "what should the agent do next", and you cannot judge an answer without a standard. The two standards give different verdicts on the same program.
Take a program that adds two five-digit numbers. A person doing this makes carrying errors, slows down when tired, and sometimes writes the digits in the wrong order. A program built to think like a human would have to reproduce those errors, because they are part of how people add. A program built to be rational simply gets the answer right. Both are legitimate research goals, and only one of them is what a student is asked to build in this paper.
The human-thinking programme
The goal is a system whose reasoning is the same reasoning a person does. Two things follow, and both are demanding.
It needs a theory of how people think. You cannot copy a process you cannot describe. So this programme depends on cognitive psychology, and its experiments are psychological experiments: give a person a puzzle, record the order in which they try things, and check whether the program tries things in the same order.
Its evidence is a match, not a score. Success is the program's behaviour resembling a person's in detail, including the timing, the errors and the things people find hard. A program that solved the puzzle instantly and correctly would be a failure by this standard, because no person does that.
This is called cognitive modelling, and it is a genuine field. It has produced results about human memory and problem solving. But it is not engineering, and a student building a route finder does not want it.
Turing's nine objections are the best map of what this programme was up against, because most of them are objections to the idea that a machine could think at all: the Theological Objection, the "Heads in the Sand" Objection, the Mathematical Objection, the Argument from Consciousness, Arguments from Various Disabilities, Lady Lovelace's Objection, the Argument from Continuity in the Nervous System, the Argument from Informality of Behaviour, and the Argument from Extrasensory Perception. Turing's own response was to sidestep the whole question by moving to behaviour, which is where the other programme starts.
Acting Rationally, or Thinking Like a Human
The rational-agent programme
The goal is a system that acts well. "Well" is not left vague; it is defined, and the definition is the one this syllabus uses everywhere after this chapter.
An agent is rational when, for each possible percept sequence, it selects the action that is expected to maximise its performance measure, given the evidence the percept sequence provides and whatever built-in knowledge the agent has.
Read that in five pieces, because every piece is doing work and each one is worth marks.
- "For each possible percept sequence": rationality is about the action chosen in a situation, not about a fixed plan. The same agent may rightly do different things on different days.
- "Expected to maximise": expected, not guaranteed. An agent that takes the best bet and loses was still rational. This is the single most misunderstood word in the definition.
- "Its performance measure": the standard is external and stated in advance. Rationality is always relative to a measure; change the measure and the rational action changes.
- "Given the evidence the percept sequence provides": the agent is judged on what it could know, not on what was true. Missing a car that was invisible is not irrational.
- "And whatever built-in knowledge the agent has": the designer's knowledge counts as the agent's.
What rationality is not
This section exists because three separate things get confused with rationality, and each confusion loses marks.
It is not omniscience. An omniscient agent knows the actual outcome of every action. No real agent does. Rationality asks for the best decision from the available evidence, and a bad outcome from a good decision is not irrationality. Crossing a road after looking both ways is rational even if a cargo door falls from an aircraft.
It is not perfection. Perfection maximises actual performance; rationality maximises expected performance. Only the second is achievable, which is why it is the standard.
It is not the same as thinking. The definition mentions no internal process whatever. A lookup table that always happens to choose the best action is rational by this definition. Whether that is satisfying is a philosophical question; whether it is what MU examines is not.
Where each programme leads on this syllabus
| Thinking like a human | Acting rationally | |
|---|---|---|
| The goal | reproduce human reasoning | choose the best expected action |
| Validated against | psychological evidence | a stated performance measure |
| Success looks like | the same steps, the same errors | a higher score |
| Needs a theory of | the human mind | the task |
| Where it appears on this syllabus | this chapter, and nowhere else | every chapter after it |
| Who it serves | somebody studying people | somebody building a system |
Acting Rationally, or Thinking Like a Human
Every following row of Module 1 is a tool for the second column. Search finds the action sequence that reaches a goal; logic works out what is true so the goal can be chosen; probability lets the agent act when it does not know. Module 2 is how the agent improves its own choices from experience. Not one of them claims to be how a person does it.
The one place the human side still earns its keep
Two arguments for studying human cognition survive even inside an engineering programme, and they are worth knowing because they are fair.
People are the only working example of general intelligence. Every narrow system in this paper does one task. If the goal is breadth, the only existing proof that breadth is possible is a person, so how people do it is evidence.
Some tasks are defined by human judgement. A translation is good if a speaker of the language accepts it. A recommendation is good if the person likes it. In these tasks the performance measure itself refers to people, so human behaviour cannot be left out of the specification.
Distinctions
| Rationality | Omniscience | |
|---|---|---|
| Knows | what the percepts provide | the actual outcome of every action |
| Maximises | expected performance | actual performance |
| Achievable | yes | no |
| Judged after the fact | no, the decision is judged | yes, the result is judged |
| A rational agent | A human being | |
|---|---|---|
| Standard | a stated performance measure | none stated |
| Errors | only when the expectation was wrong | routine and systematic |
| Memory and arithmetic | exact | approximate |
| Studied by | computer science | psychology |
Quick revision
- Rational agent: for each percept sequence, selects the action expected to maximise its performance measure, given the evidence of the percepts and its built-in knowledge.
- The five load-bearing parts of that definition: per percept sequence, expected, its performance measure, the evidence available, built-in knowledge.
- Human thinking is cognitive modelling: the standard is resemblance to a person, validated by psychology, and a program that does the task perfectly can fail that standard.
- Rationality is not omniscience, not perfection and not a claim about internal thought.
- A good decision with a bad outcome is rational. A lucky guess is not.
- This syllabus takes the rational-agent view from here to the end.
- The human side still matters for two honest reasons: people are the only working general intelligence, and some performance measures refer to people by definition.
Test yourself
1. Define a rational agent. An agent that, for each possible percept sequence, selects the action expected to maximise its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.
Acting Rationally, or Thinking Like a Human
2. Distinguish rationality from omniscience. An omniscient agent knows the actual outcome of its actions; a rational agent maximises the expected outcome from the evidence it has. Omniscience is impossible, which is why rationality is defined on expectation.
3. An agent crosses a road after looking both ways and is hit by debris falling from an aircraft. Was it rational? Yes. The decision was the best available on the evidence. Rationality judges the decision, not the outcome.
4. Why would a program that performs a task perfectly count as a failure in the human-thinking programme? Because that programme's standard is resemblance to human reasoning, including human slowness and human errors. Perfect performance is evidence that the program is not doing what a person does.
5. Give two honest reasons for studying human cognition even if you only want to build systems. People are the only existing example of general intelligence, so their methods are evidence that breadth is achievable; and in tasks such as translation or recommendation the performance measure is defined by human acceptance, so people are part of the specification.
6. Rationality is always relative to something. To what? To the performance measure. Change the measure and the rational action can change, so the measure must be stated before any agent can be called rational.
7. Which of the four definitions of AI does this syllabus adopt, and give one piece of evidence from the syllabus itself. Acting rationally. The evidence is that the very next labels are agent architectures, search, logic and probability, all of which are means of choosing a good action, and none of which claims to model human thought.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.