Probability and inference

1Why probability?

Last week's umbrella agent could never be sure it would rain. Nothing in its world, or ours, comes with guarantees. Yet some guesses are clearly better than others. "It rained on most cloudy mornings this month" is useful even though it is not a promise.

Probability is the tool for being precise about this. A probability is a number between 0 and 1 that says how much to expect something: 0 means it will not happen, 1 means it certainly will, and 0.7 means "more often than not, about seven times in ten".

You will meet two ways of reading that number:

• As a frequency. "The chance of rain on a cloudy day is 0.6" means that over many cloudy days, it rains on about 60% of them.
• As a degree of belief. "I am 80% sure we are in the rainy season" describes what an agent thinks about one particular situation, given what it knows.

Both readings follow exactly the same rules, and intelligent systems need both. They learn frequencies from experience, and they hold beliefs about things they cannot see directly. This week shows how one turns into the other.

2Counting your way to probability

The simplest way to get a probability is to count. Here are 1,000 days from the umbrella world of Week 1, sorted by what the morning sky looked like and whether it then rained.

Every probability you need can be read off this table by dividing:

• One thing. It rained on 425 of 1,000 days, so P(rain) = 0.425.
• Not. If it rains with probability 0.425, it stays dry with probability 1 − 0.425 = 0.575. Something either happens or it does not, so the two must add to 1.
• And. Cloudy and rained is one cell: 255 days, so P(cloudy and rain) = 0.255. This is called a joint probability.
• Or. Cloudy or rained covers 255 + 172 + 170 = 597 days, so 0.597. Using the totals, 0.427 + 0.425 = 0.852 is too much, because the 255 days that were both cloudy and rainy got counted twice. The fix: P(A or B) = P(A) + P(B) − P(A and B) = 0.427 + 0.425 − 0.255 = 0.597.

3"Given that…": conditional probability

The umbrella agent does not care about rain in general. It cares about rain this morning, given that the sky looks cloudy. That is a conditional probability, written P(rain | cloudy). The bar "|" is read "given".

To work it out, throw away every day that does not match what you know, and count again among the days that are left. There were 427 cloudy days. On 255 of them it rained. So:

On clear mornings it is 170 / 573 ≈ 0.30. Now the Week 1 result makes sense. The agent with no memory took the umbrella on cloudy mornings (60% chance of rain) and left it on clear mornings (30%). It was right on 255 cloudy-and-rainy days plus 403 clear-and-dry days: 658 out of 1,000, or 0.658. That matches the "sky only" accuracy in Week 1's Table 2 (0.656 and 0.658 in the two simulated worlds). The code was quietly estimating conditional probabilities all along.

Independence

Two things are independent when knowing one tells you nothing about the other: P(A | B) = P(A). Rain and clouds are not independent here, because a cloudy sky raises the chance of rain from 0.425 to 0.60. Two separate coin flips are independent: the first landing heads does not change the second. Much of intelligence is about finding things that are not independent of what you care about, because those are the clues worth paying attention to.

4Bayes' rule at the clinic

Here is a situation where getting conditional probability wrong has real consequences. The numbers are made up, but realistic.

A clinic sees many patients with fever. Among them, 10% have malaria. A rapid test is positive for 95% of people who have malaria, and also, wrongly, for 10% of people who do not. A patient tests positive. How likely is it that they have malaria?

Many people, including many trained professionals, answer "95%". Let us count instead. Imagine 1,000 feverish patients:

• 100 have malaria. The test is positive for 95 of them.
• 900 do not. The test is positive for 10% of them: 90 people.
• So 95 + 90 = 185 people test positive, and only 95 of those have malaria.

The answer is 95 / 185 ≈ 0.51. A positive result means roughly a coin flip, not near-certainty. The test is good. The problem is that healthy patients so outnumber sick ones that even a small false-positive rate produces almost as many false alarms as true detections.

The rule itself

What we just did by counting is Bayes' rule. Write H for the hypothesis (has malaria) and E for the evidence (positive test):

The bottom line, P(E), is all the ways the evidence could have happened, whether H is true or not:

So P(H | E) = 0.095 / 0.185 ≈ 0.51, as before. Each piece has a name you will hear constantly:

Prior, P(H)

What you believed before the evidence: 10% of patients have malaria.

Likelihood, P(E | H)

How expected the evidence is if the hypothesis is true: 0.95.

Posterior, P(H | E)

What you should believe after the evidence: about 0.51.

Part A of week02_bayes.py checks the rule against brute force. It simulates a million patients and counts. Bayes' rule gives 0.514; counting gives 0.515. The rule is just counting done with fractions instead of people.

Today's posterior is tomorrow's prior

Suppose the patient takes a second, independent test and it is also positive. Start from the new belief, 0.51, and apply the rule again: 0.95 × 0.51 on top, and 0.95 × 0.51 + 0.10 × 0.49 underneath. The answer is about 0.91. Two positives make malaria very likely. Evidence accumulates, and Bayes' rule says exactly how.

5Beliefs that update: inference about hidden things

Inference means reasoning from what you can see to what you cannot. The clinic inferred a hidden disease from a visible test. The umbrella agent faces the same kind of problem every day: the season is hidden, and rain is the evidence.

In Week 1 the agent dealt with this by remembering the last k days. Here is another approach. Keep a single number, the belief b = P(we are in the rainy season), and update it twice a day.

1. Overnight: predict. The season might have switched. With persistence 0.98, a rainy season stays rainy with probability 0.98. So tomorrow's belief, before seeing anything, is 0.98 × b + 0.02 × (1 − b).
2. Evening: update. Did it rain today? Apply Bayes' rule, with "rainy season" as the hypothesis and today's weather as the evidence.

def predict(belief, persistence):
    # Overnight the season may switch.
    return persistence * belief + (1 - persistence) * (1 - belief)

def update(belief, rained):
    # Bayes' rule: how expected was today's weather in each season?
    like_rainy = 0.7 if rained else 0.3      # rainy season: rain 70% of days
    like_dry = 0.15 if rained else 0.85      # dry season: rain 15% of days
    top = like_rainy * belief
    return top / (top + like_dry * (1 - belief))

One day, worked by hand

Start knowing nothing: b = 0.5. It rains on day 1. Update: 0.7 × 0.5 = 0.35 on top, and 0.35 + 0.15 × 0.5 = 0.425 underneath, giving 0.35 / 0.425 ≈ 0.82. Rain is much more typical of the rainy season, so belief jumps. Overnight the season might switch, so it relaxes slightly: 0.98 × 0.82 + 0.02 × 0.18 ≈ 0.81. That is the belief on the morning of day 2, exactly what the program prints.

Read down the table. Each rainy day pushes belief up; each dry day pulls it down. Three dry days in a row (days 2–4) take it from 0.81 to 0.17. The belief never reaches 0 or 1, because the agent always allows that the evidence might be a fluke.

One number against a week of memory

Each morning the belief agent combines its belief with the sky to estimate the chance of rain, and takes the umbrella if that is above 0.5. On the same 20,000 test days as Week 1:

The belief agent does slightly better while storing far less. That is a big idea in a small package: the whole relevant past can be squeezed into one well-chosen number. An agent does not need to remember everything that happened; it needs to remember what the past says about the hidden things that matter now. A summary like this is called a belief state.

Exploratory observation In this simulation, a one-number belief matched seven days of raw memory. Whether compact summaries like this explain how real systems use memory is one of the questions behind the course's research programme.

6Vectors and matrices, without fear

"Linear algebra" sounds intimidating. At this level it is mostly a tidy way to write lists of numbers and weighted sums. You have already done it without the names.

A belief is a vector

The belief agent's state can be written as a list of two probabilities, one for each season: [P(rainy), P(dry)]. After day 1 that is [0.82, 0.18]. A list of numbers like this is a vector.

A change is a matrix

The overnight step can be written as a small table of how seasons turn into seasons:

Multiplying the matrix by the belief vector means: for each row, multiply across and add up.

def mat_vec(T, v):
    return [sum(T[i][j] * v[j] for j in range(len(v))) for i in range(len(T))]

T = [[0.98, 0.02],
     [0.02, 0.98]]
print(mat_vec(T, [0.82, 0.18]))    # [0.8072, 0.1928], about [0.81, 0.19]

That is exactly the predict function from Section 5, written for any number of seasons. Seminar exercise 4 uses it for a world with three seasons.

An expected cost is a dot product

Week 1's last seminar exercise asked about costs: getting soaked costs 3, carrying the umbrella for nothing costs 1. Suppose the chance of rain this morning is 0.6. Write the chances as a vector [0.6, 0.4] for [rain, no rain] and each action's costs the same way:

• Take the umbrella: costs [0, 1]. Expected cost = 0.6 × 0 + 0.4 × 1 = 0.4.
• Leave it: costs [3, 0]. Expected cost = 0.6 × 3 + 0.4 × 0 = 1.8.

Multiplying two vectors position by position and adding up is called a dot product, and the result here is an expected value: the average cost you would pay if you faced this morning many times. Take the umbrella, because 0.4 < 1.8. In general, taking it is better whenever 3p > 1 − p, that is, whenever p > 0.25. When mistakes have different costs, the right threshold is not 0.5.

7Summary

• A probability is a number from 0 to 1. It can describe a frequency or a degree of belief, and both follow the same rules.
• Counting gives probabilities directly. "Not" is 1 minus; "or" adds and subtracts the overlap.
• A conditional probability counts only the cases that match what you know. P(A | B) and P(B | A) can be very different.
• Bayes' rule turns a prior and a likelihood into a posterior. Ignoring the base rate leads to badly wrong conclusions.
• Inference reasons from visible evidence to hidden causes. Repeated Bayes updates give a belief that tracks a hidden state over time.
• A one-number belief did as well as seven days of raw memory, because it keeps what matters about the past and drops the rest.
• Vectors, matrices and dot products are compact ways to write beliefs, changes and expected costs.

·Key terms

Probability

A number from 0 to 1 describing how much to expect something.

Joint probability

The probability that two things both happen, P(A and B).

Conditional probability

The probability of A once you know B has happened, P(A | B).

Independence

Knowing B tells you nothing about A: P(A | B) = P(A).

Prior

Belief before seeing the evidence.

Likelihood

How expected the evidence is if a hypothesis is true.

Posterior

Belief after seeing the evidence.

Bayes' rule

Posterior = likelihood × prior ÷ probability of the evidence.

Base rate

How common something is to begin with, before any specific evidence.

Inference

Reasoning from what can be observed to what cannot.

Belief state

A summary of the past that captures what it implies about hidden things now.

Vector

An ordered list of numbers.

Matrix

A table of numbers that turns one vector into another.

Dot product

Multiply two vectors position by position and add the results.

Expected value

The probability-weighted average of the possible outcomes.

·Check your understanding

Try each question before opening the answer.

1. Using Table 1, what is P(clear and dry)? What is P(dry | clear)?
   Show answer

   P(clear and dry) = 403 / 1,000 = 0.403. P(dry | clear) = 403 / 573 ≈ 0.70: among clear mornings only, it stayed dry about 70% of the time.

2. The malaria test is right 95% of the time for sick patients. Why is a positive result only about 51% reliable at this clinic?
   Show answer

   Because most patients do not have malaria. In 1,000 patients, 900 are healthy, and 10% of them (90 people) test positive anyway. That is almost as many as the 95 true positives, so a positive result is close to a coin flip.

3. Suppose instead that half the patients at a clinic have malaria. With the same test, what is P(malaria | positive)?
   Show answer

   0.95 × 0.5 = 0.475 on top; 0.475 + 0.10 × 0.5 = 0.525 underneath. 0.475 / 0.525 ≈ 0.90. The same test means much more when the condition is common.

4. In Table 2, belief falls from 0.35 on the morning of day 4 to 0.17 on the morning of day 5. Explain why in one sentence.
   Show answer

   It stayed dry on day 4, and dry days are much more typical of the dry season (85% of days) than the rainy season (30%), so Bayes' rule lowers the belief that it is the rainy season.

5. With costs of 3 for a soaking and 1 for carrying the umbrella for nothing, should you take it when the chance of rain is 0.2?
   Show answer

   Take it: 0.2 × 0 + 0.8 × 1 = 0.8. Leave it: 0.2 × 3 + 0.8 × 0 = 0.6. Leave it, because 0.6 < 0.8. This agrees with the rule "take it only when p > 0.25".

·Seminar exercises

As always, write your prediction before you run anything.

1. Two tests. Work out by hand the chance of malaria after two independent positive tests (the notes say about 0.91). Then extend count_positive to simulate patients who take two tests, and check your answer.
2. Wrong about the world. Give the belief agent wrong numbers: tell it the persistence is 0.8 when it is really 0.98, or that the rainy season has rain 90% of the time. How much does accuracy drop? Which mistake hurts more, and why?
3. No seasons to track. Run the belief agent in the Week 1 world where the season flips at random every day (persistence 0.5), telling it the true persistence. Predict its accuracy first. Compare with Week 1's Table 2.
4. Three seasons. Add a third season, harmattan: dry and dusty, with rain on 5% of days. Write a 3 × 3 matrix T where each column adds up to 1, and use mat_vec for the overnight step. Rewrite update so that it works for any number of seasons.

·Readings

• Core. Blitzstein, J. K., & Hwang, J. (2019). Introduction to probability (2nd ed.). CRC Press. Chapters 1 and 2. The book is freely available online from its authors.
• Core. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157). Find the passage on insensitivity to prior probability. How does it relate to the clinic example?
• Further. Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101. Read the introduction and the description of belief states; skip the algorithms for now.