$\require{cancel} \newcommand{\Ket}[1]{\left|{#1}\right\rangle} \newcommand{\Bra}[1]{\left\langle{#1}\right|} \newcommand{\Braket}[1]{\left\langle{#1}\right\rangle} \newcommand{\Rsr}[1]{\frac{1}{\sqrt{#1}}} \newcommand{\RSR}[1]{1/\sqrt{#1}} \newcommand{\Verti}{\rvert} \newcommand{\HAT}[1]{\hat{\,#1~}} \DeclareMathOperator{\Tr}{Tr}$

Probability - Basics

^{First created in October 2012}

Possible Outcomes of a Repeatable Experiment

If an experiment can be performed repetitively under the same conditions, some outcomes are bound to happen more often and some less so. Probability, or chance, is the ratio of the number of "successful" outcomes (statistically) to the number of all "possible" outcomes.

Fundamental Formula: $\boxed{\frac{\text{Number of successful outcomes}}{\text{Number of all possible outcomes}}}$

In this context, no experiments need to be performed. We assume all "simple" outcomes (like drawing a particular ball from a bag) are equally probable. The mathematics here is to calculate the probability of "complex" events from the probability of "simple" events based on some rules.

Also assumed is that all elements are "distinguishable" for the purpose of statistical experiments. For example, there are ten "distinguishable" ways to draw a ball out of a bag of ten, even though all balls are identical.

Set Theory Basics

Set notations used here: $~\mathcal P(A)$: Power set - set of all subsets of $A$; $~S$: Universal set; $~\emptyset$ (or $\{~\}$): Empty set.

$A^c$: Complement of $A$; $~|A|$: Number of elements in $A$; $~|A^c|=|S|-|A|$; $~|A\cup B|=|A|+|B|-|A\cap B|.$

$A$ and $B$ are disjoint iff $A\cap B=\emptyset.$ Disjoint sets $A_r (r=1,2,\ldots,k)$ partition $B$ iff $\bigcup_{r=1}^k A_r=B$.

Definitions

Sample space: Set of all possible outcomes.

Event: Subset of a sample space.

Probability: A real function $P$ on $\mathcal P(S)$ that satisfies:

• For all $A\subseteq S,~~0\le P(A)\le 1$ (Probability is always between 0 and 1 inclusively.)
• $P(\emptyset)=0$ (Nothing is impossible.)
• $P(S)=1$ (All are certain.)
• $A\cap B=\emptyset\Rightarrow P(A\cup B)=P(A)+P(B).$
  (The probability of either one of two mutually exclusive events equals to the sum of their probabilities)

From the above definition, the following rules can be derived for a finite sample space $S$:

$P(A)=\sum_{a\in A}P(\{a\}).$ Probability is the sum of the probabilities of (mutually exclusive) elements.

$P(A)=\frac{|A|}{|S|},$ if $P(\{a\})$ is constant for all $a\in S$. Probability is the ratio of successes over possibles.

$\sum_{a\in S}P(\{a\})=1.$ All are certain.

Also: $P(A\cup B)=P(A)+P(B)-P(A\cap B);~~P(A^c)=1-P(A);~$ If $A\subseteq B$, then $P(A)\le P(B)$.

Conditional probability of $A$ given $B$: $\displaystyle~P(A|B)=\frac{P(A\cap B)}{P(B)},~$ where $P(B)\ne 0.$

This derives the Multiplication Rule: $\displaystyle P(A\cap B)=P(A|B)P(B)=P(B|A)P(A).~~$

Alternatively, Bayes' Rule: $\displaystyle P(A|B)=\frac{P(B|A)P(A)}{P(B)}~$ where $P(B)\ne 0.$

Partitioning

$P$ is a partition of $X$ means:

• $\forall A\in P, A\subset X.$
• $\emptyset\notin P.$
• $\cup_{A\in P}A=X.$ ($P$ covers $X$.)
• $(\forall A,B\in P)~A\ne B\Rightarrow A\cap B=\emptyset.$

Let $A_1,\ldots,A_n$ partition $X$.

Total Probability Rule: If $B$ is an event in $X$ ($B\subseteq X$), then $\displaystyle P(B)=\sum_{i=1}^n P(B|A_i)P(A_i).$

Statistical Independence:

Events $A$ and $B$ are statistically independent iff $P(A\cap B)=P(A)P(B).~$ Alternative view: $P(A)=P(A|B)~\big(\text{given }P(A\cap B)=P(A|B)P(B)=P(A)P(B)\big).$

Disjoint events $A_1,\ldots,A_n$ are mutually independent iff $\displaystyle\forall A_{i_1},\ldots,A_{i_m}, P\left(\bigcap_{k=1}^m A_{i_k}\right)=P\left(\prod_{k=1}^m A_{i_k}\right),$ where $m\le n.$

%Sample Space: %The set of all permutations. % %e.g. For 3 distinct elements $A$, $B$, and $C$, there are 6 permutations, which forms the sample space %$S=\{ABC, ACB, BCA, BAC, CAB, CBA\}.$ Its cardinality (number of members) is $|S|=3!=6.$

End of Article