2. Probability

Please refresh the page if equations are not rendered correctly.
---------------------------------------------------------------

For observation of a random phenomenon, the probability of a particular outcome is the proportion of times that outcome would occur in an indefinitely long sequence of like observations, under the same conditions.

• complement 补
• intersection 交 (and)
• union 并 (or)
• A probability of event A, given event B occurred, is called conditional probability. It is denoted by P(A \mid B).
  

  Placing P(B) in the denominator corresponds to conditioning on (i.e., restricting to) the sample points in B, and it requires P(B)>0. We place P(A B) in the numerator to find the fraction of event B that is also in event A (see Figure 2.3).

  

  P(A \mid B)=\frac{P(A B)}{P(B)}

• Random variable: For a random phenomenon, a random variable is a function that assigns a numerical value to each point in the sample space. Random variables can be discrete or continuous. We use
  • upper-case letters to represent random variables,
  • with lower-case versions for particular possible values.
• Probability distribution: A probability distribution lists the possible outcomes for a random variable and their probabilities.
  • Probability density function for continuous random variables (pdf)
  • probability mass function for discrete random variable (pmf)
  • Cumulative distribution function (cdf)

  The probability P(Y \leq y) that a random variable Y takes value \leq y is called a cumulative probability. The cumulative distribution function is F(y) = P(Y \leq y), for all real numbers y.

  By convention, we denote a pdf or pmf by lower-case f and a c d f by upper-case F.

  F(y) = P(Y \leq y) = \int_{-\infty}^{y} f(u) d u .

  Because we obtain the cdf for a continuous random variable by integrating the pdf, we can obtain the pdf from the cdf by differentiating it.

• Expected value (mean, expectation) of a discrete random variable:

• E(Y)=\sum_{y} y f(y),

• For a discrete random variable Y with pmf f(y) and E(Y)=\mu, the variance is denoted by \sigma^{2} and defined to be:

• \sigma^{2}=E(Y-\mu)^{2}=\sum_{y}(y-\mu)^{2} f(y),

  The variance of a discrete (or continuous) random variable has an alternative formula, resulting from the derivation:

  \begin{aligned}
  \sigma^{2} & =E(Y-\mu)^{2}=\sum_{y}(y-\mu)^{2} f(y) \\
  & =\sum_{y}\left(y^{2}-2 y \mu+\mu^{2}\right) f(y) \\
  & = \sum_{y} y^{2} f(y)-2 \mu \sum_{y} y f(y)+\mu^{2} \sum_{y} f(y) \\
  & =E\left(Y^{2}\right)-2 \mu E(Y)+\mu^{2} \\
  & =E\left(Y^{2}\right)-\mu^{2}
  \end{aligned}

  \operatorname{var}(Y)=\sigma^{2}=E\left(Y^{2}\right)-\mu^{2}

• The standard deviation of a discrete random variable, denoted by \sigma, is the positive square root of the variance.
• Expected value and variability of a continuous random variable for a continuous random variable Y with probability density function f(y),

• E(Y)=\mu=\int_{y} y f(y) d y , \\
  \text { variance } \sigma^{2}=E(Y-\mu)^{2}=\int_{y}(y-\mu)^{2} f(y) d y,

  and the standard deviation \sigma is the positive square root of the variance.

Bayes’ theorem

For any two events A and B in a sample space with P(A)>0,

P(B \mid A)=\frac{P(A \mid B) P(B)}{P(A \mid B) P(B)+P\left(A \mid B^{c}\right) P\left(B^{c}\right)}

Generally, for two events A and B, the multiplicative law of probability states that:

P(A B)=P(A \mid B) P(B)=P(B \mid A) P(A) .

Sometimes A and B are independent events, in the sense that P(A \mid B)=P(A), that is, whether A occurs does not depend on whether B occurs. In that case, the previous rule simplifies:

P(A B)=P(A) P(B) .