Subscribe to DSC Newsletter

# Free Book: Probability and Statistics Cookbook

The format is very similar to a BIG cheat sheet. This cookbook integrates a variety of topics in probability theory and statistics. It is based on literature and in-class material from courses of the statistics department at the University of California in Berkeley but also influenced by other sources .

Author: Matthias Vallentin

Contents

1 Distribution Overview 3

• 1.1 Discrete Distributions . . . . . . . . . . 3
• 1.2 Continuous Distributions . . . . . . . . 4

2 Probability Theory 6

3 Random Variables 6

• 3.1 Transformations . . . . . . . . . . . . . 7

4 Expectation 7

5 Variance 7

6 Inequalities 8

7 Distribution Relationships 8

8 Probability and Moment Generating Functions 9

9 Multivariate Distributions 9

• 9.1 Standard Bivariate Normal . . . . . . . 9
• 9.2 Bivariate Normal . . . . . . . . . . . . . 9
• 9.3 Multivariate Normal . . . . . . . . . . . 9

10 Convergence 9

• 10.1 Law of Large Numbers (LLN) . . . . . . 10
• 10.2 Central Limit Theorem (CLT) . . . . . 10

11 Statistical Inference 10

• 11.1 Point Estimation . . . . . . . . . . . . . 10
• 11.2 Normal-Based Confidence Interval . . . 11
• 11.3 Empirical distribution . . . . . . . . . . 11
• 11.4 Statistical Functionals . . . . . . . . . . 11

12 Parametric Inference 11

• 12.1 Method of Moments . . . . . . . . . . . 11
• 12.2 Maximum Likelihood . . . . . . . . . . . 12
• 12.2.1 Delta Method . . . . . . . . . . . 12
• 12.3 Multiparameter Models . . . . . . . . . 12
• 12.3.1 Multiparameter delta method . . 13
• 12.4 Parametric Bootstrap . . . . . . . . . . 13

13 Hypothesis Testing 13

14 Bayesian Inference 14

• 14.1 Credible Intervals . . . . . . . . . . . . . 14
• 14.2 Function of parameters . . . . . . . . . . 14
• 14.3 Priors . . . . . . . . . . . . . . . . . . . 15
• 14.3.1 Conjugate Priors . . . . . . . . . 15
• 14.4 Bayesian Testing . . . . . . . . . . . . . 15

15 Exponential Family 16

16 Sampling Methods 16

• 16.1 The Bootstrap . . . . . . . . . . . . . . 16
• 16.1.1 Bootstrap Confidence Intervals . 16
• 16.2 Rejection Sampling . . . . . . . . . . . . 17
• 16.3 Importance Sampling . . . . . . . . . . . 17

17 Decision Theory 17

• 17.1 Risk . . . . . . . . . . . . . . . . . . . . 17
• 17.2 Admissibility . . . . . . . . . . . . . . . 17
• 17.3 Bayes Rule . . . . . . . . . . . . . . . . 18
• 17.4 Minimax Rules . . . . . . . . . . . . . . 18

18 Linear Regression 18

• 18.1 Simple Linear Regression . . . . . . . . 18
• 18.2 Prediction . . . . . . . . . . . . . . . . . 19
• 18.3 Multiple Regression . . . . . . . . . . . 19
• 18.4 Model Selection . . . . . . . . . . . . . . 19

19 Non-parametric Function Estimation 20

• 19.1 Density Estimation . . . . . . . . . . . . 20
• 19.1.1 Histograms . . . . . . . . . . . . 20
• 19.1.2 Kernel Density Estimator (KDE) 21
• 19.2 Non-parametric Regression . . . . . . . 21
• 19.3 Smoothing Using Orthogonal Functions 21

20 Stochastic Processes 22

• 20.1 Markov Chains . . . . . . . . . . . . . . 22
• 20.2 Poisson Processes . . . . . . . . . . . . . 22

21 Time Series 23

• 21.1 Stationary Time Series . . . . . . . . . . 23
• 21.2 Estimation of Correlation . . . . . . . . 24
• 21.3 Non-Stationary Time Series . . . . . . . 24
• 21.3.1 Detrending . . . . . . . . . . . . 24
• 21.4 ARIMA models . . . . . . . . . . . . . . 24
• 21.4.1 Causality and Invertibility . . . . 25
• 21.5 Spectral Analysis . . . . . . . . . . . . . 25

22 Math 26

• 22.1 Gamma Function . . . . . . . . . . . . . 26
• 22.2 Beta Function . . . . . . . . . . . . . . . 26
• 22.3 Series . . . . . . . . . . . . . . . . . . . 27
• 22.4 Combinatorics . . . . . . . . . . . . . . 27

Related articles

DSC Resources

Popular Articles

Views: 49364

Comment

### You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Claude Cundiff on February 25, 2018 at 6:04am

Thank You!

Comment by Dusty Mustain on October 4, 2017 at 10:02am
Comment by Jörg Zapf on October 4, 2017 at 9:54am

This article links to a 2011 version. The author moved the book to a separate domain where you can download the current 2017 version. http://statistics.zone

Comment by Andrei Macsin on October 4, 2017 at 9:50am

Link it's at the bottom just before the Related articles section.

Comment by Thomas Hauck on October 4, 2017 at 9:40am

Does the book Probability and Statistics Cookbook exist in PDF format?

Comment by Hafiza Mamona Nazir on October 3, 2017 at 12:20am

## Videos

• ### DSC Webinar Series: Data, Analytics and Decision-making: A Neuroscience POV

Added by Sean Welch

• ### DSC Webinar Series: Knowledge Graph and Machine Learning: 3 Key Business Needs, One Platform

Added by Sean Welch

• ### ODSC APAC 2020: Non-Parametric PDF estimation for advanced Anomaly Detection

Added by Kuldeep Jiwani