Instructor:

Kim Border, 205 Baxter Hall, x 4218, kcborder@caltech.edu

Office Hours:

Fridays, 1:30–3:00 pm

Lead TA:

Marius Lemm, mlemm@caltech.edu

Office Hours:

5:30 PM, Sunday
156 Sloan

Course Secretary:

Kristy Aubry, 253 Sloan, x 4087, kaubry@caltech.edu

• Send anonymous feedback to the professor. This really is anonymous, so you cannot receive a personal response. If you want a response, just send an ordinary email to the address above. Please be constructive rather than abusive.

• March 19. Solutions for final exam have been posted.
• March 9. Professor Gillen has recommended perusing this supplementary reading for Lecture 27.
• March 8. It seems some of you like to typeset your homework, so by popular demand, here is the \(\LaTeX\) source code for the Table in HW 8.
• March 6. On Wednesday, March 11, which is the last day of instruction, there will be a special guest lecture by Professor Ben Gillen. The topic will be empirical Bayesian estimation and its applications. There will be no lecture notes posted on line, but there may be a question on the final related to the material.
• March 6. The Final Exam will be available by Noon on Wednesday, March 11, which is the last day of instruction. It will be due on Wednesday, March 18 at Noon. There will be a Review Session on Friday the 13th of March in Baxter Lecture Hall at 2:00 p.m.
• January 16, 3:30 pm. The second Bonus Question already has two winners: Youry Aglyamov and Grace Lee. If you get me a solution soon, you may still qualify for a free lunch.
• January 14. The first Bonus Question winners are Milica Kolundzija and Adrian Costantino.
• Reading schedule for the term (Subject to change)
  • Reading list

Introduction to the fundamental ideas and techniques of probability theory and statistical inference.

Prerequisites

Ma 1abc. In addition, some familiarity with a scientific computing language or program (e.g., Matlab, Mathematica, R, NumPy) is assumed.

Probability will be covered in the first half of the term (using Pitman) and statistics (using Larsen and Marx) in the second half (see below for information regarding textbooks). Main topics covered are:

• Properties of probability
• Independence, conditional probability, Bayes' Law
• Random variables, distributions, densities, and expectation
• Joint distributions, marginals, covariance, correlation
• The Law of Large Numbers
• The Central Limit Theorem
• Important distributions
  • Bernoulli, Binomial
  • Uniform
  • Normal (Gaussian)
  • Exponential, Poisson
  • Gamma, Beta, Chi-square
  • Conjugate prior/posterior pairs
• Introduction to stochastic processes
  • Random walk
  • Markov chains
  • Martingales
• Estimation of parameters
  • Consistency, unbiasedness
  • Maximum likelihood estimation
  • Confidence intervals
  • Cramér–Rao lower bound
• Testing statistical hypotheses
  • Significance tests
    • Likelihood ratio tests
    • Monotone Likelihood Ratio Property and the Neyman–Person Lemma
    • Type I and Type II errors
    • Critical values
  • Specification tests
    • Kolmogorov–Smirnov
    • \(\chi^2\)
  • Nonparametric tests
    • Wilcoxon
• Linear regression analysis
  • Gauss Markov–Theorem
  • ANOVA
• Introduction to modern statistical techniques
  • Monte Carlo methods
  • Bootstrap
  • Statistics and machine learning

Late Work

As rule, late work is not accepted. This is to protect the TAs, who are talented hardworking students, just as you are. At the discretion of the Head TA, late homework turned in the day it is due, but after the 4:00 pm deadline will be accepted with a 25% penalty. (If there are extenuating circumstances, you must notify the Head TA in advance and a note from the Dean will be needed.) As partial compensation, your lowest homework score will be discarded. (Since the homework assignments have different weights, the Kasatkin Algorithm will be employed to decide which assignment to drop. It is designed to give you the best score possible.)

Grading:

Your course grade will be based on the weekly homework (40%), the midterm (25% or 35%), and the final (35% or 25%). The weights on the final and midterm will put the greater weight on the better exam. In computing the homework average, your lowest homework score will be dropped. This is in lieu of a late homework policy: no late homework will be accepted without both prior notification of your TA by midnight the night before the set is due, and a note from the Dean. As this course is for a letter grade, no one will be excused from the final.

Homework:

Homework will be typically be due at 4:00 pm on Mondays in the appropriate homework box outside 253 Sloan. (If Monday is a holiday [which happens twice this term] homework will be due on Tuesday. Assignment 0 is a major exception.) Problems (and later solutions) will be posted on this course webpage. You are encouraged to start the homework well in advance of the due date in order not to risk missing the deadline. Homework is turned in to locked boxes, so it can safely be submitted as soon as it is completed.

Collaboration:

Collaboration is allowed on the homework, but your write-up must be in your own words and may not be copied. Collaboration is not allowed on the exams. Please ask for clarification if anything is unclear.

*Information is subject to change*

The required textbooks for the course are:

• Jim Pitman. 1993. Probability. Springer, New York, Berlin, and Heidelberg. ISBN: 0-387-97974-8.
• Richard J. Larsen and Morris L. Marx. 2012. An Introduction to Mathematical Statistics and Its Applications, fifth edition. Prentice Hall. ISBN: 0-321-69394-9.

There will be additional readings from time to time, either as handouts or articles available on line.

There are other books that you may find useful for this course or perhaps later in life. Here are some of my recommendations.

• Robert B. Ash. 2008. Basic Probability Theory. Dover, Mineola, New York. Reprint of the 1970 edition published by John Wiley and Sons. ISBN: 0-486-46628-0. This book, being published by Dover, is very affordable. (I think it's still just under $20.) The first chapter, especially sections 1.4 through 1.7 are very good at explaining how to count for combinatorial problems.
• Calvin Dytham. 2011. Choosing and Using Statistics: A Biologist's Guide. Wiley-Blackwell. ISBN: 978-1-4051-9839-4. This is a cookbook and reference geared toward biologists, but is a useful reference for almost everyone.
• John B. Walsh. 2012. Knowing the Odds: An Introduction to Probability. American Mathematical Society, Providence, Rhode Island. ISBN: 978-0-8218-8532-1. I almost used this as the textbook for the course, but decided to stay with the status quo.
• Kai Lai Chung and Farid Ait-Sahlia. 2003. Elementary Probability Theory with Stochastic Processes and an Introduction to Mathematical Finance. Springer-Verlag, New York, Heidelberg, and Berlin. ISBN: 978-0-387-95578-0. This is a very well-written introduction to probability theory. Chapter 3 on counting is especially good.
• Richard Isaac. 1995. The Pleasures of Probability. Springer-Verlag, New York, Berlin, and Heidelberg. ISBN: 0-387-94415-X. Another good introduction to probability theory, but a bit too eccentric to use as the main text for this course.

Modern statistical practice is computationally intensive, but this course is not especially so. But you will have to use computers to do some of the assignments. Many of the people on campus that I have talked to recommend the statistical programming language R (the open source alternative to AT&T's S). The new Mathematica 9 and 10 claim to be highly integrated with R, but I haven't tried it yet. Others I have talked to rave about NumPy, an extension of Python that provides much of the functionality of Matlab. Still others continue to use other packages because they have invested a lot of effort in learning to use them. (I myself often use Mathematica because I started using it in 1992, so my recommendation of R falls into the category of do as I say, not as I do.) My son recommends R and this video as an endorsement.

Here are a couple of highly recommended books on R that I mostly have not read. But I find the first two to be useful.

• Claus Thorn Ekstrom. 2011. R Primer. Chapman & Hall/CRC Press. Available for on-line reading from the Caltech Library.
• Paul Teetor. 2011. R Cookbook. O'Reilly Media. ISBN: 978-0-596-80915-7.
• Joseph Adler. 2012. R in a Nutshell, 2nd edition. O'Reilly Media. ISBN: 978-1449312084.
• Alain F. Zuur, Elena N. Ieno, and Erik H. W. G. Meesters. 2009. A Beginner's Guide to R. Springer Science+Business Media, New York. ISBN: 978-0-387-93836-3.
• Peter Dalgaard. 2008. Introductory Statistics with R, second edition. Springer Science+Business Media, New York. ISBN: 978-0-387-79053-4

Please report any errors or suspected errors or confusing bits to the professor. You can use the anonymous feedback form if you wish.

Notes for Lectures:

• Lecture 1
• Lecture 2
• Lecture 3
• Lecture 4
• Lecture 5
• Lecture 6
• Lecture 7
• Lecture 8
• Lecture 9
• Lecture 10
• Lecture 11
• Lecture 12
• Lecture 13
• Lecture 14
• Lecture 15
• Lecture 16
• Lecture 17
• Lecture 18
• Lecture 19
• Lecture 20
• Lecture 21
• Lecture 22
• Lecture 23
• Lecture 24
• Lecture 25
• Lecture 26
• Lecture 27

Data for the Final

• HubbleData.txt HubbleData.xlsx
• TimesDeathData.txt

Supplementary handouts:

• Supplementary reading for Lecture 27.
• Some sums
• A PDF of PDFs (An anonymous student suggested this title for the review of selected distributions. Thanks.)
• Quick review of linear algebra
• Supplement to Lecture 17: The Fréchet-Cramér-Rao Lower Bound.
• Additional supplement to Lecture 17, providing citations for when you can differentiate under an integral sign.

• Assignment 0. Due Thursday, January 8, 4:00 p.m. on line here.
• Assignment 1. Due Monday, January 12, 4:00 pm.
  • .
• Assignment 2. Due Tuesday, January 20, 4:00 pm.
• Assignment 3. Due Monday, January 26, 4:00 pm.
• Assignment 4. Due Groundhog Monday, February 2, 4:00 pm.
• Updated 2015.02.11, 8:43 p.m.
• Assignment 5. Due Fat Tuesday, February 17, 4:00 pm.
• Assignment 6. Due Monday, February 23, 4:00 pm.
• Assignment 7. Due Monday, March 2, 4:00 pm.
• Assignment 8. Due Monday, March 9, 4:00 pm.
  • By popular demand, here is the \(\LaTeX\) source code for the Table in HW 8.