Ask HN: Learning advanced math
I have an undergraduate background in applied math and have taken basic linear algebra, differential equations, mathematical statistics and multivariate calculus. I'm rusty though, and I've been considering applying for PhD programs in statistics. I'd like to put myself on a healthy math regiment and I was wondering if people had suggestions on books or other materials to work on advanced (linear) algebra and analysis? I'm more than willing to spend an hour per page and do all the exercises, but I would like good exposition. My end goal is to have a reasonable understanding of things to make limit theorems in probability (during the first year of my PhD), etc, easier.
42 comments
For real analysis, I suggest Bartle's "Introduction...". I had a horrible teacher (well, I loved him, but he never prepped for class), and was able to learn everything by working through the proofs in the text. Bartle has a multi-dimensional version too.
For linear algebra, I was suggest Strang's "Linear Algebra with applications", 3rd edition. Then "Linear Algebra Done Right".
While you are at it, get Hungerfords "Abstract Algebra: An Introduction"; you will need an easy reference to fields and groups and polynomials.
All these books require an hour per page, but they lay it all out for you if you work for it. These are definitely undergrad books, but that is their beauty.
Bartle also has "Elements of integration and lebesgue measure" -- I bet it is great, but I haven't used it.
And if you find a good probability book, please post the title ;)
For linear algebra, I was suggest Strang's "Linear Algebra with applications", 3rd edition. Then "Linear Algebra Done Right".
While you are at it, get Hungerfords "Abstract Algebra: An Introduction"; you will need an easy reference to fields and groups and polynomials.
All these books require an hour per page, but they lay it all out for you if you work for it. These are definitely undergrad books, but that is their beauty.
Bartle also has "Elements of integration and lebesgue measure" -- I bet it is great, but I haven't used it.
And if you find a good probability book, please post the title ;)
Upvoted for suggesting Linear Algebra Done Right.
To get a sense of the approach, read his earlier article Down with Determinants at http://www.axler.net/DwD.html. That article was the first time I felt I really understood a lot of the material I had already theoretically learned and done well at.
On the real analysis side, I learned from Royden and liked it. But my mind definitely heads the analysis way, and what I find easy someone else might not.
To get a sense of the approach, read his earlier article Down with Determinants at http://www.axler.net/DwD.html. That article was the first time I felt I really understood a lot of the material I had already theoretically learned and done well at.
On the real analysis side, I learned from Royden and liked it. But my mind definitely heads the analysis way, and what I find easy someone else might not.
I definitely enjoyed Royden a lot -- I got through my real analysis qual back in grad school using Royden to teach myself essentially all of the material. Readable and excellent. And as to minds heading the analysis way: I'm about as algebraicly-minded as they come and I thought it was highly readable.
Al Drake's book on probability is pretty good. If you know of any good introductory level Combinatorics books then please enlighten me ;)
I learned linear algebra from axler and real analysis from bartle and sherbert too. :) Great books.
Good probability book? What's wrong with Feller? ;-)
Linear algebra: well, there are lots of good theory books but I'm a big fan of the computational approach, and Golub and Van Loan is my pick there hands down!
Linear algebra: well, there are lots of good theory books but I'm a big fan of the computational approach, and Golub and Van Loan is my pick there hands down!
Real Analysis! OMG that brings back bad memory. Got a horrible teacher as well. Somehow Complex Analysis was easier. Sadly I have forgot all of them now. :(
Advanced Linear Algebra, Roman; Linear Algebra, Hoffman & Kunze; Matrix Analysis, Horn & Johnson; Principles of Mathematical Analysis, Rudin; Real Analysis, Royden.
Miscellaneous comments:
- Reading pure abstract algebra (e.g. Dummit & Foote) isn't a good use of time if you intend to go into statistics, since it only shows up in a few very special subareas. If you decide to go into one of these areas, you can learn this later.
- More advanced books on linear algebra usually emphasize the abstract study of vector spaces and linear transformations. This is fine, but you also need to learn about matrix algebra (some of which is in that Horn & Johnson book) and basic matrix calculus, since in statistics, you'll frequently be manipulating matrix equations. The vector space stuff generally does not help with this, and this material isn't in standard linear algebra books. (Similarly, you should learn the basics of numerical linear algebra and optimization -- convex optimization in particular shows up a lot in statistics.)
- People have different opinions on books like Rudin, but you need to learn to read material like this if you're going into an area like probability. It's also more or less a de facto standard, so it is worth reading partly for that reason as well. So read Rudin/Royden (or equivalent, there are a small handful of others), but supplement them with other books if you need (e.g. 'The Way of Analysis' is the complete opposite of Rudin in writing style). It helps to read a few different books on the same topic simultaneously, anyway.
- Two books on measure-theoretic probability theory that are more readable than many of the usual suspects are "Probability with Martingales" by Williams and "A User's Guide to Measure-Theoretic Probability" by Pollard. There is also a nice book called "Probability through Problems" that develops the theory through a series of exercises.
Miscellaneous comments:
- Reading pure abstract algebra (e.g. Dummit & Foote) isn't a good use of time if you intend to go into statistics, since it only shows up in a few very special subareas. If you decide to go into one of these areas, you can learn this later.
- More advanced books on linear algebra usually emphasize the abstract study of vector spaces and linear transformations. This is fine, but you also need to learn about matrix algebra (some of which is in that Horn & Johnson book) and basic matrix calculus, since in statistics, you'll frequently be manipulating matrix equations. The vector space stuff generally does not help with this, and this material isn't in standard linear algebra books. (Similarly, you should learn the basics of numerical linear algebra and optimization -- convex optimization in particular shows up a lot in statistics.)
- People have different opinions on books like Rudin, but you need to learn to read material like this if you're going into an area like probability. It's also more or less a de facto standard, so it is worth reading partly for that reason as well. So read Rudin/Royden (or equivalent, there are a small handful of others), but supplement them with other books if you need (e.g. 'The Way of Analysis' is the complete opposite of Rudin in writing style). It helps to read a few different books on the same topic simultaneously, anyway.
- Two books on measure-theoretic probability theory that are more readable than many of the usual suspects are "Probability with Martingales" by Williams and "A User's Guide to Measure-Theoretic Probability" by Pollard. There is also a nice book called "Probability through Problems" that develops the theory through a series of exercises.
The roadmap would be to first learn analysis (advanced calculus), then measure theory, then get a measure theory probability book.
I'll assume you are teaching this to yourself.
For analysis, I wouldn't get Rudin's book (concepts are poorly motivated). There are plenty of good Dover books. But I haven't read them because, well, I learned from Rudin.
For measure theory, I'd read Kolmogorov and Fomin's book. Rudin also has a measure theory book, which is much better than his analysis book, but it's hefty. Good problems though.
For a book on Probability, we read A Probability Path at my university. I wasn't fond of it. Someone referred me to Probability with Martingales, and though I didn't read it, it looked very good.
I'll assume you are teaching this to yourself.
For analysis, I wouldn't get Rudin's book (concepts are poorly motivated). There are plenty of good Dover books. But I haven't read them because, well, I learned from Rudin.
For measure theory, I'd read Kolmogorov and Fomin's book. Rudin also has a measure theory book, which is much better than his analysis book, but it's hefty. Good problems though.
For a book on Probability, we read A Probability Path at my university. I wasn't fond of it. Someone referred me to Probability with Martingales, and though I didn't read it, it looked very good.
One of my favorite not-much-talked about analysis books is "A Radical Approach to Real Analysis" by Bressoud. Must read if you enjoy reading about the history of mathematics and famous mathematicians. A lot of people here recommend Rudin or Royden, but I have seen many people become turned off to the whole field of analysis because of how terse and user-unfriendly those are. In that sense, ARTRA is the polar opposite. The follow-up, A Radical Approach to Lebesgue's Theory of Integration, is also superb.
Baby Rudin is a great choice, especially for self study, since there are a number of study guides for it that go through solutions, additional explanation of material, etc. I'd recommend against the chapters on differential forms, though, just because the treatment is outdated. Marsden and Hoffman's approach is to focus on explanations for why theorems are true before giving proofs, which some people find useful.
Axler is a good choice for linear algebra. Dummit and Foote is the standard choice for algebra generally. I'm of the opinion that we should teach algebra before linear algebra in general, but this seems like a minority view.
Axler is a good choice for linear algebra. Dummit and Foote is the standard choice for algebra generally. I'm of the opinion that we should teach algebra before linear algebra in general, but this seems like a minority view.
I second Baby Rudin.
The problem sets are nearly legendary and the writing is terse. I might have to read through it again myself sometime.
"Algebra" by Artin is also a great choice if you want to learn about modern algebra. There's some good stuff in there for looking at linear transformations in, say, 2-space, as groups.
The problem sets are nearly legendary and the writing is terse. I might have to read through it again myself sometime.
"Algebra" by Artin is also a great choice if you want to learn about modern algebra. There's some good stuff in there for looking at linear transformations in, say, 2-space, as groups.
For linear algebra, Sheldon Axler's _Linear Algebra Done Right_.
For analysis, Stephen Abbott's _Understanding Analysis_.
These two books will give you a very solid grounding in the undergraduate linear algebra and analysis. Personally, I also worship the style of Baby Rudin (it's the nickname of his _The Principles of Mathematical Analysis_), but it can be too dry to many people.
These two books will give you a very solid grounding in the undergraduate linear algebra and analysis. Personally, I also worship the style of Baby Rudin (it's the nickname of his _The Principles of Mathematical Analysis_), but it can be too dry to many people.
Trefethen and Bau is a good numerical linear algebra text, a fair bit of statistics can be understood in the linear algebra setting and this book helps teach how to think about linear algebra in a very useful way. I don't have a great reference, but stochastic PDEs is definitely a hot topic and promises to remain as such for a long time.
I would recommend video lectures from the Khan Academy (http://www.khanacademy.org/). They are organized in nice little chucks of topics and have a large range of topics in math.
The Khan academy ends where OP says he left off. (i.e., advanced linear algebra).
Michaek Spivak's "Calculus" is pretty great for learning real analysis. The other canonical text is "Principles of Mathematical Analysis" by Walter Rudin, which I've never used, but is purported to be of the same quality.
Actually,
I wouldn't recommend either of those two book for self-teaching.
For whatever reason, I taught myself advance mathematics in High Schools. Spivak and Rubin were pretty inaccessible. Sure, they are rigorous and high level but that meant they didn't lend themselves to an easy read. "Real Analysis" by Royden was relatively quick to go through - and I had only the start of calculus. Royden gives a fairly simplistic and accessible development of higher mathematics starting with set theory.
I'd imagine that would be the most helpful.
(In my High School years, I went from algebra sophomore year to reading Royden and doing a community college calculus class junior year to passing a differential geometry course and the undergraduate honors seminar at UCLA).
I wouldn't recommend either of those two book for self-teaching.
For whatever reason, I taught myself advance mathematics in High Schools. Spivak and Rubin were pretty inaccessible. Sure, they are rigorous and high level but that meant they didn't lend themselves to an easy read. "Real Analysis" by Royden was relatively quick to go through - and I had only the start of calculus. Royden gives a fairly simplistic and accessible development of higher mathematics starting with set theory.
I'd imagine that would be the most helpful.
(In my High School years, I went from algebra sophomore year to reading Royden and doing a community college calculus class junior year to passing a differential geometry course and the undergraduate honors seminar at UCLA).
op, i don't know your mind but i agree, Rudin/Spivak may prove challenging, especially without the help of a professor/TA to answer your questions in real life... i would not go and buy them blindly. the best course of action you could take would be to go visit a college bookstore and sit down with one of the analysis texts, and then read through it for a few hours. if you feel comfortable with it and don't experience any contempt for the author for introducing terms without explaining them, etc., then go ahead and get it. i would recommend finding something that holds your hand through the new fancy notation of proofs (for all, exists, epsilon, blah blah blah)...something that combines a basic intro to topology/set theory in its introduction
Rudin is deceptive. At UCSD it takes a full academic year to get through that book. It is damn hard. UCSD was recently ranked in the top 50 universities in the world so it is not some lousy lazy place.
If you're going to try to learn that stuff on your own you'll need other books to supplement it and provide examples, counter-examples, intuition, and alternative explanations.
If you're going to try to learn that stuff on your own you'll need other books to supplement it and provide examples, counter-examples, intuition, and alternative explanations.
It's an undergrad text so it can't assume a great deal of mathematical maturity, hence everything takes longer.
Walter Rudin's book is a masterpiece. I read this book 35 years ago for my comps, and remember it as one of the best. Bear in mind there's no easy way to learn mathematics - it's going to be hard no matter what approach you take because that's the nature of the subject. My advice to undergrads is to skip computing completely because it's much easier to learn when you need it (unlike mathematics). You'll build your first computer program long before you prove your first theorem.
I've had "The Princeton Companion to Mathematics" sitting in my Amazon wish list for quite a while now, more or less for this purpose - I think on a HN recommendation.
Any thoughts on this book?
Any thoughts on this book?
This is a great book to sit down and read a few articles from (at least if you're into reading encyclopedias ;)), but you aren't going to learn much "actual math" from it.
Most research mathematicians don't have a clue what people outside of their field work on, so this is a great book if only because it lets to you talk to the guy down the hall :-) Use it to figure out what interests you mathematically, and then drill down to various subjects with more in-depth texts.
I just got it, and it's huge. Bigger than most dictionaries.
The articles are great. It's a useful resource to look up something you hear a reference to but haven't seen before, or to get more cultural exposure around something you're studying in a narrow context. I recommend it, though you'll need another text for a thorough, linear introduction to a field.
The articles are great. It's a useful resource to look up something you hear a reference to but haven't seen before, or to get more cultural exposure around something you're studying in a narrow context. I recommend it, though you'll need another text for a thorough, linear introduction to a field.
I highly recommend this book for Real Analysis
http://www.amazon.com/Introduction-Analysis-Maxwell-Rosenlic...
We used it at the University of Texas at Austin for the first semester in Real Analysis. I found it very clear and easy to follow.
We used it at the University of Texas at Austin for the first semester in Real Analysis. I found it very clear and easy to follow.
I used this for my undergrad analysis class and then relied on it as a foundation for reading Rudin/Royden in grad school. Killed qualifying exams. Use it. Sure, it's all Riemann integration but you will not suffer until you get to measure-theoretic integration later.
Rosenlicht.
Linear Algbera done Right. Then Finite Dimensional Vector Spaces.
Halmos' Measure Theory. Awesome.
"Naive Set Theory" by Halmos is a true gem, and readable, too.
You will find course material from MIT OCW mathematics to be very useful:
http://ocw.mit.edu/courses/mathematics/
http://ocw.mit.edu/courses/mathematics/
iTunes U has some amazing lectures. Gilbert Strang (MIT) has some video lectures (MIT OCW) on linear algebra. They are a bit dull, but they are pretty decent.
I also recommend academicearth.org.
If you are near a university, take a class "mathematical physics". These kind of classes usually cover a lot of undergraduate material in a semester and are offered by many physics department. They usually use "Mathematical Methods" by Boas as a text.
I also recommend academicearth.org.
If you are near a university, take a class "mathematical physics". These kind of classes usually cover a lot of undergraduate material in a semester and are offered by many physics department. They usually use "Mathematical Methods" by Boas as a text.
Polya and Szego's "Problems and Theorems in Analysis I & II" are classics. If you can work through any part, you will have learned a lot.
How can nobody here mention Courant and John's Introduction to Calculus and Analysis? I think I'm going to cry.
Can anyone suggest a good probability or statistics book?
If you already know analysis (well), the best probability book is "Probability & Measure" by Billingsley. A gentler introduction is the SUMS book "Measure, Probability, and Integral."
If you don't care about measure theory and just want to learn how to calculate the probability that a coin comes up heads in the first five attempts, the book by Larsen and Marx is pretty good.
If you don't care about measure theory and just want to learn how to calculate the probability that a coin comes up heads in the first five attempts, the book by Larsen and Marx is pretty good.
About a year ago, I refreshed my memory of basic probability theory with Bertsekas & Tsitsiklis Introduction to Probability and really enjoyed their approach.
For stats:
Statistical Methods by Snedecor and Cochran
Statistical Data Analysis by Glen Cowan
As a gentle introduction to statistics and probability, Statistics by Freedman, Pisani, and Purves.
Statistical Methods by Snedecor and Cochran
Statistical Data Analysis by Glen Cowan
As a gentle introduction to statistics and probability, Statistics by Freedman, Pisani, and Purves.
I had "Mathematical statistics" by Freund and Walpole and still refers to it. Also Drake's Applied Probability theory is a classic.
Your
"I'm more than willing to spend an hour per page and do all the exercises, but I would like good exposition."
is essentially necessary and sufficient.
If an exercise takes more than two hours, then swallow your pride and skip the exercise (it may be misplaced, have an error, or just be way too difficult for effective education).
For linear algebra:
(1) Work through a few introductory texts.
(2) Work carefully through the long time, unchallenged, world-class classic,
Halmos, Finite Dimensional Vector Spaces.
and there note near the back his cute ergodic convergence theorem.
The glory here is the polar decomposition.
(3) Get some contact with some applications, including in elementary multi-variate statistics, numerical techniques, optimization, etc.
(4) Pick from
Horn and Johnson, Matrix Analysis.
and
Horn and Johnson, Topics in Matrix Analysis.
My first course was an "advanced course" from one of Horn, Johnson, and I knocked the socks off all the other students. How'd I do that? Brilliant? Worked hard? Learned a lot? Nope. Instead the key was just my independent work with (1) -- (3).
So if you do (1) -- (4), then you will be fine.
For analysis, (Baby Rudin)
Walter Rudin, Principles of Mathematical Analysis.
Note in the back that a function is Riemann integrable if and only if it is continuous everywhere except on a set of Lebesgue measure 0.
Also know cold that a uniform limit of continuous functions is continuous.
Royden, Real Analysis.
and the first, real, half of (Papa Rudin)
Rudin, Real and Complex Analysis.
Of course, emphasize the Radon-Nikodym theorem; I like the easy steps in Royden and Loeve (below), but see also the von Neumann proof in Papa Rudin.
For probability based on measure theory and the limit theorems,
Breiman, Probability.
Note his result on regular conditional probabilities.
Neveu, Mathematical Foundations of the Calculus of Probability.
If you can work all the Neveu exercises, then someone should buy you a La Tache 1961.
Loeve, Probability Theory.
Note the classic Sierpinski counterexample exercise on regular conditional probabilities (also in Halmos, Measure Theory),
Cover the Lindeberg-Feller version of the central limit theorem as well as simpler versions. Do the weak law of large numbers as an easy exercise. Cover the martingale convergence theorem (I like Breiman here) and use it to give the nicest proof of the strong law of large numbers. Cover the ergodic theorem (Garcia's proof) and its (astounding) application to Poincare recurrence. Cover the law of the iterated logarithm and its (astounding) application to the growth of Brownian motion.
Of course apply the Radon-Nikodym theorem and conditioning to sufficient statistics and note that order statistics are always sufficient. Show that sample mean and variance are sufficient for i.i.d. Gaussian samples and extend to the exponential family.
Give yourself an exercise: In Papa Rudin, just after the Radon-Nikodym theorem, note the Hahn decomposition and use it to give a quite general proof of the Neyman-Pearson lemma.
To appreciate the law of large numbers in statistics, read the classic Halmos paper on minimum variance, unbiased estimation.
For tools for research in statistics, might want to get going in stochastic processes. So, for elementary books, look for authors Karlin, Taylor, and Cinlar and touch on some applications, e.g., Wiener filtering and power spectral estimation. Note the axiomatic derivation of the Poisson process and the main convergence theorem in finite Markov chains (also a linear algebra result). Then for more, note again the relevant sections of Breiman and Loeve and then:
Karatzas and Shreve, Brownian Motion and Stochastic Calculus.
"I'm more than willing to spend an hour per page and do all the exercises, but I would like good exposition."
is essentially necessary and sufficient.
If an exercise takes more than two hours, then swallow your pride and skip the exercise (it may be misplaced, have an error, or just be way too difficult for effective education).
For linear algebra:
(1) Work through a few introductory texts.
(2) Work carefully through the long time, unchallenged, world-class classic,
Halmos, Finite Dimensional Vector Spaces.
and there note near the back his cute ergodic convergence theorem.
The glory here is the polar decomposition.
(3) Get some contact with some applications, including in elementary multi-variate statistics, numerical techniques, optimization, etc.
(4) Pick from
Horn and Johnson, Matrix Analysis.
and
Horn and Johnson, Topics in Matrix Analysis.
My first course was an "advanced course" from one of Horn, Johnson, and I knocked the socks off all the other students. How'd I do that? Brilliant? Worked hard? Learned a lot? Nope. Instead the key was just my independent work with (1) -- (3).
So if you do (1) -- (4), then you will be fine.
For analysis, (Baby Rudin)
Walter Rudin, Principles of Mathematical Analysis.
Note in the back that a function is Riemann integrable if and only if it is continuous everywhere except on a set of Lebesgue measure 0.
Also know cold that a uniform limit of continuous functions is continuous.
Royden, Real Analysis.
and the first, real, half of (Papa Rudin)
Rudin, Real and Complex Analysis.
Of course, emphasize the Radon-Nikodym theorem; I like the easy steps in Royden and Loeve (below), but see also the von Neumann proof in Papa Rudin.
For probability based on measure theory and the limit theorems,
Breiman, Probability.
Note his result on regular conditional probabilities.
Neveu, Mathematical Foundations of the Calculus of Probability.
If you can work all the Neveu exercises, then someone should buy you a La Tache 1961.
Loeve, Probability Theory.
Note the classic Sierpinski counterexample exercise on regular conditional probabilities (also in Halmos, Measure Theory),
Cover the Lindeberg-Feller version of the central limit theorem as well as simpler versions. Do the weak law of large numbers as an easy exercise. Cover the martingale convergence theorem (I like Breiman here) and use it to give the nicest proof of the strong law of large numbers. Cover the ergodic theorem (Garcia's proof) and its (astounding) application to Poincare recurrence. Cover the law of the iterated logarithm and its (astounding) application to the growth of Brownian motion.
Of course apply the Radon-Nikodym theorem and conditioning to sufficient statistics and note that order statistics are always sufficient. Show that sample mean and variance are sufficient for i.i.d. Gaussian samples and extend to the exponential family.
Give yourself an exercise: In Papa Rudin, just after the Radon-Nikodym theorem, note the Hahn decomposition and use it to give a quite general proof of the Neyman-Pearson lemma.
To appreciate the law of large numbers in statistics, read the classic Halmos paper on minimum variance, unbiased estimation.
For tools for research in statistics, might want to get going in stochastic processes. So, for elementary books, look for authors Karlin, Taylor, and Cinlar and touch on some applications, e.g., Wiener filtering and power spectral estimation. Note the axiomatic derivation of the Poisson process and the main convergence theorem in finite Markov chains (also a linear algebra result). Then for more, note again the relevant sections of Breiman and Loeve and then:
Karatzas and Shreve, Brownian Motion and Stochastic Calculus.