Eigenvectors and Eigenvalues (2015)(setosa.io)

Eigenvectors and Eigenvalues (2015)

http://setosa.io/ev/eigenvectors-and-eigenvalues/

86 コメント

3Blue1Brown has a good series on YouTube for building intuition in linear algebra:

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

In one of the last videos in the (relatively short) series, he discusses eigen-*:

~'eigen-stuffs are straight-forward but only make sense if you have a solid visual understanding of the pre-requisites (linear transformations, determinants, linear systems of equations, change of basis, etc.). Confusion about eigen-stuffs usually has more to do with a shaky foundation than the eigen-things themselves'

https://youtu.be/PFDu9oVAE-g

All of the videos in the series, including this later one on eigen-things, focus on animations to show what the number crunching is doing to the coordinate system.

AceJohnny28 年前

3Blue1Brown (Grant Sanderson) is really, really good. I follow a number of education channels on YouTube, and Grant blows them all out of the water for the kind of insights, new perspectives, and inspiration he provides. His animations are fantastically put together to clearly and unobtrusively illustrate the point he's making. I also really like his voice, soothing, clear and with enough intonation to avoid boredom, and perfect pace. I wish I had his linear algebra series back in college, I suspect I would have done much better.

He's the creator I support the most on Patreon: https://www.patreon.com/3blue1brown

picklesman8 年前

I’ve taken 3 college level courses on linear algebra and only really got an intuition for eigenvectors/eigenvalues when watching his video series. Super great!

emmelaich8 年前

I had problems with Principal Component Analysis; nice to see he's got one on that as well!

http://setosa.io/ev/principal-component-analysis/

(edit: ugh, contains an ad for MacKeeper)

jjoonathan8 年前

What, you don't want to remove your mac from trash?

Incidentally, that particular grammatical quirk is one I've seen a number of times. Is it a thing for people coming from Russian? Or just general difficulty with remembering the order of the two subjects in two-subject phrases?

bashcorrupt8 年前

>I follow a number of education channels on YouTube

Any recommendations? I would love to look into channels that you think are like 3Blue1Brown but in other subjects (natural sciences, history, art etc.).

AceJohnny28 年前

Edit: I realize I overlooked the "like 3Blue1Brown" prereq, instead sharing a list of the educational/interesting channels I find worthwhile. The most like 3Blue1Brown will be the PBS ones (especially SpaceTime), MinutePhysics, and Mathologer, for using diagrams to convey abstract concepts.

PBS Space Time and Eons are both awesome:

* PBS Space Time, covers cosmology and quantum physics:

https://www.youtube.com/channel/UC7_gcs09iThXybpVgjHZ_7g

* PBS Eons, for geology and paleontology:

https://www.youtube.com/channel/UCzR-rom72PHN9Zg7RML9EbA

* Smarter Every Day, Destin's enthusiasm is contagious:

https://www.youtube.com/user/destinws2

* Extra Credits various topics (video game design, History, and recently history of Sci-Fi) are great:

https://www.youtube.com/user/ExtraCreditz

* Today I Found Out is just on this side of clickbaity, and is this age's "Ripley's Believe It Or Not", but still interesting and more importantly well researched:

https://www.youtube.com/user/TodayIFoundOut

* Crash Course, of course:

https://www.youtube.com/channel/UCX6b17PVsYBQ0ip5gyeme-Q

* Gaming Historian, for some insight into the making of systems that formed my (and earlier) childhood:

https://www.youtube.com/channel/UCnbvPS_rXp4PC21PG2k1UVg

* Minute Physics, whose latest few videos made Special Relativity understandable to this peon:

https://www.youtube.com/user/minutephysics/videos

* Practical Engineering, for some insight into civil engineering topics that we take for granted:

https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ

* Real Engineering, for insight into various other mechanical engineering topics:

https://www.youtube.com/channel/UCR1IuLEqb6UEA_zQ81kwXfg

* Standup Maths, host Matt Parker was the first to make maths approachable for me again (before 3Blue1Brown took the lead):

https://www.youtube.com/channel/UCSju5G2aFaWMqn-_0YBtq5A

* Steve Mould, who covers various topics both mathematical and physical. You may have seen that gif of him demonstrating the "levitating" siphoning "pearl necklace" (also a friend of Matt Parker, above):

https://www.youtube.com/channel/UCEIwxahdLz7bap-VDs9h35A

* The 8-Bit Guy, for some history of early home computer systems:

https://www.youtube.com/channel/UC8uT9cgJorJPWu7ITLGo9Ww

* Numberphile, the second-greatest math channel (after 3Blue1Brown), whose recent video finally made me take the Golden Ratio seriously, rather than an architectural gimmick/conspiracy theory:

https://www.youtube.com/user/numberphile

* Mathologer, another good math channel (but I must sheepishly admit I prefer 3Blue1Brown... sensing a pattern here?):

https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg

* Periodic Videos, for chemistry and physics, often featuring the iconic Dr Martyn Poliakoff:

https://www.youtube.com/channel/UCtESv1e7ntJaLJYKIO1FoYw

* NileRed, for some homegrown chemistry, I particularly appreciate the candor of the approach and results:

https://www.youtube.com/user/TheRedNile

Not quite as much "educational", but still very very good:

* Every Frame A Painting, now finished, but a great explanation of what makes good cinematography:

https://www.youtube.com/channel/UCjFqcJQXGZ6T6sxyFB-5i6A

* NoClip, long-form documentaries about the making-of video games. Danny O'Dwyer is a treasure:

https://www.youtube.com/channel/UC0fDG3byEcMtbOqPMymDNbw

Apologies for the link spam, this list turned out longer than I expected as I went down my subscriptions, and I've probably missed a few worthy ones!

Edit the final: I discovered many of these channels through referrals from others I was watching, including from the twitter feeds of the authors. Turns out the educational landscape on YouTube is a well-connected graph!

swaggyBoatswain8 年前

I just realized I had 80% of these subscribed already, then I looked at how many channels I subscribed too (700+)

I like tinkering with things so I have some mechanical / electrical engineering / makers on my list as well.

I listed my favorites of that list here

* Tom Scott - Lots of interesting unique topics related to engineering, programming, and history

https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A

* Colin Furze - Lots of crazy welding projects

https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A

* Mark rober - ex-nasa engineer, similar to SmarterEveryDay

https://www.youtube.com/channel/UCY1kMZp36IQSyNx_9h4mpCg

* Hacksmith - just interesting project build videos normally related to marvel comics

https://www.youtube.com/user/MstrJames

* Live overflow - web / binary hacking with great case studies

https://www.youtube.com/channel/UClcE-kVhqyiHCcjYwcpfj9w

* Wendover productions - Great documenatires on things like Airplane and shippping logistics

https://www.youtube.com/wendoverproductions

* Strange parts - ex-programmer who sources components and parts overseas

https://www.youtube.com/channel/UCO8DQrSp5yEP937qNqTooOw

AceJohnny28 年前

700? Damn! And I have trouble tracking what to watch with my few dozen subscriptions!

Also, something rant about YouTube's recommendation engine, that doesn't mesh with my subscriptions into account well enough, offering many videos from channels I don't subscribe to and not enough from those I do subscribe to. Does it work better for you with your firehose of subscriptions?

swaggyBoatswain8 年前

Youtubes recommendations are terrible but occasionally I find some good things. Firehosing my videos subs this way has not made the experience any better

I tend to have a really bad habit of subbing things I dont really need too. I used subbing more as a way to "bookmark" channels I like. I do the same with github, I am that type of person that 700ish repos bookmarked

I do put alerts on channels I know won't constantly spam me with too many videos (e.g. smarter everyday, wendover productions, mark rober, tom scott, primitive technology). Usually i watch every video that gets released there

I dont put alerts on channels that are good quality but high spam (e.g. casey neistat, tested, vsauce, etc). Normally i just binge watch these in one setting.

I dont put alerts on any programming or shiny tech channels. Like tutorials on how to use react, etc. I go to youtube to get away from all of that and would naturally seek those videos out with search terms anyhow.

I put a like button on every video ive watched that doesnt suck and leave comments to timestamp videos. I put dislike on videos that are poorly explained or didntt work for me (e.g. these are usually how to install xyz videos). This way i can instantly know if I've seen a video before ans found it useful or had to debug something years later (IT type stuff) to find it didnt work out

If i liked a video i check the top comment if its mine.

Basically I use google and youtube as my own personal search repository. Its like forking a github repo and adding annotations. I treat it the same way. If im going to 9 times out of 10 google something in the end, i might as well make the process easier to do next time.

Im still looking for a better UX youtube experience that doesnt suck. Havent found it yet.

How i would like to have it is organized by the following.

1. High quality videos low spam first uptop in feed.

2. Everything else sorted by its video category type and or playlist

3. General recommendations from youtube

emmelaich8 年前

Awesome but no Vsauce?

https://www.youtube.com/channel/UC6nSFpj9HTCZ5t-N3Rm3-HA

AceJohnny28 年前

Y'know, I hesitated. I admit I'm somewhat ambivalent about VSauce, because Michael's somewhat wandering style makes it closer to entertainment than the usually more focused topics the other channels on my list work with. That said, Vsauce is definitely interesting and great for fostering curiosity and an inquisitive mindset.

Also his new channel DONG ("Do Online Now Guys"... wut?):

https://www.youtube.com/channel/UClq42foiSgl7sSpLupnugGA

Also I should've mentioned Veritasium:

https://www.youtube.com/channel/UCHnyfMqiRRG1u-2MsSQLbXA

chaotic_clanger8 年前

seems that pbs has a new one on Noethers theorem: https://www.youtube.com/watch?v=04ERSb06dOg

bashcorrupt8 年前

Thank you! Introduces me to a whole new universe out there!

smrq8 年前

Just to throw in my own anecdote, I took linear algebra twice (in high school with no college credit, and in college) and I still couldn't ever remember afterwards what an eigenvector was until I watched that series. Now I'll probably never forget. He is an astonishing educator.

bladecatcher8 年前

It’s possible to understand eigen-* without having an understanding of determinants. That’s how they’re introduced in “Linear Algebra Done Right” - http://linear.axler.net/

tgb8 年前

And they exist in situations where determinants are difficult or impossible to define! Infinite dimensional vector spaces can still have transformations with eigenvectors but you generally can't define a determinant coherently for them (certainly they might have infinitely many eigenvalues and if the determinant is the product, then you have convergence problems). A classic example is that the standard Gaussian distribution is an eigenvector of the Fourier transform.

in98 年前

You just blew my mind with this comment. I'm a statistics undergrad student and I really need to up my math knowledge. Where is this fact from? Functional analysis?

tgb8 年前

Yeah that would be a good course to learn it - functional analysis is basically linear algebra on infinite dimensional vector spaces. (Though that sells it a little short - moving to infinite dimensional vector spaces requires incorporating some topology too.)

swaggyBoatswain8 年前

I just finished watching both his calculus series and linear algebra. I have to say 3blue1brown has made a mathematical masterpiece of a youtube series

Neither of those subjects really clicked with me until I could visualize it in a 2D / 3D representation

xevb3k8 年前

Whenever this kind of stuff comes up I feel like a bit of a fraud...

I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc.

For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in handy. Maybe it’s because most of what I encounter is novel datasets where there’s no established method?

I reasonably regularly pick up new discrete methods, but the numerical stuff never seems super useful...

I don’t know, just a confession I guess... it never comes up on interviews either for what it’s worth.

soVeryTired8 年前

For a large fraction of probability theory, you only need two main facts from linear algebra.

First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors.

Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform.

That more or less covers covariances, PCA, and change of variables. Whenever I try to understand or re-derive a fact in probability, I almost always end up back at one or the other fact.

They're also useful in multivariate calculus, which is really just stitched-together linear algebra.

sakuronto8 年前

Not a very useful addition but hypercube is to cube as parallelotope is to parallelepiped.

zawerf8 年前

I use the 2nd point a lot for debugging 3d transforms. To expand upon it, for example in three dimensions the three axes are:

(1, 0, 0)

(0, 1, 0)

(0, 0, 1)

To find out where those axes are after a 3x3 matrix transform, you just read off the first, second, and third columns of the matrix respectively. Then you can mentally visualize another unit cube in the new coordinate system using those three vectors as the edges of the cube.

Really basic change-of-basis stuff but academic lectures don't emphasize how useful it is to be able to look at a matrix of numbers immediately know what it does.

aidos8 年前

This concept totally changed my intuitive understanding of matrices. Beautifully illustrated in the below 3blue1brown video.

https://youtu.be/kYB8IZa5AuE?t=3m15s

qmalzp8 年前

I think the first point is only true for symmetric matrices (which includes those that show up in multivariable calc). In general, the eigenvectors need not be orthogonal.

soVeryTired8 年前

Yep, you could well be right. The image of an ellipse under a linear transform is definitely an ellipse, but I'm not sure about the eigenvectors in the general case.

The symmetric case is by far the most relevant for probability theory though.

tprice78 年前

In general it's the eigenvectors of the positive-semidefinite (hence symmetric) part of the left polar decomposition.

jdonaldson8 年前

Eigenvectors and Eigenvalues show up everywhere, although sometimes it's in the form of an iterative estimate (PageRank is basically the power method estimation of the first eigenvector of a connected graph of web pages).

They're in the same class as logarithms and Fourier transforms IMHO. You won't need to calculate them by hand, but you should know what they do and why they're important.

madengr8 年前

I use logarithms and FFTs daily, but got a C in Linear Algebra, after getting A in all other EE/Math/Phys courses. It kicked my ass once we got to proofs.

xevb3k8 年前

Interesting, perhaps that’s it... I’ve used FFTs a few times, but even FFTs have never been make or break in terms of getting a pipeline or analysis working well.

cowboysauce8 年前

The Fourier transform is actually linked with linear algebra. You can think of it as taking a vector in an infinite dimensional Hilbert space (your signal) and decomposing it into its components (the amplitudes of the frequencies).

stephencanon8 年前

The Fourier transform as used in practice in signal processing is ~always discrete, hence finite-dimensional. It's literally "just" a rotation with an especially nice decomposition that allows efficient multiplication.

danharaj8 年前

A particular fft in practice may be discrete but the relation between fft's of different resolutions of the "same" signal hints at the infinite structure which bundles up all the finite subspaces into one conceptual object.

WhompingWindows8 年前

Why should you know what they do and why they're important? How does that practically change my R code?

angersock8 年前

Because, say, knowing about Fourier transforms can help you write more efficient filtering or open up new ways to view your data--perhaps there's a really interesting behavior in the frequency domain you'd miss otherwise.

If you just want to be a statistical script kiddie you do you. :)

ItsMe0000018 年前

When working with real world data almost everything is more important than being able to use the most abstract methods "to extract the last bit of data". It's often extremely fuzzy to begin with, the collection process to what it represents, for me, while I love math and see it as the "magical language" in a magical world, I find common sense and a certain kind of work ethics go soooo much further than any math Ph.D. I (with a CS degree) wanted to start another study (over a decade later, thirsting for new theory and new knowledge) and chose math - but that was around the same time edX and Coursera etc. got up and running with loads of courses, and I ended up ditching the additional math degree for loads and loads of courses in fields like medicine and biology, deciding to go for breadth instead. Of course, that's purely personal and even as an anecdote not worth much to anyone. It's just that I too got disillusioned with (higher and higher) math as an helpful tool in practical life. Of course I still see the benefits in many fields, but I think a surprisingly low number of very good specialists may be all we need. The rest of us can just ask them when we actually do need something.

Right now I'm taking a "math. modeling" course. Still, the only use case I ever found was... other courses! I already modeled a little bit in a biology course. Sure, in real life I could model this or that, but the truth is that a very rough estimate guided by experience and "feeling" has always been enough. There are too many variables that cannot be accurately measured, so going for a nice model is kind of useless.

For example, I was just asked today about the performance of the crypto-hash-connected data storage and exchange library I wrote. Now that sounds like something I could model! Only experience tells me that's useless. The only worthwhile answer is to set up a concrete scenario, with a concrete app using it, concrete network and concrete systems, and test it. Could be anything from smartphones to well-connected servers. Sure I could create a sophisticated model and simulation - and it would be useless.

Maybe I'm just a bit, or more than just a bit, disappointed that all the considerable amount of math I learned in my life didn't seem to be of nearly as much use as I would have hoped. I'm also frustrated each time such a topic comes up and everyone is so excited about how great it is, and I always feel like I'm missing something despite trying hard, like the color blind guy looking at paintings. I mean the usefulness to me, not understanding it.

pphysch8 年前

As a younger person (finishing up a Math BS) this resonates with my perspective.

IMHO, it comes down to individual beliefs about mathematical realism. Is there anything inherently real about math, or is it just a man-made, arbitrary set of cognitive tools? Is it valid to presume the existence of a Grand Mathematical Framework that can solve any problem a priori? Or, is every problem unique and independent of mathematical developments?

From the little I've read about Math history, it seems pretty clear that the Problems came first, and the Mathematics followed. Infintesimal calculus, game theory, etc. were mathematical ideas developed primarily to solve real problems. Then 20th century formalism came along and rebranded much of mathematics under a "clean" framework, while giving little attention to the human environment in which much of it was developed.

To me, it is a great shame that abstract mathematical concepts are made further abstract (e.g. in math education) by distancing them from their human roots. Instead of forcing oneself to understand this mathematical "new testament", I think it's far more productive to adopt this sort of irreverent attitude towards math as you describe.

Einstein:

>"As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality."

hyperpallium8 年前

Reminds me of economic rationalism: model makes sense, and conclusions seem logical and compelling - but you can't tell if the model represents reality or even if crucial factors have been omitted.

aje4038 年前

Perhaps colleges can start adding an "applied" math major with a focus on subjects that more directly involve the human environment more directly to alleviate the problem you're describing?

akvadrako8 年前

Applied math is definitely a major. Do you mean something else?

hyperpallium8 年前

I'm doing some fluid simulation (CFD), and the actual code for finite differences is simple. But the analysis (of stability etc) is more mathy, and I don't feel confident reading other needed papers, because they are couched in math.

I mean, I can and have coded it, but can't be sure how it how it will behave in all situations.

So I can understand these papers, I'm going back to study math properly. I'm not fully convinced it's really needed (though how could I tell?), but I'm fully convinced it's needed to understand the papers.

Math is the latin of CFD.

bigger_cheese8 年前

I work in Engineering and feel somewhat similar about CFD. There's always some element of doubt lurking in the back of my mind "is it really correct in all circumstances".

The most useful info I learnt at university were a couple of equations: Bernoulli's (for general observation about expected pressure drop), Ergun's (for flow through packed beds) and the general laws of thermodynamics.

Those are mostly enough to be able to sketch out an intuitive 'guess' about expected behavior in a large range of systems and the underlying math is not particularly demanding.

hyperpallium8 年前

Have you seen the Method of Manufactured Solutions (MMS)?

You guess ("manufacture") a solution to the PDE. Then you plug it into the PDE. It won't be correct, so you just add source terms to make it correct. You now have an analytical solution for this PDE with those source terms. You can now run your simulation (with those source terms) and compare it with the "correct" solution.

Unfortunately, you'll introduce discretization error, because your delta x and delta t aren't infinitessimal. But using the order of accuracy of your discretization method, you know how the error should change with delta x and delta t. So you graph error against delta x, on log-log, and see if the curve matches the order of accuracy. Apparently, it shows up even minor bugs in your code really well.

This approach scares me a little, because how do I know my math is correct? I need better appreciation of how the order of accuracy interacts over time as well as space; the behaviour of error summaries; and how log-log plotting works.

xevb3k8 年前

Really interesting, thanks! Yes, it’s certainly been my experience that adding more domain knowledge has helped me a huge amount.

Jack0008 年前

well you don't need this stuff until you do - some things still need definite, analytical performance guarantees.

I'd be pretty nervous riding an airplane that didn't use modern control theory, or going over a bridge that didn't use FEA - or an self-driving car that ran on a raspberry pi instead of a RTOS...

ItsMe0000018 年前

Why do you think in absolutes? I'm tempted to cite a certain Dilbert... but that would make me appear incredibly rude, which I really don't want to be. Of course it is easy to find examples where it's needed, and it is easy to see so too. So? Did I propose at any point "Nobody needs higher math for anything" or something similar? I don't see a need to argue about an argument never made.

Bukhmanizer8 年前

It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just by knowing how to do the same thing in a more efficient way.

My feeling is having a basic knowledge of testing/caching/memory management is way more useful when you're doing large image analysis.

xevb3k8 年前

Interesting, well I’ll try to keep reviewing this stuff and hoping I find an application.

I really would like to find an application in my work, because without that I find new techniques don’t really stick and after a few months I forget them...

gh02t8 年前

It depends on the field you're in. For example, if you're in an area that heavily uses differential equations (many engineering disciplines) then you're probably gonna be using eigenvectors a lot, as they are important for solving a lot of problems. Other areas may not need them at all. It also depends on your depth in the field. A rank and file engineer may not need to know anything about them - they underpin a lot of numerical methods, but get hidden away in software packages. Someone developing those software packages likely will, though. Techniques based on eigenvectors and eigenvalues are extremely important in my field (nuclear engineering... you've probably heard the term "critical", that refers to an eigenvalue), but I know someone who is an excellent civil engineer and knows next to nothing about them (or linear algebra in general) because they aren't that important for what he works on.

Forgetting stuff you don't use is pretty normal, the important thing is to be able to recognize when a technique you don't remember the details of might be applicable, and to know where to look to refresh your memory.

Bukhmanizer8 年前

I'm the exact same way, my job is really heavy in linear algebra, so it sticks more easily for me.

Usually I go through the code and ask "what am I trying to do here" and "can I do this a better way". A lot of the aforementioned speedups have come because the previous developer was obviously trying to do something, like create a linear projector, but were following some sort of math formula, so made a bunch of extraneous matrices that were huge.

It's a simple fix, but adds up when you're dealing with massive datasets.

electricslpnsld8 年前

It comes up all the time when trying to build second order optimization methods. With the eigensystem of your objective in hand you have a complete understanding of the (non-) convexity of your energy landscape, which is useful to ensure you always have good search directions, etc.

tzahola8 年前

What kind of novel dataset are we talking about?

Just from the top of my head, you could have encountered eigenvectors/eigenvalues:

- if you ever used spectral graph algorithms

- if you ever done dimensionality reduction via principal component analysis

- if you ever calculated the steady state distribution of a Markov chain

xevb3k8 年前

Image analysis and signal corrections on time series data. Extracting features from microscope images for example, correcting for signal convolutions.

wglb8 年前

It is certainly frequent in engineering. If you need to analyze the stability of an electrical grid, there isn't much alternative.

A fun book on this is https://openlibrary.org/books/OL2398351M/The_algebraic_eigen...

rxhernandez8 年前

Just my anecdotal experience, I've seen it come up in interviews for an algorithms group at a medium sized biomed company.

aje4038 年前

This is frightening but believable. I've worked with a few "quants" who stared at me doe eyed explaining eigen* and basic calculus concepts to them in the context of why their calculations don't add up. You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.

ajross8 年前

> You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.

That's a bit uncharitable. A fourier decomposition can absolutely be understood as an explicit bag of calculus tricks, with no loss of precision or generality. And an awful lot can be done with just those tools -- you don't need to explain JPEG compression or VLBI astronomy in terms of eigenvectors, for example.

Obviously (heh, "obviously") it's true that the space of decomposed functions form an orthogonal basis, so technically we're "really" operating in a linear space and that has expressive power too. But there are lots of ways of looking at problems.

To wit, you're not wrong. You're just... Well, you know.

aje4038 年前

Dude your post is orthogonal

xevb3k8 年前

In the context I used it, FFT wasn’t really of fundamental importance. Using FFT (and in particular FFTW) gave a performance improvement (execution speed), but no real advantage in terms of accuracy over an alternative naive method...

So... yes I guess I’ve just not seen anywhere in my work where this stuff has proved useful...

aje4038 年前

It's pretty much ubiquitous in any quantitative field... would not even know where to start

lewis5008 年前

Interesting to see this back on the front page after three years. Still remember us sitting in our living room drawing this on paper and arguing about the right approaches.

Maybe one day vicapow and I will make a triumphant return to the explorables space, but life has a way of getting in the way as you get older.

dang8 年前

That was https://news.ycombinator.com/item?id=8918259.

sannee8 年前

Eigen{vectors,values} seemed like this totally arbitrary concept when I first learned about them. Later it turned out that they are actually really awesome and pop up all the time.

Multivariable function extrema? Just look at the eigenvalues of the hessian. Jacobi method convergence? Eigenvalues of the update matrix. RNN gradient explosion? Of course, eigenvalues.

danlugo928 年前

I highly recommend 1Blue1Brown's Essence of Linear Algebra series[0] to highly grasp and comprehend linear algebra.

[0] https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

olskool8 年前

This is truly a favorite of mine.

toppy8 年前

Classic paper on Google's PageRank: "The $25,000,000,000 eigenvector"

https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.p...

WhompingWindows8 年前

Am I the only one with [Math Processing Error] all over this source? Ctrl+F gives me 57 instances of that string

akvadrako8 年前

I see that on some pages. Try reloading.

abiox8 年前

i didn't see anything like that. possibly a mathjax.org hiccup?

asafira8 年前

Quantum mechanics should be listed as another reason to learn about eigenvectors and eigenvalues! =)

deepdiving128 年前

Beautiful demos/explanations. Would've been really handy back in school.

lr4444lr8 年前

Great interactive demos.

KasianFranks8 年前

There's nothing to see here.

abiox8 年前

what do you mean? this seems like a strange statement, given the context.

haskellandchill8 年前

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to grasp these concepts.

whatshisface8 年前

As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have an eigenset that was also along the x axis, consisting of the points that were moved straight up or down.

To imagine finding the eigenset, just ask, could I draw a line through 0,0 such that any point I put on it would stay on it after the matrix acted?

msla8 年前

> So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have an eigenset that was also along the x axis, consisting of the points that were moved straight up or down.

Maybe I'm being pedantic, but rotation matrices have eigenvectors and eigenvalues, but the eigenvalues are imaginary because imaginary numbers are rotations in the complex plane.

It's exactly like saying x^2 = -1 has no solutions: It has two solutions, like any other quadratic, but neither of them are real.

In three dimensions, rotation matrices have three eigenvalues, one of them being 1, and the eigenvector corresponding to that eigenvalue is, naturally, the rotation axis.

https://en.wikipedia.org/wiki/Rotation_matrix

electricslpnsld8 年前

> So a rotation would have no eigenvectors

Rotations have eigenvectors: a 2D rotation has two complex eigenvectors, a 3D rotation has one real and two complex eigenvectors, ...

whatshisface8 年前

That's a fair and true catch, but I can cover myself by pointing out that the article was only talking about matrices in R^(m x n). ;)

sakuronto8 年前

When your field of interest is the reals, those complex eigenvectors don't matter.

haskellandchill8 年前

I want to know it. I have to manipulate the mathematical objects computationally using proof techniques to know it. That just takes time. Thanks though.

pizza8 年前

Does this gif help? https://commons.wikimedia.org/wiki/File:Eigenvectors.gif

haskellandchill8 年前

No gif will help. There is no visual explanation that will help was my point.

Retra8 年前

Surely visual information helps with geometric problems? Our geometric intuition basically evolved to predict the relevance of light patterns on our retinas, and geometry is a language designed to encode these intuitions, so there's no reason to think visual tools presenting geometric facts would inherently fall short.

haskellandchill8 年前

And yet I still think that. What's up? I'm broken. Good to know.