Ask HN: How do you read an academic paper?
In my new course I'm provided academic papers to read, as well as in my job. I don't have any background in how to read research papers as a software engineer. What are some tips you can recommend for a developer when reading a paper?
14 comments
Former graduate student here. The most important thing to remember is that every paper has a main point, and everything in it is reinforcing that main point. Figure out what the point is. The whole story is in the figures.
Start with the title. Then look at each of the figures carefully. Figure out the limits of the paper from the methods. Certain methods can only give certain types of results. Again, look at the figures. The figures tell the whole story and are worth way more than the text. What is the story the figures tell?
Ignore the rest of the text. If you must read something, read the text of the results and look at the supplement. If you have a lot of spare time, skim the citations and identify the couple of sentences in the discussion that express the author’s methodological regrets. Under no circumstances should you read an introduction.
Start with the title. Then look at each of the figures carefully. Figure out the limits of the paper from the methods. Certain methods can only give certain types of results. Again, look at the figures. The figures tell the whole story and are worth way more than the text. What is the story the figures tell?
Ignore the rest of the text. If you must read something, read the text of the results and look at the supplement. If you have a lot of spare time, skim the citations and identify the couple of sentences in the discussion that express the author’s methodological regrets. Under no circumstances should you read an introduction.
That's a great take on introduction! Everybody writes the introduction to convince the reader how it's the most impactful work, while obviously most of them are not.
Best to understand the images, read the discussion/conclusion and only dig into details if you find the prior reading interesting.
Best to understand the images, read the discussion/conclusion and only dig into details if you find the prior reading interesting.
Generally it depends what you need to get from it. If it's a standard research paper with introduction, methodology, results, and discussion you can use the introduction to gauge your understanding of the essential background and then check the findings.
Discussion will usually tell you where the researchers see issues in their methodology and findings, as well as suggestions on where to go from there. Reading the methodology may be extraneous to you if you trust the researchers and results. It's good for attempting to reproduce or to critique findings, but you'll ideally get a sense of the methods from the other sections.
On the other hand, if you're reading Audre Lorde you'll have to break out a highlighter and gut it out.
Discussion will usually tell you where the researchers see issues in their methodology and findings, as well as suggestions on where to go from there. Reading the methodology may be extraneous to you if you trust the researchers and results. It's good for attempting to reproduce or to critique findings, but you'll ideally get a sense of the methods from the other sections.
On the other hand, if you're reading Audre Lorde you'll have to break out a highlighter and gut it out.
Depends on the type of paper.
If it's in the model of a STEM style paper, then read abstract, read intro + conclusion, see if findings are relevant or interesting or significant. If they are, read the whole paper while recreating the logic/experiment/problem in your head. Question, question, question. Actively read and constantly question the findings or nature of the experiment.
If it's in the model of a humanities or argumentative paper, then read the abstract with a focus on the central thesis or argument, it should be clearly stated. If it's not, then the paper might be poorly written, or too vague to be very useful. If it is, then judge if it's relevant or interesting or significant. If it is, read the intro section. Decide if you should continue reading. When you read the rest of the paper, always refer back to the argument and thesis in your head (which should be clearly stated and summarized in the intro section). Actively question the logic and steps of the argument, evidence, analysis. Assume they will address counterarguments; if they don't, be savage and see if pulling on the thread of a counterargument undoes the whole logic of the paper. Take notes but don't just copy, note your reactions to the paper.
If it's in the model of a STEM style paper, then read abstract, read intro + conclusion, see if findings are relevant or interesting or significant. If they are, read the whole paper while recreating the logic/experiment/problem in your head. Question, question, question. Actively read and constantly question the findings or nature of the experiment.
If it's in the model of a humanities or argumentative paper, then read the abstract with a focus on the central thesis or argument, it should be clearly stated. If it's not, then the paper might be poorly written, or too vague to be very useful. If it is, then judge if it's relevant or interesting or significant. If it is, read the intro section. Decide if you should continue reading. When you read the rest of the paper, always refer back to the argument and thesis in your head (which should be clearly stated and summarized in the intro section). Actively question the logic and steps of the argument, evidence, analysis. Assume they will address counterarguments; if they don't, be savage and see if pulling on the thread of a counterargument undoes the whole logic of the paper. Take notes but don't just copy, note your reactions to the paper.
The papers are all based in cryptography and zero-knowledge proofs specifically. I would argue they are in the first category you described. They seem to be descriptive and walk through a problem but there are no experiments or anything, just theorems and proofs. I get lost pretty easily
Most papers are divided into roughly 5 sections: what we know, what we wondered, what we tried, what we found, and what it means, followed by a set of citations. You should look through these carefully to figure out what the foundational papers in your field are.
Some exceptions to this are literature reviews, which try to summarize all recent progress in a field and document the state of the art, and problem statements, which suggest directions for new research by pointing out a new or overlooked issue, and which are typically a prequel to grant applications.
You need to figure out whether the goal of people handing you papers is for you to become expert in the topic by fully understanding the methods and results, or to just trust that the papers you receive are the state of the art in this field and think about how to implement their conclusions in software, perhaps translating the key mathematical assertions or pseudocode algorithms into working prototypes.
Get used to looking up the paper online (the doi:// link on most papers will take you to the journal of original publication), because you can also check what papers have subsequently cited the work, and download the Supplementary Information, which usually exists as a 2nd pdf that goes into greater detail about methodology, intended to help a lab assistant or non-expert (ie you) reproduce the results. They are typically written in a more casual style and often include links to code or datasets, as well as practical information about what equipment was used or pitfalls to be avoided. The SI can often help you make sense of an otherwise confusing paper.
Remember, papers are not tutorials. They're checklists intended to back up an argument for reconsidering the conventional wisdom in a field. They may be trying to add detail, or show that one piece of knowledge sheds light on another topic in an unexpected way, resolve (or at least make tractable) some knotty problem that has caused progress in some direction to come to a halt, or (occasionally) to suggest that the conventional wisdom is actually wrong and some new concept explains prior results more economically or completely. Papers that trumpet their own importance usually aren't; papers that express surprise often are.
Some exceptions to this are literature reviews, which try to summarize all recent progress in a field and document the state of the art, and problem statements, which suggest directions for new research by pointing out a new or overlooked issue, and which are typically a prequel to grant applications.
You need to figure out whether the goal of people handing you papers is for you to become expert in the topic by fully understanding the methods and results, or to just trust that the papers you receive are the state of the art in this field and think about how to implement their conclusions in software, perhaps translating the key mathematical assertions or pseudocode algorithms into working prototypes.
Get used to looking up the paper online (the doi:// link on most papers will take you to the journal of original publication), because you can also check what papers have subsequently cited the work, and download the Supplementary Information, which usually exists as a 2nd pdf that goes into greater detail about methodology, intended to help a lab assistant or non-expert (ie you) reproduce the results. They are typically written in a more casual style and often include links to code or datasets, as well as practical information about what equipment was used or pitfalls to be avoided. The SI can often help you make sense of an otherwise confusing paper.
Remember, papers are not tutorials. They're checklists intended to back up an argument for reconsidering the conventional wisdom in a field. They may be trying to add detail, or show that one piece of knowledge sheds light on another topic in an unexpected way, resolve (or at least make tractable) some knotty problem that has caused progress in some direction to come to a halt, or (occasionally) to suggest that the conventional wisdom is actually wrong and some new concept explains prior results more economically or completely. Papers that trumpet their own importance usually aren't; papers that express surprise often are.
Thanks, this is really helpful!
I think I'm on the "just trust that the papers you receive are the state of the art in this field and think about how to implement their conclusions in software" path. I actually prefer that, because I'm not really much of an academic. I would prefer to focus on the code and implementation details versus the theory stuff.
Distilling it down from a really abstract level to something to implement is hard though. To be fair, I'm just starting out with this so it should get easier from here I suppose.
I think I'm on the "just trust that the papers you receive are the state of the art in this field and think about how to implement their conclusions in software" path. I actually prefer that, because I'm not really much of an academic. I would prefer to focus on the code and implementation details versus the theory stuff.
Distilling it down from a really abstract level to something to implement is hard though. To be fair, I'm just starting out with this so it should get easier from here I suppose.
In my experience, academic papers are written by academics to be read by other academics. Depending on the state of the field, there often is a huge gap between the theory and practice.
Unless you are already an expert in your field, you may need to read many of the cited papers as well in order to know what paths have already been taken and why the present paper improves upon them. Unfortunately negative results are rarely published so there is an ever present danger of going down an obvious path which turns out to be a dead-end.
Unless you are already an expert in your field, you may need to read many of the cited papers as well in order to know what paths have already been taken and why the present paper improves upon them. Unfortunately negative results are rarely published so there is an ever present danger of going down an obvious path which turns out to be a dead-end.
My usual first steps are to read the title, the abstract, and the figures/tables, in that order. Pause there and mentally summarize what I just read. If there are terms I didn't know or I'm not familiar with the topic, then I go through the introduction parts of the paper. After that I read the discussion sections and conclusions. I usually skip the methods sections and read those last only if needed.
All during that I try to keep track of questions I have or claims the authors are making. Once I've gone through a lot of the paper, I go back and look for answers to my questions. Interrogate the paper until you are convinced.
I remember being younger and getting stressed about reading papers. So I want to add that not all paper are well written. Don't worry if you're struggling. It's not easy in the beginning and the papers themselves aren't perfect. Keep working at it because it does get easier!
All during that I try to keep track of questions I have or claims the authors are making. Once I've gone through a lot of the paper, I go back and look for answers to my questions. Interrogate the paper until you are convinced.
I remember being younger and getting stressed about reading papers. So I want to add that not all paper are well written. Don't worry if you're struggling. It's not easy in the beginning and the papers themselves aren't perfect. Keep working at it because it does get easier!
I usually skim over it to see if I can make sense of it at a high level. If there are a lot of terms I don't understand I'll try to clarify those first.
If there are lots of references I'll try to figure out if one of the references is an "important" or highly referenced by other authors, then I'll get that paper and see what the foundational work in the field is.
All this assumes that you have time to really get into the material, it could take several weeks to get a sense of a single paper if I'm new to the field.
If there are lots of references I'll try to figure out if one of the references is an "important" or highly referenced by other authors, then I'll get that paper and see what the foundational work in the field is.
All this assumes that you have time to really get into the material, it could take several weeks to get a sense of a single paper if I'm new to the field.
You're a college student with a college student's job. Your ability to
read research papers has absolute no bearing on school or work. It only
matters in graduate school.
I assume you mean “in graduate school on”. Having worked in scientific modelling I read a hell of a lot of field specific papers when implementing tools. They were the spec sheets for the work, containing the mathematical model, it’s validation or application.
For me, paper with a hiliter and a pen or the digital equivalents worked. I ended up with a marked up document that made the fine details easy to find and had text that could be used in documentation.
For me, paper with a hiliter and a pen or the digital equivalents worked. I ended up with a marked up document that made the fine details easy to find and had text that could be used in documentation.
I would agree. I'm actually quite a few years out of school though, I have around 5 years of experience. I do have a bit of graduate-level experience but basically never did any real kind of research. That's why I think I'm struggling lol.
Here was a short guide on a disciplined three pass method. Depending on your needs you can vary the amount of effort and time invested.
https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape...
https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape...