As a graduate student, I’ve looked at hundreds of research papers. Over the years, I realized the necessity to develop a system for myself to effectively keep track of, organize, and process – in a word, harness – all this knowledge. Further, I realized I needed to have a system that made it easy to quickly review many different papers and start structuring the topic in my mind. What are the important questions that have been answered? What labs or authors are the main players in this field? Which papers have I already reviewed and how thoroughly?
Using a systemized approach has helped me:
- cut down on extraneous reading (i.e., papers that I don’t end up citing/using)
- re-accessing the knowledge from those papers
- writing better papers that give a more structured view of the field (Intro/Background) and a more in-depth discussion of the results
- make the actual writing process more efficient because I can reference papers more easily (that are already in my system)
So, to get into my system.. The specific purpose of a literature review for a standard research paper falls into one of a few categories:
- Identify an important gap in the field (Introduction section)
- Check the originality of a research question (has it been answered before?)
- Motivate your research question (Introduction section)
- Provide a sufficient, accurate, and supportive research background for your research question (Background section)
- Verify something about your methods/experiment design (Methods section)
- Find papers to provide context for the discussion of your results (Discussion section)
All of these purposes require a thorough look at a paper’s research question, first and foremost, with how that hypothesis was tested following close behind. Nuances within these two aspects are often important, and the Methods section often contains these nuances. The results of a study must inform most literature reviews, but the results are not generally the reason why a particular paper is initially of interest or not.
The key here is targeted reading through a systemized approach where you cut way down on extraneous reading. The systematic approach that I take is as follows:
- Based on your objective(s) for your literature review (see above), identify topic of interest and 2-3 keywords.
- Search keywords on Google Scholar, PubMed, etc. (I just use Google Scholar, but your search engine may change depending on your field.)
- Find papers by looking at titles (primarily) and abstracts (maybe). Once you get a feel for the field, you will get better at understanding the purpose of the paper from the title, so you can get good at initially filtering papers based on just the title.
- Also keep an eye on a paper’s references and people who cited that paper (Google Scholar will show you this) for those paper’s you decide to take a deeper dive into.
- Try to find a review paper related to your topic (it can make your life easier)
- Download these papers you want to take a closer look at.
- If you just have 30 tabs open, it’s harder to see what is what, and you can’t highlight. You can always delete them later.
- Make sure it’s easy to identify downloaded papers; put the title as filename. Be sure to reference papers (e.g., (last name, year)) whenever and wherever they are used in written work, so backtracking is not required.
- Add the paper to the spreadsheet and fill in the fields as you go (template here). Highlight here and there in the paper. Use your discretion. They’re called highlights for a reason.*
- Quote the paper, but make sure you put quotes around that text in your notes. This way you’ll never wonder what parts are your words. Using quotes makes things much faster to process, and cuts down on unnecessary writing work for papers you don’t end up citing.
- For the initial pass, try not to do more than read abstract and the last paragraph of intro. Skim the methods and conclusion. You’re not writing a report on one paper right now. Just get the general idea. Assign the paper a main keyword (“Crutch Energy Balance”, “Force Control”, etc.) and categorize it in a folder structure based on that keyword. **
- Delete paper PDF if you discover it doesn’t fit, and update the spreadsheet to reflect this. Only keep good papers. It’s important to keep a record of what you looked that didn’t fit because you can save yourself time if you expand that literature review in six months.
- Don’t wait too long to start writing the Background. You do need a decent understanding of part of the puzzle, but you certainly don’t need to see the whole puzzle to start putting some pieces together.
- You may find it helpful to initially categorize the Background the same way you do the paper PDFs. You can always remove the labels later, but if you divide into sections, it removes some of the initial burden of making things flow well.
- Expect to cite between 30 and 50 papers for a typical journal paper. That means you’ll consider quite a few more papers than what you cite.
- Start with the work you know you need to review and include in the Background, and work out from there. You may find that the amount of room you have for the “maybe” topics will change after writing the background for the “must-have” topics.
*A note for beginners: Don’t feel bad when you find that reading research papers is a tedious and time consuming undertaking.
**A note on paper reading strategy: Often, the only things you need to read from a paper are the abstract and methods. The abstract is obviously important but why the methods? The methods provide what is perhaps the best insight into the heart of the paper: the research question, the hypothesis. Evaluating the methods will give you a good idea of the authors’ and the study’s credibility. More fundamentally, the methods are the primary place to identify things the authors, reviewers, and editors may have missed or that were significant but unknown to them at the time of publication. This is analogous to the relationship between premise, logical path, and conclusion in an argument. The methods will also contain the most precise question that is being answered. You will see for yourself what the research question is by understanding the methods. The authors may generalize results in other sections of the paper, which can make it difficult to get an exact understanding of the research question.