As a standard practice, many literature reviews exclude ‘gray literature,’ a category that describes research and literature published outside of the traditional academic publishing industry. However, completely overlooking gray literature results in a wide array of valuable and excellent research being excluded from the overall body of scientific knowledge. A thorough understanding of what gray literature is and the ideal circumstances and cautions for using it could help you uncover hidden evidence that greatly improves your research.
What’s Considered Gray Literature?
In the broadest sense, gray literature comprises any material that isn’t published through a traditional academic publisher (e.g., published through a journal). Importantly, this typically means that the piece hasn’t undergone the traditional peer review process. These pieces may report on a research study, an individual’s opinion, an event, a stakeholder or advisory board discussion, an organizational policy, and more. They’re usually published online only. Some common examples of gray literature include governmental or industry resorts/white papers, graduate dissertations, conference proceedings, newsletters or mail-outs, policy documents, and blog posts.
When Should Gray Literature Be Used?
Early during the information synthesis process for a research project, it’s a good idea to complete at least a cursory review the gray literature. By searching through gray literature, you could find niche or null-results studies that may influence your research direction—for example, for a hypothesis-driven research study, you may find a report from researchers who attempted to answer the same or similar question but ran into unexpected hurdles or null results. Often, null-result studies aren’t accepted published traditional academic journals, but this type of crucial information could affect how you choose to proceed.
In many cases, gray literature expresses findings and opinions that are more indicative of the ‘real world’ than the often contrived or carefully constructed scenarios used in academic research. Additionally, gray literature often includes a more diverse range of authors who may often be excluded from the profit-driven traditional publishing industry.
However, there are some times when gray literature isn’t appropriate to include. Gray literature often isn’t used in highly regimented systematic reviews with strict inclusion/exclusion criteria. Fast-paced or urgent projects that must be published quickly may be hampered by the time required to sort through the large pool of gray literature available online.
Cautions of Using Gray Literature
It’s absolutely crucial to critically evaluate all gray literature that contributes to your research. While you should never fully depend upon peer review and blindly trust the reliability and rigor of journal-published research, it’s especially important to note that gray literature undergoes no such review. Some gray literature sources (such as governmental reports or conference proceedings) may be less inclined to bias and misinformation than others (such as blog posts), but all sources should be critically reviewed. Consider using medical librarian Jess Tyndall’s AACODS checklist for evaluating gray literature, which includes:
- Authority. Who wrote the piece? Do they have the expected credentials or experience to speak knowledgably on the topic?
- Accuracy. Does this piece have adequately rigorous methodology, evidence, or data to support its claims? Are their sources properly cited?
- Coverage. Does the piece outline its limitations or the authors’ biases/conflicts of interest? Is the scope of the piece clearly outlined?
- Objectivity. Do the authors or the organization have any explicit bias, such as financial interest in promoting specific results or opinions? Are counterarguments or conflicting evidence/perspectives presented?
- Date. How long ago was the piece published? Have new advancements or discoveries been made that might disprove, support, or otherwise affect the information in the piece?
- Significance. Does the piece include enough important, feasible, and relevant information that enriches your own research to justify its inclusion in your citation list?
Additionally, it should be noted that including gray literature in your evidence synthesis will greatly expand the scope of your search. Plan to budget in extra time to evaluate the relevance of sources as well as their credibility.
Tips for Finding Gray Literature
There are a few key strategies you can use to help you navigate the wide range of resources available online. For example, be sure to make the most of Google search techniques! Some helpful search modifiers include:
- Phrase searching, or using quotation marks around a word or phrase to only pull results that use the exact same words. Example: “Art in sustainability” university programs
- Site type searches, or using site: in combination with specific website extensions (e.g., .edu, .org., .gov) to only pull results from those pages. Example: Biophysics of mitosis site:edu
- File type searches, or using filetype: in combination with a specific file extension (e.g., ppt, pdf, xlsx [for excel documents]) to only generate results for a specific document type. Example: Romanesque vs classical art filetype:.ppt
You can also explore databases that collect high-quality gray literature, such as WorldCat, Open Grey, and GreyNet International. Consider also talking to your colleagues about your search— they may know of subject-specific resources that would be hard to discover independently!