A case study of how bad science spreads

Fake news has been the go-to criticism of the media for some politicians, which in turn has been rejected as propaganda and fear-mongering by journalists. However, as former journalist Simon Linacre argues, the fourth estate needs to have its own house is in order first, and ensure they are not tripped up by predatory journals.


I class myself as a ‘runner’, but only in the very loosest sense. Inspired by my wife taking up running a few years ago, I decided I should exercise consistently instead of the numerous half-hearted, unsuccessful attempts I had made over the years. Three years later I have done a couple of half-marathons, run every other day, and track my performance obsessively on Strava. I have also recently started to read articles on running online, and have subscribed to the magazine Runners World. So yes, I think I may actually be a runner now.

But I’m also an experienced journalist, a huge cynic, and have a bulls*** radar the size of the Grand Canyon, so even while relaxing with my magazine I like to think I can spot fakery a mile off. And so it proved earlier this summer while reading a piece on how hill running can improve your fitness. This was music to my ears as someone who lives half-way up a valley side, but my interest was then piqued when I saw a reference to the study that formed the basis for the piece, which was to an article in the International Journal of Scientific Research. Immediately, I smelt a rat. “There is no way that is the name of a reputable, peer-reviewed journal,” I thought. And I was right.

But that wasn’t even half of the problem.

After checking Cabells’ Predatory Reports database, I found not one but TWO journals are listed on the database with that name, both with long lists of breaches of the Cabells’ criteria that facilitate the identification of predatory journals. I was still curious as to the nature of the research, as it could have been legitimate research in an illegitimate journal, or just bad research, full stop. As it turned out, neither journal had ever published any research on hill running and the benefits of increasing VO2 max. So where was the story from?

After some more digging, an article matching the details in the Runners World piece could be found in a third similarly-named journal, the International Journal of Scientific and Research Publications. The article, far from the recent breakthrough suggested in the August 2020 issue of Runners World, was actually published in August 2017 by two authors from Addis Ababa University in Ethiopia. While the science of the article seems OK, the experiment that produced the results was on just 32 people over 12 weeks, which means it really needs further validation across greater subjects to confirm its findings. Furthermore, while the journal itself was not included in Cabells’ Predatory Reports database, a review found significant failings, including unusually quick peer review processes and, more seriously, that the “owner/Editor of the journal or publisher falsely claims academic positions or qualifications”. The journal has subsequently been added to Predatory Reports, and the article itself has never been cited in the three years since publication.

Yet one question remains: how did a relatively obscure article, published in a predatory journal and that has never been cited, find its way into a news story in a leading consumer magazine? Interestingly, similar research was also quoted on MSN.com in May 2020 which also quoted the International Journal of Scientific Research, while other sites have also quoted the same research but from the International Journal of Scientific and Research Publications. It appears likely that, having been quoted online once, the same story has been doing the rounds for three years like a game of ‘Telephone,’ all based on uncited research that may not have been peer reviewed in the first place, that used a small sample size and was published in a predatory journal.

While no damage has been done here – underlying all this, it does make sense that hill running can aid overall performance – one need only need to think about the string of recent health news stories around the coronavirus to see how one unverified article could sweep like wildfire through news outlets and online. This is the danger that predatory journals pose.

They’re not doctors, but they play them on TV

Recently, while conducting investigations of suspected predatory journals, our team came across a lively candidate. At first, as is often the case, the journal in question seemed to look the part of a legitimate publication. However, after taking a closer look and reading through one of the journal’s articles (“Structural and functional brain differences in key opinion journal leaders“) it became clear that all was not as it seemed.

Neurology and Neurological Sciences: Open Access, from MedDocs Publishers, avoids a few of the more obvious red flags that indicate deceitful practices, even to neophyte researchers, but lurking just below the surface are several clear behavioral indicators common to predatory publications.

1a

With a submission date of August 22, 2018, and a publication date November 13, 2018, the timeline suggests that some sort of peer review of this article may have been carried out. A closer examination of the content makes it evident that little to no peer review actually took place. The first tip-off was the double-take inducing line in the “Material and methods” section, “To avoid gender bias, we recruited only males.” Wait, what? That’s not how that works.

It soon became clear to our team that even a rudimentary peer review process (or perhaps two minutes on Google) would have led to this article’s immediate rejection. While predatory journals are no laughing matter, especially when it comes to medical research in the time of a worldwide pandemic, it is hard not to get a chuckle from some of the “easter eggs” found within articles intended to expose predatory journals. Some of our favorites from this article:

  • Frasier Crane, a listed author, is the name of the psychiatrist from the popular sitcoms Cheers and Frasier
  • Another author, Alfred Bellow, is the name of the NASA psychiatrist from the TV show I Dream of Jeannie
  • Marvin Monroe is the counselor from The Simpsons
  • Katrina Cornwell is a therapist turned Starfleet officer on Star Trek: Discovery
  • Faber University is the name of the school in Animal House (Faber College in the film)
  • Orbison University, which also doesn’t exist, is likely a tribute to the late, great musician Roy Orbison

And, perhaps our favorite find and one we almost missed:

  • In the “Acknowledgments” section the authors thank “Prof Joseph Davola for his advice and assistance.” This is quite likely an homage to the Seinfeld character “Crazy Joe Davola.”

Though our team had a few laughs with this investigation, they were not long-lived as this is yet another illustration of the thousands (Predatory Reports currently lists well over 13,000 titles) of journals such as this one in operation. Outlets that publish almost (or literally) anything, usually for a fee, with no peer review or other oversight in place and with no consideration of the detrimental effect it may have on science and research.

MedDocs PR card
Predatory Reports listing for Neurology and Neurological Sciences: Open Access

A more nuanced issue that deceptive publications create involves citations. If this was legitimate research, the included citations would not ‘count’ or be picked up anywhere since this journal is not indexed in any citation databases. Furthermore, any citation in a predatory journal that cites a legitimate journal is ‘wasted’ as the legitimate journal cannot count or use that citation appropriately as a foundation for its legitimacy. However, these citations could be counted via Google Scholar, although (thankfully) this journal has zero. Citation ‘leakage’ can also occur, where a legitimate journal’s articles cite predatory journals, effectively ‘leaking’ those citations out of the illegitimate scholarly publishing sphere into legitimate areas. These practices can have the effect of skewing citation metrics which are measures often relied upon (sometimes exclusively, often too heavily) to gauge the legitimacy and impact of academic journals.

When all is said and done, as this “study” concludes, “the importance of carefully selecting journals when considering the submission of manuscripts,” cannot be overstated. While there is some debate around the use of “sting” articles such as this one to expose predatory publications, not having them exposed at all is far more dangerous.

Cabells’ top 7 palpable points about predatory publishing practices

In his latest post, Simon Linacre looks at some new stats collated from the Cabells Predatory Reports database that should help inform and educate researchers, better equipping them to evade the clutches of predatory journals.


In recent weeks Cabells has been delighted to work with both The Economist and Nature Index to highlight some of the major issues for scholarly communication that predatory publishing practices represent. As part of the research for both pieces, a number of facts have been uncovered that not only help us understand the issues inherent in this malpractice much better, but should also point researchers away from some of the sadly typical behaviors we have come to expect.

So, for your perusing pleasure, here are Cabells’ Top 7 Palpable Points about Predatory Publishing Practices:

  1. There are now 13,500 predatory journals listed in the Predatory Reports database, which is currently growing by approximately 2,000 journals a year
  2. Over 4,300 journals claim to publish articles in the medical field (this includes multidisciplinary journals) – that’s a third of the journals in Predatory Reports. By discipline, medical and biological sciences have many more predatory journals than other disciplines
  3. Almost 700 journals in Predatory Reports start with ‘British’ (5.2%), while just 50 do on the Journalytics database (0.4%). Predatory journals often call themselves American, British or European to appear well established and legitimate, when in reality relatively few good quality journals have countries or regions in their titles
  4. There are over 5,300 journals listed in Predatory Reports with an ISSN (40%), although many of these are copied, faked, or simply made up. Having an ISSN is not a guarantee of legitimacy for journals
  5. Around 41% of Predatory Reports journals are based in the US, purport to be from the US, or are suspected of being from the US, based on information on journal websites and Cabells’ investigations. This is the highest count for any country, but only a fraction will really have their base in North America
  6. The average predatory journal publishes about 50 articles a year according to recent research from Bo-Christer Björk of the Hanken School of Economics in Helsinki, less than half the output of a legitimate title. Furthermore, around 60% of papers in such journals receive no future citations, compared with 10% of those in reliable ones
  7. Finally, it is worth noting that while we are in the throes of the Coronavirus pandemic, there are 41 journals listed on in Predatory Reports (0.3%) specifically focused on epidemiology and another 35 on virology (0.6% in total). There could be further growth over the next 12 months, so researchers in these areas should be particularly careful now about where they submit their papers.

Reversal of fortune

One of the most common questions Cabells is asked about its Predatory Reports database of journals is whether it has ever “changed its mind” about listing a journal. As Simon Linacre reports, it is less a question of changing the outcome of a decision, but more of a leopard changing its spots.


This week saw the annual release of Journal Impact Factors from Clarivate Analytics, and along with it the rather less august list of journals whose Impact Factors have been suppressed in Web of Science. This year there were 33 journals suspended, all of which for “anomalous citation patterns found in the 2019 citation data” which pertained to high levels of self-citation. Such a result is the worst nightmare for a publisher, as while they can be due to gaming citation levels, they can also sometimes reflect the niche nature of a subject area, or other anomalies about a journal.

Sometimes the decision can be changed, although it is often a year or two before the data can prove a journal has changed its ways. Similarly, Cabells offers a review process for every journal it lists in its Predatory Reports database, and when I arrived at the company in 2018, like many people one of the first things I asked was: has Cabells ever had a successful review to delist a journal?

Open for debate

The answer is yes, but the details of those cases are quite instructive as to why journals are included on the database in the first place, and perhaps more importantly whey they are not. Firstly, however, some context. It is three years since the Predatory Reports database was first launched, and in that time almost 13,500 journals have been included. Each journal has a link next to the violations on its report for anyone associated with that journal to view the policy and appeal the decision:

1a

This policy clearly states:

The Cabells Review Board will consider Predatory Journal appeals with a frequency of one appeal request per year, per journal. Publications in Predatory Reports, those with unacceptable practices, are encouraged to amend their procedures to comply with accepted industry standards.

Since 2017, there have been just 20 appeals against decisions to list journals in Predatory Reports (0.15% of all listed journals), and only three have been successful (0.02%). In the first case (Journal A), the journal’s peer review processes were checked and it was determined that some peer reviews were being completed, albeit very lightly. In addition, Cabells’ investigators found a previous example of dual publication. However, following the listing, the journal dealt with the problems and retracted the article it had published as it seemed the author had submitted two identical articles simultaneously. This in turn led to Cabells revising its evaluations so that particular violation does not penalize journals for something where an author was to blame.

In the second review (Journal B), Cabells evaluated the journal’s peer review process and found that it was also not completing full peer reviews and had a number of other issues. It displayed metrics in a misleading way, lacked editorial policies on its website and did not have a process for plagiarism screening. After its listing in PR, the journal’s publisher fixed the misleading elements on its website and demonstrated improvements to its editorial processes. In this second case, it was clear that the journal’s practices were misleading and deceptive, but they chose to change and improve their practices.”

Finally, a third journal (Journal C) has just had a successful appeal completed. In this case, there were several problems that the journal was able to correct by being more transparent on its website. It added or cleared up confusion about the necessary policies and made information about its author fees available. Cabells was also able to evaluate its peer review process after it submitted peer review notes on a few articles and it was evident the journal editor was managing a good quality peer review, hence it has now been removed from the Predatory Reports database (it should be noted that, as with the other two successful appeals, journals removed from Predatory Reports are not then automatically included in the Cabells Journalytics database).

Learning curve

Cabells’ takeaway from all of these successful reviews was they were indeed successful – they showed that the original identification was correct, and they enabled improvements that identified them as better, and certainly non-predatory, journals. They also fed into the continuing improvement Cabells seeks in refining its Predatory Reports criteria, with a further update due to be published later this summer.

There are also things to learn from unsuccessful reviews. In one case a publisher appealed a number of its journals that were included on Predatory Reports. However, their appeal only highlighted how bad the journals actually were. Indeed, an in-depth review of each journal not only uncovered new violations that were subsequently added to the journals, but also led to the addition of a brand new violation that is to be included in the upcoming revision of the Predatory Reports criteria.

Five dos and don’ts for avoiding predatory journals

HAVE YOUR SAY

Publication ethics is at the core of everything that Cabells does, and it continually promotes all scholarly communication bodies which seek to uphold the highest standard of publishing practices. As such, we would like to express our support for Simon Linacre (Cabells’ Director of International Marketing and Development) in his candidacy to become a COPE Trustee. COPE plays an essential role in ensuring scholarly publishing maintains the highest standards, and if you are a COPE member is it important you use your vote to support the organization’s progress.

Simon has been with Cabells two years, and involved in academic publishing for almost 20 years. In that time he has gained wide experience of all aspects of journal publishing, and in particular Open Access issues which this role focuses on.

If you would like to vote in the election, please go to the COPE website, log in and cast your vote for your favored candidate.

Thanks, The Cabells Team

It is three years since Cabells first launched its database on predatory journals, and a good deal has happened in that time in the world of scholarly publishing. In his latest post, Simon Linacre reflects on these changes and offers some ‘dos and don’ts’ on the latest version of the database.


In June 2017 – which seems a lifetime ago now for all sorts of reasons – Cabells launched a new database that included details on over 4,000 predatory journals. It was the first time that a resource of that size had been made available to researchers who wanted to check the legitimacy or otherwise of journals they may be considering as a destination for their articles. In the intervening years, it is to be hoped many authors have been alerted to the dangers of publishing their research in such journals and benefited from worthwhile publishing experiences in good journals.

At the time, Cabells chose to name the database the ‘Blacklist’ as the most straightforward description of the intent of the database. As some may have seen, we brought forward the decision to change its name to ‘Predatory Reports’ last week in the first of a number of changes Cabells intends to introduce in 2020 and beyond.

5.5 x 8.5 – PR - front

The new name includes the word ‘Reports’ for an important reason. The database has been designed as more than a simple list of predatory, fake or questionable journals – it has also been put together so that researchers can use the information that has been collated on all 13,400 journals to inform their understanding of scholarly communications, and as a result, make better decisions about their research publications and career into the future. In this spirit, here are FIVE DOS AND DON’TS of how to use the Cabells Predatory Reports database:

  1. DO check all violations listed for each journal on Predatory Reports to fully understand what the journal is NOT doing properly, as this can to help identify predatory behavior in future
  2. DON’T trust a journal because it has an ISSN on its website – over 40% of journals listed on Predatory Reports include one, with many copied from legitimate journals or simply invented
  3. DO check the publisher’s name in the ‘Advanced Search’ option if a journal is not included on the database, as the same publisher could have created a new journal with the same predatory behaviors
  4. DON’T visit a predatory journal website unnecessarily as they could contain malware – hover the cursor over the website to view the full URL to see if it corresponds to that of the journal being checked out
  5. DO send Cabells updates or information on potential new predatory journals by sending an email to ‘journals@cabells.com’

And as a final ‘DO’, do click the link to our 70+ criteria that we use to identify predatory journals – these will be updated soon to streamline and clarify the process of reviewing journals for inclusion in Predatory Reports, and offer a much more robust checklist than currently exists to help researchers avoid falling into the predatory journal trap.

Predatory Report Appeals

Following this appeal process, a journal found to no longer qualify for Cabells Predatory Reports will be removed. Please note that removal from Cabells Predatory Reports does, by no means, qualify a journal for Cabells Journalytics. Journals wishing to be included in Cabells Journalytics must follow the proper application procedures and are subject to additional review to confirm qualification.

Cabells Predatory Report Criteria v 1.1

VERSION INFORMATION

This post serves to outline the revision to the Cabells Predatory Report criteria that went into effect on March 13, 2019. Journals evaluated under this version of the criteria will link to this page.

VERSION CHANGE NOTES

  • The indicator that read “Gender bias in the editorial board” was removed
  • To increase the granularity of our evaluations on this subject, the indicator “The publisher hides or obscures relationships with for-profit partner companies” was split into two separate indicators:
    • The journal/publisher hides or obscures relationships with for-profit companies that could result in corporate manipulation of science
    • The journal/publisher hides or obscures information regarding associated publishing imprints or parent companies
  • The indicator that read “Emails from journals received by researchers who are clearly not in the field the journal covers” was changed and split into two separate indicators:
    • Emailed solicitations for manuscripts from the journal are received by researchers who are clearly not in the field the journal covers
    • Email invitations for editorial board members or reviewers from the journal are received by researchers who are clearly not in the field the journal covers
  • The following behavioral indicators were added:
    • Evident data that little to no peer review is being done and the journal claims to be “peer reviewed.”
    • No affiliations are identified for editorial board members and/or editors.
    • Editorial board members (appointed over 2 years ago) have not heard from the journal at all since being appointed to the board.
    • The journal has a large editorial board but very few articles are published per year.
    • The journal’s website attempts to download a virus or malware.
    • The number of articles published has increased by 25-49% in the last year.
    • The number of articles published has increased by 50-74% in the last year.
    • The number of articles published has increased by 75% or more in the last year.
    • The journal is open access but no information is given about how the journal is supported financially (i.e. author fees, advertising, sponsorship, etc.).

GENERAL INFORMATION

This policy establishes the criteria for identifying deceptive, fraudulent, and/or predatory journals for inclusion in Cabells Predatory Reports. The Cabells Predatory Report Review Board uses the following criteria to evaluate all journals suspected of deceptive, fraudulent, and/or predatory practices. Each identified behavior listed is assigned a score based on the severity of the offense. The behaviors are grouped according to relative severity and subject matter.

CRITERIA

The following criteria are considered are considered SEVERE:

  • Integrity
    • The same article appears in more than one journal.
    • Hijacked journal (defined as a fraudulent website created to look like a legitimate academic journal for the purpose of offering academics the opportunity to rapidly publish their research for a fee).
    • Information received from the journal does not match the journal’s website.
    • The journal or publisher claims to be a non-profit when it is actually a for-profit company.
    • The owner/Editor of the journal or publisher falsely claims academic positions or qualifications.
    • The journal is associated with a conference that has been identified as predatory.
    • The journal gives a fake ISSN.
  • Peer Review
    • No editor or editorial board listed on the journal’s website at all.
    • Editors do not actually exist or are deceased.
    • The journal includes scholars on an editorial board without their knowledge or permission.
    • Evident data that little to no peer review is being done and the journal claims to be “peer reviewed.”
  • Publication Practices
    • The journal publishes papers that are not academic at all, e.g. essays by laypeople or obvious pseudo-science.
    • No articles are published or the archives are missing issues and/or articles.
    • Falsely claims indexing in well-known databases (especially SCOPUS, DOAJ, JCR, and Cabells).
    • Falsely claims universities or other organizations as partners or sponsors.
    • Machine-generated or other “sting” abstracts or papers are accepted.
  • Indexing & Metrics
    • The journal uses misleading metrics (i.e., metrics with the words “impact factor” that are not the Clarivate Analytics Impact Factor).
  • Fees
    • The journal offers options for researchers to prepay APCs for future articles.
    • The journal states there is an APC or another fee but does not give information on the amount or gives conflicting information.
    • The journal or publisher offers membership to receive discounts on APCs but does not give information on how to become a member and/or on the membership fees.
    • The author must pay APC or publication fee before submitting the article (specifically calls the fee a publication fee, not a submission fee).
    • The journal does not indicate that there are any fees associated with publication, review, submission, etc. but the author is charged a fee after submitting a manuscript.

The following criteria are considered MODERATE:

  • Integrity
    • The title of the journal is copied or so similar to that of a legitimate journal that it could cause confusion between the two.
    • The name of the journal references a country or demographic that does not relate to the content or origin of the journal.
    • The journal uses language that suggests that it is industry leading, but is in fact a new journal.
    • The journal/publisher hides or obscures relationships with for-profit partner companies that could result in corporate manipulation of science.
  • Peer Review
    • The journal has a large editorial board but very few articles are published per year.
    • Inadequate peer review (i.e., a single reader reviews submissions; peer reviewers read papers outside their field of study; etc.).
    • The journal’s website does not have a clearly stated peer review policy.
    • The founder of the publishing company is the editor of all of the journals published by said company.
    • Evident data showing that the editor/review board members do not possess academic expertise to reasonably qualify them to be publication gatekeepers in the journal’s field.
    • No affiliations are given for editorial board members and/or editors.
    • Little geographical diversity of board members and the journal claims to be International.
    • The journal includes board members who are prominent researchers but exempt them from any contribution to the journal except the use of their names and/or photographs.
    • Editorial board members (appointed over 2 years ago) have not heard from the journal at all since being appointed to the board.
  • Publication Practices
    • No copyediting.
    • Little geographical diversity of authors and the journal claims to be International.
    • The Editor publishes research in his own journal.
    • The journal purposefully publishes controversial articles in the interest of boosting citation count.
    • The journal publishes papers presented at conferences without additional peer review.
    • The name of the publisher suggests that it is a society, academy, etc. when it is only a publisher and offers no real benefits to members.
    • The name of the publisher suggests that it is a society, academy, etc. when it is only a solitary proprietary operation and does not meet the definition of the term used or implied non-profit mission.
    • Authors are published several times in the same journal and/or issue.
    • Similarly titled articles published by same author in more than one journal.
    • The publisher displays prominent statements that promise rapid publication and/or unusually quick peer review (less than 4 weeks).
    • The number of articles published has increased by 75% or more in the last year.
    • The number of articles published has increased by 50-74% in the last year.
  • Fees
    • The publisher or journal’s website seems too focused on the payment of fees.
  • Access & Copyright
    • States the journal is completely open access but not all articles are openly available.
    • No way to access articles (no information on open access or how to subscribe).
    • The journal is open access but no information is given about how the journal is supported financially (i.e. author fees, advertising, sponsorship, etc.).
    • No policies for digital preservation.
    • The journal has a poorly written copyright policy and/or transfer form that does not actually transfer copyright.
    • The journal publishes not in accordance with their copyright or does not operate under a copyright license.
  • Business Practices
    • The journal has been asked to quit sending emails and has not stopped.
    • The journal or publisher gives a business address in a Western country but the majority of authors are based in developing countries.
    • Emailed solicitations for manuscripts from the journal are received by researchers who are clearly not in the field the journal covers.
    • Email invitations for editorial board members or reviewers from the journal are received by researchers who are clearly not in the field the journal covers.
    • Multiple emails received from a journal in a short amount of time.
    • Emails received from a journal do not include the option to unsubscribe to future emails.
    • The journal copyproofs and locks PDFs.

The following criteria are considered MINOR:

  • Integrity
    • Insufficient resources are spent on preventing and eliminating author misconduct that may result in repeated cases of plagiarism, self-plagiarism, image manipulation, etc. (no policies regarding plagiarism, ethics, misconduct, etc., no use of plagiarism screens).
    • The journal/publisher hides or obscures information regarding associated publishing imprints or parent companies.
  • Website
    • The website does not identify a physical address for the publisher or gives a fake address.
    • The journal or publisher uses a virtual office or other proxy business as its physical address.
    • The website does not identify a physical editorial address for the journal.
    • Dead links on the journal or publisher’s website.
    • Poor grammar and/or spelling on the journal or publisher’s website.
    • No way to contact the journal/only has web-form.
    • The journal’s website attempts to download a virus or malware.
  • Publication Practices
    • The number of articles published has increased by 25-49% in the last year.
  • Indexing & Metrics
    • The publisher or its journals are not listed in standard periodical directories or are not widely catalogued in library databases.
  • Business Practices
    • No subscribers / nobody uses the journal.
    • The journal’s website does not allow web crawlers.