Library DIY: Finding Books and Articles

Your assignment requires you to find "significant" research on a topic, but what makes a piece of research significant?

A significant piece of research is one that has had a large impact on the field. This is measured by the article's "impact factor", or how many times that article has been cited in other articles and books (traditional metrics), or shared on blogs, websites, and social media (alt-metrics). A high impact factor usually means that the material presented in the article is important or foundational to the field.

The impact of an article can also be influenced by the journal it is published in. Not all academic journals are the same. Some journals are more widely read, cited more frequently, and seen as more impactful and thus more competitive.

This does not mean that every article published in these journals will be a high-impact or significant article. It also doesn't mean that these articles are always better, or that they will be more helpful for your research project, but significance can be an important factor to consider, especially when reviewing the literature on a topic.

 

Some Criticisms of Impact Factors 

• Citations aren't always endorsements.  Sometimes articles are cited so that they can be disagreed with.  Worse, sometimes an article is cited many times but later retracted because it was found to contain inaccurate or misrepresented information. 
• Impact factors and h5-index scores can vary by discipline.  A Chemistry or Biology article will likely be cited many more times, perhaps hundreds more, than an article about Art History or Music Education because of the nature of the field. That doesn't make the Chemestry article "better" than the Music Education article, they are different and they are for a different audience.  It is important to compare "apples-to-apples" when looking at impact factors. 
• Data can be manipulated/impacted by personal and cultural factors.  Some authors purposefully cite their own previous work, or works of their academic friends, specifically to boost impact factor scores.  It is, unfortunately, also the case that some authors are given more (or less) credence based on their age, race, or gender and that can impact how many people read, cite, and share their work, regardless of how good the work itself is.
• Traditional metrics (impact factor and h5-index) can't capture the true usefulness of an article.  An article may have only be cited a few times by other professionals, but it may be saved to Mendeley accounts of thousands of undergraduate students, shared on Twitter hundreds of times, or bookmarked by dozens of professionals who are currently writing articles that haven't been published yet.  Alt-metrics are starting to capture this type of use, but are not included in impact factor or h5-index scores. 

 

What else should you consider?

Evaluating significance is one way to think critically when selecting which articles to use, but it is not the only possible criteria. You might also look for articles:

• that have been published recently (the meaning of "recent" will vary by field or discipline),
• that use a similar methodology,
• look at a certain population, or
• include your research question in their ‘directions for future research’.

Articles that have been cited many times are typically important or foundational.

When choosing between two relevant articles, you may include the number of times an article was cited, or it's impact factor, as part of your decision-making process.

Let's look at an example! 

For my review of research on CRISPR, I have found these two short research briefs:

Both of these articles give me background information on how CRISPR was used and publicized in 2013.

So, how do I choose one article over the other?

I might look at how many times each source has been cited to see which is more impactful.

 

I can search for them by title in Scopus

Scopus will show me how many times these sources were cited in other ‘top’ journals. The numbers of times cited will be lower and reflect the core of the scholarly conversation happening on this topic.

I can click the number to see/search within sources that have cited these articles.

 

Or in Google Scholar

Google scholar will show me how many times these sources were cited in many peer-reviewed journals, government reports, dissertations, patents, etc. The numbers will be higher and reflect use more broadly. I can click “Cited by” to see/search within citing sources.

 

Based on these results, I would choose “Multiplex genome engineering using CRISPR/Cas systems", because it seems to be much more significant to the scholarly conversation around CRISPR.

 

Some journals are seen as more prestigious and more competitive because they are more widely read, their articles are cited more frequently, and they publish more highly-cited works. You may want to take into account where an article was published when choosing which article to use.

 

Consider these three articles written by the same person, on similar topics, and published in journals with similar names. All of these will likely be useful for research on professional development for Music Teachers, but are they all from high-impact journals?

 

This 2003 article was published in the Journal of Research in Music Education:

This 2007 article was published in the Journal of Music Teacher Education:

While this 2010 article was published in the Music Educators Journal:

 

How can you tell a journal's impact factor?

An h5-index score tells you how many articles published in the last 5 years have been cited that same number of times.  For example, an h5-index score of 16 means that 16 articles published within the last 5 years have at least 16 citations each. An h5-index score of 20 means that 20 articles published within the last 5 years have at least 20 citations each

 

Google Scholar maintains a free and easy to use list based on a journals’ h5-index (a measure of impactfulness).

 

Go to scholar.google.com. Click the menu (three parallel lines) and then select “Metrics”

 

You will see the “Top Publications” screen - a list of journals, in order by h5-index score.

Search for a specific journal title by clicking the magnifying glass

 

Or select a category and subcategory from the drop-down lists.

 

It's recommended that you do both - search for your specific journal first to get its h5-index score, and then return to the main list, and sort by category to see where your journal compares to the top publications in that field.  

 

So, let's compare the h5-index scores of our three journals about music education.

 

The Journal of Music Teacher Education has an h5-index of 15.

 

Music Educators Journal has an h5-index of 14.

 

And the Journal of Research in Music Education has an h5-index of 23.

 

We can see that the Journal of Research in Music Education has the highest h5-index score, so we can say that this is the most impactful journal of the three.

 

Using the Categories drop-down menu, we can see how these three journals compare to other journals in the field of “Music & Musicology”. This will help us to determine if an h5-index of 23 is really good within the field of Music & Musicology (it is truly a high-impact journal) or if it is just the highest-scoring journal of the three we happened to pick.

 

We can see that the Journal of Research in Music Education is ranked third among all of the Music & Musicology journals. This is a high-impact journal for the field.