Impact Challenge Day 4: Connect with other researchers on Mendeley.com

Next up for our Impact Challenge is Mendeley.

Are you surprised? While there was pushback against Mendeley after it was unexpectedly bought by Elsevier a few years ago, and it is marketed more as a reference manager than a social network, Mendeley remains popular with many academics and librarians. It offers ways to connect with other researchers that you can’t find on other platforms.

Mendeley Web (the online counterpart to the desktop reference management software) is similar to Google Scholar in several ways. What’s distinctive about Mendeley is that it offers better opportunities to interact with other researchers and get your research in front of communities that might be interested in it, in a context where they’re largely interacting with scholarship they intend to actually read and cite.

Moreover, Mendeley’s Readership Statistics can tell you a lot about the demographics that have bookmarked your work–an important indicator of who’s reading your work and who might cite it in the future.

We’re also going to talk in this post about Zotero, which is quite similar to Mendeley. We’re big supporters of Zotero because it’s an open-source non-profit, and we see that as a killer feature for science tools. However, although it really shines as a reference manager, Zotero’s community features are less powerful–mostly because they have less activity. So we’ll provide links and information on how to do some of these steps in Zotero, but not in as much detail.

Step 1: Create a profile

Logon to Mendeley.com and click the “Create a free profile” button. Create a login and, on the next screen, enter your general field of study and your academic status (student, professor, postdoc, etc).

As you advance to the next screen, beware: Mendeley Desktop will automatically start downloading to your computer. (You’ll need it to make the next step a bit easier on yourself, but you can also make do without it. Your call.) Download it and install it if you plan to use it for the next step–importing your publications.

Zotero alternative: Logon to Zotero.org, click “Register” in the upper right-hand corner, and register for an account. Once you’ve validated your new account, click your username in the upper right-hand corner (where it says, “Welcome, username!”) and then click on the “Edit Profile” link on the next screen to head to the Profile section of your Zotero settings. There, you can create a profile.

Step 2: Import your publications

If you didn’t install Mendeley Desktop, here’s how to add your references manually using Mendeley Web:

  • Click the “My Library” tab, then the “Add Document” icon.

  • On the “Add New Document” dialog box that appears, select “My Publications” from the “Add to” drop-down menu, then use the “Type” drop-down menu to specify what type of document you’re adding to your “My Publications” list (article, book section, thesis, etc).

  • The dialog box will automatically expand, giving you many fields to fill out with descriptive information for that publication. Complete as many as possible, so others can find your publication more easily. If an Open Access link to the full-text of your publication exists, provide it in the URL box. And be sure to add a DOI, if you’ve got one. Click “Save” when finished.

  • Rinse and repeat as necessary, until all your articles are added to your profile.

If you’ve got Mendeley Desktop installed, your job is much easier. Export your publications in .bib format from Google Scholar (which we covered in yesterday’s challenge), and then:

  • Fire up Desktop and select “My Publications” from the “My Library” panel in the upper left corner of the screen.

  • Click File > Import > BibTeX (.bib) on the main menu.

  • On your computer, find the citations.bib file you exported from Google Scholar, select it, and click “Open”. Mendeley will begin to import these publications automagically.

  • In the dialog box that appears, confirm that you are the author of the documents that you’re importing, and that you have the rights to share them on Mendeley. Click “I agree.”

  • Click the “Sync” button at the top of the Desktop screen to Sync your local Mendeley library with your Mendeley Web library.

That’s it! You’ve just added all your publications to your Mendeley profile. And you know how to add any missing publications that didn’t auto-import, to boot.

Here’s what your profile page will look like, now that you’ve added publications to your My Publications library:

Screen Shot 2014-11-04 at 9.12.32 PM.png

Zotero alternative: to auto-import your publications from a BiBTeX file, follow these instructions. To manually add publications, follow these instructions.

Step 3: Follow other researchers

Now you’re ready to connect with other researchers. Consider this step akin to introducing yourself at a conference over coffee: informal, done in passing, and allowing others to put a face to a name.

First, you’ll need to find others to follow. Search for colleagues or well-known researchers in your field by name from the Mendeley search bar in the upper right-hand screen of Mendeley Web:

Screen Shot 2014-11-04 at 9.17.39 PM.png

Be sure to select “People” from the drop-down menu, so you search for profiles and not for papers that they’ve authored.

When you find their profile, click on their name in the search results, and then click the “Follow” button on the right-hand side of the profile:

Screen Shot 2014-11-04 at 9.20.56 PM.png

That’s it! Now you’ll receive updates on your Mendeley homepage when they’ve added a new publication to their profile or done something else on the site, like join a group.

Zotero alternative: Zotero works in a very similar way. Search for your colleague, find their profile, and click the red “Follow” button at the top-right of their profile to begin following them.

Step 4: Join groups relevant to your research

If Step 3 was like introducing yourself during a conference coffee break, Step 4 is like joining a “Birds of a Feather” group over lunch, to talk about common interests and get to know each other a bit better.

Screen Shot 2014-11-04 at 9.27.13 PM.png

Mendeley groups are places where researchers interested in a common topic can virtually congregate to post comments and share papers. It’s a good place to find researchers in your field who might be interested in your publications. And it’s also the single best place on the platform to learn about work that’s recently been published and is being talked about in your discipline.

To find a group, search for a subject using the search toolbar you used for Step 3, making sure to select “Groups” from the drop-down menu. Look through the search results and click through to group pages to determine if the group is still active (some groups were abandoned long-ago).

If so, join it! And then sit back and enjoy all the new knowledge that your fellow group members will drop on you in the coming days, which you can view from either the group page or your Mendeley homescreen.

And you can feel free to drop some knowledge on them, too. Share your articles, if relevant to the group’s scope. Pose questions and answer others’ questions. Openly solicit collaborators if you’ve got an interesting project in the pot that you need help on, like Abbas here has:

Screen Shot 2014-11-04 at 9.43.54 PM.png

Use groups like you would any other professional networking opportunity: as a place to forge new connections with researchers you might not have a chance to meet otherwise.

Zotero alternative: Zotero works in a very similar way. Search for a group topic, find a group you want to join, and click the red “Join Group” button at the top of the page.

Step 5: Learn who’s bookmarking your work

Once your work is on Mendeley, you can learn some basic information about who’s saving it in their libraries via Mendeley’s Readership Statistics. And that’s interesting to know because Mendeley bookmarks are a leading indicator for later citations.

To see the readership demographics for your publications, head to the article’s page on Mendeley. On the right side of the screen, you’ll see a small Readership Statistics panel:

Screen Shot 2014-11-04 at 9.58.08 PM.png

Readership Statistics can tell you how many readers you have on Mendeley (meaning, how many people have bookmarked your publication), what discipline they belong to, their academic status, and their country. Very basic information, to be sure, but it’s definitely more than you’d know about your readers if you were looking at the number of readers alone.

Zotero alternative: Zotero doesn’t yet offer readership statistics or any other altmetrics for publications on their site, but they will soon. Stay tuned!

Limitations

Perhaps the biggest limitation to Mendeley is their association with Elsevier. Mendeley was acquired by the publishing behemoth in early 2013, while the ghastly, Elsevier-backed Research Works Act fail was still fresh in many academics’ minds.

As danah boyd points out, even after Elsevier dropped support for the RWA and the “#mendelete” fracas ended, Elsevier was (and is) still doing a lot that’s not researcher-friendly. And yet, some of us continue to eat at McDonald’s knowing what goes into their chicken nuggets. Like any big organization, Elsevier does some stuff right and some stuff wrong, and it’s up to researchers to decide how it all balances out; there’s lots of room for reasonable folks to disagree. For what it’s worth: at Impactstory, one of us is a Zotero early adopter and code-contributor, one of us has switched from Mendeley to Zotero, and one of us uses both 🙂

Drawbacks to the platform itself? You can’t easily extract readership information for your publications unless you use Mendeley’s open API (too high a barrier for many of us to pass). So, you’ll need to cut-and-paste that information into your website, CV, or annual review, just as you would when using Google Scholar. (It’s relatively easy to extract readership numbers using third-party services like Impactstory, on the other hand. More on that in the days to come.)

A final drawback: if you want to add new publications, you’ll have to do it yourself. Mendeley doesn’t auto-add new publications to your profile like Google Scholar or other platforms can.

Homework

First, complete your profile by manually adding any works that the BibTeX import from Google Scholar didn’t catch.

Next, build your network by following at least five other researchers in your field, and joining at least two groups. On each of the groups you’ve joined, share at least one publication, whether it’s one you’ve authored or one written by someone else. Remember, make sure they’re relevant to the group, or else you’ll be pegged as a spammer.

Over the next few days, log onto Mendeley Web (or Zotero Web) at least one more time, and become acquainted with your homescreen timeline to stay abreast of new research that’s been added to groups or your colleagues’ profiles.

Finally, learn how to export your publications–and the rest of your library–from Mendeley, so you don’t have to reinvent the wheel attempting to set up a profile for your publications on another platform. Here’s how to get your library out of Mendeley in BibTeX format:

  • In Mendeley Desktop, select all publications you want to export.

  • From the main menu, click File > Export.

  • In the dialog box that appears, choose BibTeX from the drop-down menu, rename your bibliography if you want, and choose a safe place to store the .bib file. Click “Save” and you’re done!

Are you hangin’ in there?

You’ve now completed your Day 4 challenge, meaning you’re over halfway finished with Week 1, and over 10% finished with the entire month. That’s some free math, from us to you 🙂

X7GsMZcVH1erS.gif

Tomorrow, we’ll master LinkedIn. Get ready!

Impact Challenge Day 3: Create a Google Scholar Profile

We’ve covered two of academia’s most popular social networks for the November Impact Challenge so far. Let’s now dig into the research platform that’s most often used by researchers: Google Scholar.

Screen Shot 2014-11-04 at 12.00.58 PM.png

Google Scholar offers a popular way to create a profile that showcases your own papers and the citations they’ve received. It also calculates a platform-dependent h-index, which many researchers love to track (for better or for worse).

In today’s challenge, we’re going to get you onto Google Scholar, so you can up your scholarly SEO (aka “googleability”), more easily share your publications with new readers, and discover new citations to your work from across the entire scholarly web.

Step 1: Create your basic profile

Log on to scholar.google.com  and click the “My Citations” link at the top of the page to get your account setup started.

On the first screen, add your affiliation information and university email address, so Google Scholar can confirm your account. Add keywords that are relevant to your research interests, so others can find you when browsing a subject area. Provide a link to your university homepage, if you have one. Click “Next Step,” and–that’s it! Your basic profile is done. Now, let’s add some publications to it.

Step 2: Add publications

Google has likely already been indexing your work for some time now as part of their mission as a scholarly search engine. So, this step is pretty easy, compared to what it takes to get your work on to Academia.edu or ResearchGate.

Screen Shot 2014-11-03 at 5.24.05 PM.png

Google Scholar will provide you with a list of publications they think belong to you. You’ll need to read through the list of publications that it suggests as yours and select which ones you want to add to your profile. Beware–if you have a common name, it’s likely there’s some publications in this list that don’t belong to you. And there’s also possibly content that you don’t want on your profile because it’s not a scholarly article, or is not representative of your current research path, and so on. Read through the publications list and deselect any that you do not want to add to your profile, like the below newsletter item that Google Scholar thinks is a scholarly article, then click the grey “Add” button at the top of your profile.

Screen Shot 2014-11-04 at 12.17.07 PM.png

Next, confirm you want Google to automatically add new publications to your profile in the future. Note that this might add publications you didn’t author to your profile if you’ve got a very common name, but can be worth it for the time it saves you approving new articles every month.

Your profile is now almost complete! Two more steps: add a photo by clicking the “Change Photo” link on your profile homepage, and set your private profile to “Public.”

Step 3: Make your profile public

Your profile is private if you’ve just created it. Change your profile visibility by clicking “Edit” next to “My profile is private” and then selecting “My profile is public” in the drop-down box.

Bonus: Add co-authors

Screen Shot 2014-11-03 at 6.00.26 PM.png

While your profile is technically complete, you’ll want to take advantage of Google Scholar’s built-in co-authorship network. Adding co-authors is a good way to let others know you’re now on Google Scholar, and will be useful later on in the Challenge, when we set up automatic alerts that can help you stay on top of new research in your field.

To add a suggested co-author, find the “Add Co-authors” section on the top right-hand section of your profile, then click the plus-sign next to each co-author you want to add.

That’s it! Now you’ve got a Google Scholar profile that helps you track when your work has been cited both in the peer-reviewed literature and elsewhere (more on that in a moment), and is yet another scholarly landing page that’ll connect others with your publications. The best part? Google Scholar’s pretty good at automatically adding new stuff to your profile, meaning you won’t have to do a lot of work to keep it up.

Limitations

Dirty data in the form of incorrect publications isn’t the only limitation of Google Scholar you should be aware of. The quality of Google Scholar citations has also been questioned, because they’re different from what scholars have traditionally considered to be a citation worth counting: a citation in the peer-reviewed literature.

Google Scholar counts citations from pretty much anywhere they can find them. That means their citation count often includes citations from online undergraduate papers, slides, white papers and similar sources. Because of this, Google Scholar citation counts are much higher than those from competitors like Scopus and Web of Science.

That can be a good thing. But you can also argue it’s “inflating” citation counts unfairly. It also makes Google Scholar’s citation counts quite susceptible to gaming techniques like using fake publications to fraudulently raise the numbers. We’ve not heard many evaluators complaining about these issues so far, but it’s good to be aware of.

Google Scholar also shares a limitation with ResearchGate and Academia.edu: it’s somewhat of an information silo. You cannot export your citation data, meaning that even if you were to amass very impressive citation statistics on the platform, the only way to get them onto your website, CV, or an annual report is to copy and paste them–way too much tedium for most scientists to endure. Their siloed approach to platform building definitely contributes to researchers’ profile fatigue.

Its final major limitation? There’s no telling if it will be around tomorrow. Remember Google Reader? Google has a history of killing beloved products when the bottom line is in question.  It’s not exaggerating to say that Google Scholar Profiles could literally go away at any moment.

That said, the benefits of the platform outweigh the downsides for many. And we’re going to give you a way to beat part of the “information silo” problem in today’s homework.

Homework

Google Scholar can only automate so much. To fully complete your Google Scholar profile, let’s manually add any missing articles. And let’s also teach you how to export your publication information from Google Scholar, because you’ll want to reuse it on other platforms.

1. Add missing articles

You might have an article or two that Google Scholar didn’t automatically add to your profile. If that’s the case, you’ll need to add it manually.

Click the “Add” button in the grey toolbar in the top of your profile.

Screen Shot 2014-11-03 at 5.39.53 PM.png

On the next page, click the “Add articles manually” link in the left-hand toolbar. Then you’ll see this screen:

Screen Shot 2014-11-03 at 5.41.58 PM.png

It’s here where you can add new papers to your profile. Include as much descriptive information as possible–it makes it easier for Google Scholar to find citations to your work. Click “Save” after you’ve finished adding your article metadata, and repeat as necessary until all of your publications are on Google Scholar.

2. Clean up your Google Scholar Profile data

Thanks to Google Scholar Profiles’ “auto add” functionality, your Profile might include some articles you didn’t author.

If that’s the case, you can remove them in one of two ways:

  • clicking on the title of each offending article to get to the article’s page, and then clicking the “Delete” button in the top green bar

  • from the main Profile page, ticking the boxes next to each incorrect article and selecting the “Delete” from the drop-down menu in the top grey bar

If you want to prevent incorrect articles from appearing on your profile in the first place, you can change your Profile settings to require Google Scholar to email you for approval before adding anything. To make this change, from your main Profile page, click the drop-down menu that appears in the top grey bar. Select “Profile updates”:

Screen Shot 2014-11-03 at 5.35.22 PM.png

On the next page,  change the setting to “Don’t automatically update my profile.”

Screen Shot 2014-11-03 at 5.37.03 PM.png

Prefer to roll the dice? You can keep a close eye on what articles are automatically added to your profile by signing up for alerts and manually removing any incorrect additions that appear. Here’s how to sign up for alerts: click the blue “Follow” button at the top of your profile, select “Follow new articles,” enter your email address, and click “Create alert.”

3. Learn how to export your publications list in BiBTeX format

There will likely be a time when you’ll want to export your Google Scholar publications to another service like Impactstory or Mendeley. Here’s how to do that.

Screen Shot 2014-11-03 at 5.51.17 PM.png

Tick the box next to each article whose details you want to export, or tick the top left-hand box to select all articles on your profile. With the relevant articles selected, click the“Export” button like we see on the right here, then choose BiBTeX. Next, choose to export either the selected articles or all articles from your profile, then click the final “Export” button to download your “citations.bib” file.

4. Explore your citations

Your final bit of homework is to enjoy learning about all the different places you’ve been cited. Because Google Scholar indexes citations from across the entire scholarly web, there’s likely a lot of places in the world that you’re being cited, in many different publication formats.

Take some time to look not only at the numbers Google Scholar provides, but to also click through the numbers to see the actual citing publications themselves. Read them. See if you can’t connect with the authors on ResearchGate or Academia.edu, if you’re so inclined.

And if you haven’t yet been cited, don’t despair! Now that more people have the opportunity to find your research on Google Scholar and elsewhere, the citations likely aren’t far away.

In coming days, we’ll cover how to use Google Scholar to stay abreast of new research in your field and new citations to your work. Stay tuned!

Impact Challenge Day 2: Make a ResearchGate profile

Yesterday, you used Academia.edu to make new connections, find new readers for your work, and track how often your work is being read.

Today, we’ll help you master the other major player in the scholarly social network space, ResearchGate. ResearchGate, which claims 5 million scientists as users, will help you connect with many researchers who aren’t on Academia.edu (especially those outside North America). It can also help you understand your readers through platform-specific metrics, and confirm your status as a helpful expert in your field with their “Q&A” feature.

Given ResearchGate’s similarity to Academia.edu, I won’t rehash the basics of setting up a profile and getting your publications online. Go ahead and sign up, setup your account (remember to add detailed affiliation information and a photo), and add a publication or two.

Got your basic profile up and running? Great! Let’s drill down into those three unique features of ResearchGate that you’re going to explore for your Day 2 Challenge.

Finding other researchers & publications

Finding other researchers and publications on ResearchGate works a bit differently than on Academia.edu. Rather than allow you to specify “research interests” and find other researchers that way, ResearchGate automatically creates a network for you based on who you’ve cited, who you follow and what discipline you selected when setting up your profile.

So, key to creating a robust network is uploading papers with citations to be text-mined, and searching for and following other researchers in your field.

Searching for other researchers in your field is easy: using the search bar at the top of the screen, type in your colleague’s name. If they’re on the site, they’ll appear in the dynamic search results, as we see below with Impactstory Advisor Lorena Barba:

Screen Shot 2014-11-03 at 11.51.38 AM.png

Click on their name in the search results to be taken to their page, where you can explore their publications, co-authors, and so on, and also follow them to receive updates.

Screen Shot 2014-11-02 at 9.03.43 PM.png

ResearchGate also text-mines the publications you’ve uploaded to find out who you’ve cited; they add both researchers you’ve cited and who have cited you to your network, as well as colleagues from your department and institution.

Here’s how to explore your network: click the “Publications” tab at the top of your screen to begin exploring the publications that are in your network. You can browse the most recent publications in your area of interest, your network, and so on, using the navigation bar seen above.

Screen Shot 2014-11-04 at 10.47.38 AM

If you find an interesting publication, you can click the paper title to read the paper or click on the author’s name to be taken to their profile, where you can explore their other publications or choose to follow them, adding a new colleague to your network in a snap.

ResearchGate Score & Stats

If you’re into metrics, the ResearchGate score and stats offer lots to explore. The ResearchGate score is an indicator of your popularity and engagement on the site: the more publications and followers you have, plus the more questions you ask and answer, all add up to your score. Check out Christoph Lutz’s ResearchGate score–one of the more diversely-sourced scores I’ve seen to date:

Screen Shot 2014-11-03 at 12.02.01 PM.png

ResearchGate also helpfully provides a percentile (seen above on the right-hand side), so you know how a score stacks up against other users on the site. The score isn’t normalized by field, though, so beware that using the score to compare yourself to others isn’t recommended.

Some other downsides to be aware of: ResearchGate scores don’t take into account whether you’re first author on a paper, they weigh site participation much more highly than other (more important) indicators of your scientific prowess, and don’t reflect the reality of who’s a high-impact scientist in many fields. So, caveat emptor.

All that said, ResearchGate scores are fun to play around with and explore. Just be sure not to take them too seriously.

The stats are also illuminating: they tell you how often your publications have been viewed and cited on ResearchGate both recently and over time, what your top publications are, and the popularity of your profile and any questions you may have asked on the site’s Q&A section. On your profile page, you’ll see a summary of your stats:

Screen Shot 2014-11-03 at 12.24.07 PM.png

If you click on those stats, you’ll be taken to your stats page, which breaks down all of your metrics with nice visualizations:

Screen Shot 2014-11-03 at 12.25.33 PM.png

A caveat: like Academia.edu stats, ResearchGate stats are only for content hosted on ResearchGate, so it can’t tell you much about readership or citations of your work that’s hosted on other platforms.

Q&A

Now that we’ve made some passive connections by following other researchers, let’s build some relationships by contributing to the Q&A section of the site.

On the Q&A section, anyone can pose a question, and if it’s related to your area of expertise, ResearchGate will give you the opportunity to answer. We’ll talk more about the benefits of participating in the Q&A section of the site in the coming days, but basically it’s a good opportunity to help other researchers and get your name out there.

Click on “Q&A” at the top of your screen and explore the various questions that have been posed in your discipline in recent weeks. You can also search for other topics, and pose questions yourself.

Two more cool ResearchGate features worth mentioning: they mint DOIs, meaning that if you need a permanent identifier for an unpublished work, you can get one for free (though keep in mind that they haven’t announced a preservation plan, meaning their DOIs might be less stable over time than DOIs issued by a CLOCKSSS-backed repository like Figshare). And you can also request Open Reviews of your work, which allows anyone on ResearchGate who’s in your area of expertise to give you feedback–a useful mechanism for inviting others to read your paper. It’s a feature that hasn’t seen much uptake, but is full of possibilities in terms of connecting other researchers to your work.

Limitations

Several readers have pointed out that Academia.edu and ResearchGate are information silos–you put information and effort into the site, and can’t easily extract and reuse it later. And they’re absolutely correct. That’s a big downside of these services and a great reason to check out open alternatives like PeerLibrary, ORCID, and Impactstory (more on the latter two services in the days to come).

Some other drawbacks to both Academia.edu and ResearchGate: they’re both for-profit, venture capital funded platforms, meaning that their responsibility isn’t to academics but to investors. And sure, they’re both free, which seems like an advantage until you remember that it means that you are the product, not the customer.

One solution to these drawbacks is to limit the amount of time you spend adding new content to your profiles on these sites, and instead use them as a kind of “landing page” that can simply help others find you and your three or four most important publications. Even if you don’t have all your publications on either site, their social networking features are still useful to make connections and increase readership for your most important work.

In the coming days, we’ll cover other web services that offer auto-updates and data portability, so you don’t end up suffering from Profile Fatigue.

Two more things:

  1. Be sure to check your ResearchGate notification settings to cut down on spam. They send more emails than most email-fatigued academics care to receive.
  2. Make sure you’ve opted-out of sending invitations, so you don’t accidentally contribute to spamming others.

Homework

Set up your ResearchGate profile and at least three publications you think deserve attention. Next, search for at least 5 colleagues or well-known researchers in your field and follow each of them. Once you’ve established a network, take 10 minutes to explore the “Publications” tab of ResearchGate, browsing publications that have been recently published in your network.

In the coming days, take another 10 minutes to explore your ResearchGate score and stats. Are there any that surprise you, in terms of what’s getting a lot of readers? How might you incorporate this information into your professional life outside of ResearchGate: would you put it on your CV or website, into an annual review or grant application in order to showcase your “broader impacts”? It’s ok if you say “no” to these ideas–the point is to get you thinking about what these metrics mean, and if and when you might use them professionally.

As for the Q&A section of ResearchGate–we’ll cover that soon. Stay tuned!

Day 2: Nailed it.

 

qmEboC2VVjBgQ.gif

Now you’ve got connections on two of academia’s biggest social networks, and you’ve increased potential exposure for your publications, to boot. You’ve also got two new sources of metrics that’ll show how often you’re read and cited.

Are you ready for Day 3? We’re going to cover Google Scholar Profiles–a great tool for finding citations, upping your “googleability” even further, and staying on top of new publications in your field.

Until then, we welcome bragging about your ResearchGate mastery in the comments below! Questions also welcome. 🙂

PS It’s Day 2 and the November Impact Challenge is in full swing. It’s work, right? But stick with it–the work is worth it!

In 28 more days, your network and professional visibility will be in a place many scholars take years to reach, and ready to grow even more.

And today, we’d like to give you a extra little incentive. Here’s a deal: if you can finish all 30 days, we’ll hook you up with this free t-shirt to show off your achievement!

Screencap of the "Finisher" t-shirt, showing a boxer in silhouette with the  words "Finisher: November Impact Challenge" on it.

More info to come!

November Impact Challenge Day 1: Make a profile on Academia.edu

Welcome to the November Impact Challenge!

Over the next 30 days, we’re going to work together to supercharge your research impact. You’ll:

  • upgrade your professional visibility by conquering social media,

  • boost your readership and citations by getting your work online,

  • stay atop your field’s latest developments with automated alerting,

  • lock in the key connections with colleagues that’ll boost your career, and

  • dazzle evaluators with comprehensive tracking and reporting on your own impacts.

Each day’s challenge will look like this: we’ll describe that day’s important principle–why it’s important, how you can get started, and some resources to help you excel–and then share a homework assignment, where you’ll apply the concepts we cover in that day’s post.

Are you ready? Let’s dive in, starting with scholarly social media.

Make a profile on Academia.edu

You know all those things you wish your CV was smart enough to do–embed your papers, automatically give you readership statistics, and so on? Academia.edu and ResearchGate (which we’ll cover in tomorrow’s challenge) are two academic social networks that allow you to do these things and a lot more.

Perhaps more importantly, they’re places where your colleagues are spending a lot of their time. Actively participating on one or both networks will give you ample opportunity to have greater reach with other researchers. And getting your publications and presentations onto these sites will make it easier for others to encounter your work, not only for the social network they help you build, but also to improve the search engine optimization (SEO) of your research, making you much more “googleable”.

Generally speaking, both platforms allow you to do the following:

  • Create a profile that summarizes your research

  • Upload your publications, so others can find them

  • Find and follow other researchers, so you can receive automatic updates on their new publications

  • Find and read others’ publications

  • See platform-specific metrics that indicate the readership and reach you have on those sites

Today, we’ll cover getting started with Academia.edu. Let’s dig into the basics of setting up a profile and uploading your work.

Basic profile setup

Logon to Academia.edu. If you’re a firm believer in keeping your professional online presence separate from your personal one, you’ll likely want to sign up using your university email address. Otherwise, you can sign up using your Facebook or Google profile.

From here, you’ll be directed through the basic signup process.

Post a publication

How do you choose what to share? If you’re an established researcher, this will be easy: just choose your most “famous” (read: highly cited) paper. If you’re a junior researcher or a student, choosing might be tougher. A peer-reviewed paper is always a good bet, as-is a preprint or a presentation that’s closely related to your most current topic of research.

Got a paper in mind? Now comes the not-as-fun-but-incredibly-necessary part: making sure you’ve got the rights to post it. Most academics don’t realize that they generally sign away their copyright when publishing an article with a traditional publisher. And that means you may not have the rights to post the publisher’s version of your article on Academia.edu. (If you negotiated to keep your copyright or published with an authors’ rights-respecting journal like PLOS Biology, give yourself a pat on the back and skip the following paragraph.)

If you don’t have copyright for your paper, all hope is not lost! You likely have the right to post your version of the article (often the unedited, unformatted version). Head over to Sherpa/Romeo and look up the journal you published in. You’ll see any and all restrictions that the publisher has placed on how you can share your article.

If you can post your article, let’s upload it to Academia.edu. Click the green “Upload a paper” button and, on your computer, find the publication you want to upload. Click “Open” and watch as Academia.edu begins to upload your paper.

Once it’s uploaded, the title of your publication will be automatically extracted. Make any corrections necessary to the title, then click in the “Find a Research Interest” box below the title. Add some keywords that will help others find your publication. Click save.

Add your affiliation and interests to your profile

Adding an affiliation is important because it will add you to a subdomain of Academia.edu built for your university, and that will allow you to more easily find your colleagues. The site will try to guess your affiliation based on your email address or IP address; make any corrections needed and add your department information and title. Click “Save & Continue,” then add your research interests on the following page. These are also important; they’ll help others find you and your work.

Connect with colleagues

In this final step, you’ll be prompted to either connect your Facebook account or an email account to Academia.edu, which will search your contacts and suggest connections. Select and confirm anyone you want to follow on the site. I recommend starting out small, to keep from being overwhelmed by updates.

Congrats, you’ve now got an Academia.edu profile!

You can continue to spruce it up by adding more publications, as well as adding a photo of yourself and other research interests and publications, and connecting your Academia profile to other services like Twitter and LinkedIn, if you’re already on ‘em. (If not, don’t worry–we’ll cover that soon.)

Homework

Now that you have a profile, set aside half an hour to explore three important uses of Academia.edu: exploring “research interests” in order to discover other researchers and publications; getting more of your most important publications online; and using the Analytics feature to discover who’s following you, how often others are reading and downloading your work, and in which countries your work is most popular.

Research interests: To get started exploring, click on the research interests in your profile:

Screencap of Jonathan Eisen's profile, highlighting his research interests

For the search results that appear, take some time to explore the profiles of others who share your interest(s) and follow anyone that looks interesting. Click on the Documents tab of the search results and explore relevant papers and presentations (below is an example of what the “Human Microbiome” research interest page looks like); I’m willing to bet you’ll find many papers and connections that you weren’t aware of before.

Screen Shot 2014-11-02 at 8.42.52 PM.png

You can also search for other research interests using the search bar at the top of the screen.

Upload more papers & presentations: click the “Upload papers” tab at the top  right corner of your screen and upload at least two more papers or presentations that you think are worthy of attention. Remember to abide by any copyright restrictions that might exist, and also be sure to add as much descriptive information as possible, adding the complete title, co-authors, and research interests for your paper, all of which will make it easier for others to find.

Analytics: click the “Analytics” tab at the top of your screen and poke around a bit. Because you just created your profile, it’s possible you won’t yet have any metrics. But in as little as a few days, you’ll begin to see download and pageview statistics for your profile and your publications (as seen below), and other interesting information like maps, all of which can help you better understand the use your work is getting from other researchers!

Screen Shot 2014-11-04 at 1.34.58 PM

So–you’ve claimed your professional presence on one of academia’s biggest social networks and learned how to use it to find other researchers and publications. More importantly, you’ve optimized your profile so others can find you and your research much more easily.

Congrats! Day 1 Challenge: achievement unlocked!

Let’s see your results

Post a link to your profile in the comments, and let us know if you have any questions or tips on how to use Academia.edu.

See you tomorrow for our Day 2 challenge: mastering ResearchGate!

Is ResearchGate’s new DOI feature a game-changer?

ResearchGate is academia’s most popular social network, with good reason. While some decry the platform for questionable user recruitment tactics, others love to use it to freely share their articles, write post-publication peer reviews, and pose questions to other researchers in their area.

ResearchGate quietly launched a feature recently, one that we think could be a big deal. It may have huge upsides for research–especially for tracking altmetrics for work–but it also highlights how some of the problems of scholarly communication aren’t easily solved, especially when digital persistence is involved.

The feature in question? ResearchGate is now generating DOIs for content. And that’s started to generate interesting conversations among those in the know.

Here’s why: DOIs are unique, persistent identifiers that publishers and repositories issue for their content, with the understanding that URLs break all the time. A preservation strategy is expected when one starts issuing DOIs, and yet ResearchGate hasn’t announced one, nor has DataCite (which issues ResearchGate’s DOIs).

Some other interesting questions: what happens when users decide to delete content, or leave the site altogether? Will ResearchGate force content to remain online, or allow DOIs to redirect to broken URLs?

And what if a publication already has a DOI? ResearchGate does prompt users to provide a DOI if one is available, but there are no automated checks (as far as we can tell). That may leave room for omission or error. And a DOI that potentially can resolve to more than one place will introduce confusion for those searching for an article.

As a librarian, I’m also curious about the implications for repositories. IRs’ main selling point is digital persistence and preservation. So, if ResearchGate does indeed have a preservation policy in place, repositories may have lost their edge.

We’ll be watching future developments with interest. There’s great potential here, and how ResearchGate grows and matures this feature in the future will likely have an influence on how researchers share their work and, quite possibly, what it means to be a “publisher.”

Open Science & Altmetrics Monthly Roundup (October 2014)

Open Access Week dominated Open Science conversation this month, along with interesting UK debates on metrics and several valuable studies being released. Read on for more on all of it!

UK debates use of metrics in research evaluation

Academia’s biggest proponents and critics of altmetrics descended on the University of Sussex on October 7 for the event, “In Metrics We Trust?”.

Some of the most interesting finds shared at the meeting?

  • REF peer reviewers admitted they spend less than 15 minutes reviewing papers for quality, due to the sheer volume of products that need evaluation,

  • Departmental h-indices tend to correlate with REF/RAE evaluations, leading some to argue that time and money could be saved by replacing future REF exercises with metrics, and

  • Leading bibliometrics researchers disagree on whether altmetrics could be used for evaluation. Some said they cannot, no matter what; others said that they can, because altmetrics measure different impacts than citations measure.

The meeting ended with no clear answer as to whether metrics are definitely right (or wrong) for use in the next REF. We’ll have to wait for the HEFCEmetric committee’s recommendations when they issue their report in June.

Until then, check out the full debate in our Storify of the event, as well as Ernesto Priego’s archive of related tweets.

OA Week 2014 recap

The Impactstory team is still recovering from Open Access Week 2014, which saw us talking to over 100 researchers and librarians in 9 countries over 5 days. A full recap of our talks can be found on the Impactstory blog, along with “The Right Metrics for Generation Open: a Guide to getting credit for Open Science”, based on our most popular webinar from the week.

Interest in Open Access and Open Science has risen over the past year, making this year’s Open Access Week successful according to all reports. As Heather Morrison has documented on her blog, the past year has seen an increase in the availability OA documents and data–ArXiv.org alone has grown by 11%!

Other altmetrics & Open Science news

Did we miss anything?

What was your favorite event or new study released this month? Share it in the comments below, or on Twitter (you can find us @Impactstory).

Are you ready to take the November Impact Challenge?

Graphic for the challenge shows a boxer poised, ready to bout.

In a hugely competitive research landscape, scientists can no longer afford to just publish and hope for the best. To leave a mark, researchers have to take their impact into their own hands.

But where do you start? There are so many ways to share, promote, and discuss your research, especially online. It’s tough to know where to begin.

Luckily, we’ve got your back.

Drawing on years of experience measuring and studying research impact, we’ve created a list of 30 effective steps for you to make sure your hard work gets out there, gets attention, and makes a difference–in your field and with the public.

We’ll share one of these a day in November, and we challenge you to follow along and give each one a try.

If you’re up to the challenge, we guarantee that by the end of the month, your research will get a boost in exposure and you’ll also have made important connections with other scientists around the world.

Join us here on our blog on Monday, November 3rd for the Impact Challenge kickoff, or follow along via email.

Tracking the impacts of data – beyond citations

This post was originally published on the e-Science Community Blog, a great resource for data management librarians.

"How to find and use altmetrics for research data" text in front of a beaker filled with green liquid

How can you tell if data has been useful to other researchers?

Tracking how often data has been cited (and by whom) is one way, but data citations only tell part of the story, part of the time. (The part that gets published in academic journals, if and when those data are cited correctly.) What about the impact that data has elsewhere?

We’re now able to mine the Web for evidence of diverse impacts (bookmarks, shares, discussions, citations, and so on) for diverse scholarly outputs, including data sets. And that’s great news, because it means that we now can track who’s reusing our data, and how.

All of this is still fairly new, however, which means that you likely need a primer on data metrics beyond citations. So, here you go.

In this post, I’ll give an overview of the different types of data metrics (including citations and altmetrics), the “flavors” of data impact, and specific examples of data metric indicators.

What do data metrics look like?

There are two main types of data metrics: data citations and altmetrics for data. Each of these types of metrics are important for their own reasons, and offer the ability to understand different dimensions of impact.

Data citations

Much like traditional, publication-based citations, data citations are an attempt to track data’s influence and reuse in scholarly literature.

The reason why we want to track scholarly data influence and reuse? Because “rewards” in academia are traditionally counted in the form of formal citations to works, printed in the reference list of a publication.

Data is often cited in two ways: by citing the data package directly (often by pointing to where the data is hosted in a repository), and by citing a “data paper” that describes the dataset, functioning primarily as detailed metadata, and offering the added benefit of being in a format that’s much more appealing to many publishers.

In the rest of this post, I’m going to mostly focus on metrics other than citations, which are being written about extensively elsewhere. But first, here’s some basic information on data citations that can help you understand how data’s scholarly impacts can be tracked.

How data packages are cited

Much like how citations to publications differ depending on whether you’re using Chicago style or APA style formatting, citations to data tend to differ according to the community of practice and the recommended citation style of the repository that hosts data. But there are a core set minimums for what should be included in a citation. Jon Kratz has compiled these “core elements” (as well as “common elements) over on the DataPub blog. The core elements include:

  • Creator(s): Essential, of course, to publicly credit the researchers who did the work. One complication here is that datasets can have large (into the hundreds) numbers of authors, in which case an organizational name might be used.

  • Date: The year of publication or, occasionally, when the dataset was finalized.

  • Title: As is the case with articles, the title of a dataset should help the reader decide whether your dataset is potentially of interest. The title might contain the name of the organization responsible, or information such as the date range covered.

  • Publisher: Many standards split the publisher into separate producer and distributor fields. Sometimes the physical location (City, State) of the organization is included.

  • Identifier: A Digital Object Identifier (DOI), Archival Resource Key (ARK), or other unique and unambiguous label for the dataset.

Arguably the most important principle? The use of a persistent identifier like a DOI, ARK, or Handle. They’re important for two reasons: no matter if the data’s URL changes, others will still be able to access it; and PIDs provide citation aggregators like the Data Citation Index and Impactstory.org an easy, unambiguous way to parse out “mentions” in online forums and journals.

It’s worth noting, however, that as few as 25% of journal articles tend to formally cite data. (Sad, considering that so many major publishers have signed on to FORCE11’s data citation principles, which include the need to cite data packages in the same manner as publications.) Instead, many scholars reference data packages in their Methods section, forgoing formal citations, making text mining necessary to retrieve mentions of those data.

How to track citations to data packages

When you want to track citations to your data packages, the best option is the Data Citation Index. The DCI functions similarly to Web of Science. If your institution has a subscription, you can search the Index for citations that occur in the literature that reference data from a number of well-known repositories, including ICPSR, ANDS, and PANGEA.

Here’s how: login to the DCI, then head to the home screen. In the Search box, type in your name or the dataset’s DOI. Find the dataset in the search results, then click on it to be taken to the item record page. On the item record, find and click the “Create Citation Alert” button on the right hand side of the page, where you’ll also find a list of articles that reference that dataset. Now you have a list of the articles that reference your data to date, and you’ll also receive automated email alerts whenever someone new references your data.

Another option comes from CrossRef Search. This experimental search tool works for any dataset that has a DataCite DOI and is referenced in the scholarly literature that’s indexed by CrossRef. (DataCite issues DOIs for Figshare, Dryad, and a number of other repositories.) Right now, the search is a very rough one: you’ll need to view the entire list of DOIs, then use your browser search (often accessed by hitting CTRL + F or Command +F) to check the list for your specific DOI. It’s not perfect–in fact, sometimes it’s entirely broken–but it does provide a view into your data citations not entirely available elsewhere.

How data papers are cited

Data papers tend to be cited like any other paper: by recording the authors, title, journal of publication, and any other information that’s required by the citation style you’re using. Data papers are also often cited using permanent identifiers like DOIs, which are assigned by publishers.

How to find citations for data papers

To find citations to data papers, search databases like Scopus and Web of Science like you’d search for any traditional publication. Here’s how to track citations in Scopus and Web of Science.

There’s no guarantee that your data paper is included in their database, though, since data paper journals are still a niche publication type in some fields, and thus aren’t tracked by some major databases. You’ll be smart to follow up your database search with a Google Scholar search, too.

Altmetrics for data

Citations are good for tracking the impact of your data in the scholarly literature, but what about other types of impact, among other audiences like the public and practitioners?

Altmetrics are indicators of the reuse, discussion, sharing, and other interactions humans can have with a scholarly object. These interactions tend to leave traces on the scholarly web.

Altmetrics are so broadly defined that they include pretty much any type of indicator sourced from a web service. For the purposes of this post, we’ll separate out citations from our definition of altmetrics, but note that many altmetrics aggregators tend to include citation data.

There are two main types of altmetrics for data: repository-sourced metrics (which often measure not only researchers’ impacts, but also repositories’ and curators’ impacts), and social web metrics (which more often measure other scholars’ and the public’s use and other interactions with data).

First, let’s discuss the nuts and bolts of data altmetrics. Then, we’ll talk about services you can use to find altmetrics for data.

Altmetrics for how data is used on the social web

Data packages can be shared, discussed, bookmarked, viewed, and reused using many of the same services that researchers use for journal articles: blogs, Twitter, social bookmarking sites like Mendeley and CiteULike, and so on. There are also a number of services that are specific to data, and these tend to be repositories with altmetric “indicators” particular to that platform.

For an in-depth look into data metrics and altmetrics, I recommend that you read Costas’ et al’s report, “The Value of Research Data” (2013). Below, I’ve created a basic chart of various altmetrics for data and what they can likely tell us about the use of data.

Quick caveat: there’s been little research done into altmetrics for data. (DataONE, PLOS, and California Digital Library are in fact the first organizations to do major work in this area, and they were recently awarded a grant to do proper research that will likely confirm or negate much of the below list. Keep an eye out for future news from them.) The metrics and their meanings listed below are, at best, estimations based on experience with both research data and altmetrics.

Repository- and publisher-based indicators

Note that some of the repositories below are primarily used for software, but can sometimes be used to host data, as well.

Web Service

Indicator

What it might tell us

Reported on

GitHub

Stars

Akin to “favoriting” a tweet or underlining a favorite passage in a book, GitHub stars may indicate that some who has viewed your dataset wants to remember it for later reference.

GitHub, Impactstory

Watched repositories

A user is interested enough in your dataset (stored in a “repository” on GitHub) that they want to be informed of any updates.

GitHub, PlumX

Forks

A user has adapted your code for their own uses, meaning they likely find it useful or interesting.

GitHub, Impactstory, PlumX

SourceForge

Ratings & Recommendations

What do others think of your data? And do they like it enough to recommend it to others?

SourceForge, PlumX

Dryad, Figshare, and most institutional and subject repositories

Views & Downloads

Is there interest in your work, such that others are searching for and viewing descriptions of it? And are they interested enough to download it for further examination and possible future use?

Dryad, Figshare, and IR platforms; Impactstory (for Dryad & Figshare); PlumX (for Dryad, Figshare, and some IRs)

Figshare

Shares

Implicit endorsement. Do others like your data enough to share it with others?

Figshare, Impactstory, PlumX

PLOS

Supplemental data views, figure views

Are readers of your article interested in the underlying data?

PLOS, Impactstory, PlumX

Bitbucket

Watchers

A user is interested enough in your dataset that they want to be informed of any updates.

Bitbucket

Social web-based indicators

Web Service

Indicator

What it might tell us

Reported on

Twitter

tweets that include links to your product

Others are discussing your data–maybe for good reasons, maybe for bad ones. (You’ll have to read the tweets to find out.)

PlumX, Altmetric.com, Impactstory

Delicious, CiteULike, Mendeley

Bookmarks

Bookmarks may indicate that some who has viewed your dataset wants to remember it for later reference. Mendeley bookmarks may be an indicator for later citations (similar to articles).

Impactstory, PlumX; Altmetric.com (CiteULike & Mendeley only)

Wikipedia

Mentions (sometimes also called “citations”)

Does others think your data is relevant enough to include it in Wikipedia encyclopedia articles?

Impactstory, PlumX

ResearchBlogging, Science Seeker

Blog post mentions

Is your data being discussed in your community?

Altmetric.com, PlumX, Impactstory

How to find altmetrics for data packages and papers

Aside from looking at each platform that offers altmetrics indicators, consider using an aggregator, which will compile them from across the web. Most altmetrics aggregators can track altmetrics for any dataset that’s either got a DOI or is included in a repository that’s connected to the aggregator. Each aggregator tracks slightly different metrics, as we discussed above. For a full list of metrics, visit each aggregator’s site.

Impactstory easily tracks altmetrics for data uploaded to Figshare, GitHub, Dryad, and PLOS journals. Connect your Impactstory account to Figshare and GitHub and it will auto-import your products stored there and find altmetrics for them. To find metrics for Dryad datasets and PLOS supplementary data, provide DOIs when adding products one-by-one to your profile, and the associated altmetrics will be imported. Here’s an example of what a altmetrics for dataset stored on Dryad looks like on Impactstory.

PlumX tracks similar metrics, and offers the added benefit of tracking altmetrics for data stored on institutional repositories, as well. If your university subscribes to PlumX, contact the PlumX team about getting your data included in your researcher profile. Here’s what altmetrics for dataset stored on Figshare looks like on PlumX.

Altmetric.com can track metrics for any dataset that has a DOI or Handle. To track metrics for your dataset, you’ll either need an institutional subscription to Altmetric or the Altmetric bookmarklet, which you can use when on the item page for your dataset on a website like Figshare or in your institutional repository. Here’s what altmetrics for a dataset stored on Figshare looks like on Altmetric.com.

Flavors of data impact

While scholarly impact is very important, it’s far from the only type of impact one’s research can have. Both data citations and altmetrics can be useful in illustrating these flavors. Take the following scenarios for example.

Useful for teaching

What if your field notebook data was used to teach undergraduates how to use and maintain their own field notebooks, and use them to collect data? Or if a longitudinal dataset you created were used to help graduate students learn the programming language, R? These examples are fairly common in practice, and yet they’re often not counted when considering impacts. Potential impact metrics could include full-text mentions in syllabi, views & downloads in Open Educational Resource repositories, and GitHub forks.

Reuse for new discoveries

Researcher, open data advocate, and Impactstory co-founder Heather Piwowar once noted, “the potential benefits of data sharing are impressive:  less money spent on duplicate data collection, reduced fraud, diverse contributions, better tuned methods, training, and tools, and more efficient and effective research progress.” If those outcomes aren’t indicative of impact, I don’t know what is! Potential impact metrics could include data citations in the scholarly literature, GitHub forks, and blog post and Wikipedia mentions.

Curator-related metrics

Could a view-to-download ratio be an indicator of how well a dataset has been described and how usable a repository’s UI is? Or of the overall appropriateness of the dataset for inclusion in the repository? Weber et al (2013) recently proposed a number of indicators that could get at these and other curatorial impacts upon research data, indicators that are closely related to previously-proposed indicators by Ingwersen and Chavan (2011) at the GBIF repository. Potential impact metrics could include those proposed by Weber et al and Ingwersen & Chavan, as well as a repository-based view-to-download ratio.

Ultimately, more research is needed into altmetrics for datasets before these flavors–and others–are accurately captured.

Now that you know about data metrics, how will you use them?

Some options include: in grant applications, your tenure and promotion dossier, and to demonstrate the impacts of your repository to administrators and funders. I’d love to talk more about this on Twitter or in the comments below.

Recommended reading

  • Piwowar HA, Vision TJ. (2013) Data reuse and the open data citation advantage. PeerJ 1:e175 doi: 10.7717/peerj.175

  • CODATA-ICSTI Task Group. (2013). Out of Cite, Out of Mind: The current state of practice, policy, and technology for the citation of data [report]. doi:10.2481/dsj.OSOM13-043

  • Costas, R., Meijer, I., Zahedi, Z., & Wouters, P. (2013). The Value of research data: Metrics for datasets from a cultural and technical point of view. Copenhagen, Denmark. Knowledge Exchange. www.knowledge-exchange.info/datametrics

Open Access Week 2014 – a look back and ahead

118A3TUPCSiw6Y.gif

Much like Lil Bub, we’re bushed. Our Open Access Week 2014 was very eventful–we spoke with more than 100 researchers and librarians in 9 countries over 5 days. Here’s how we spent our time.

Throughout the week, Stacy hosted several sessions of “The Right Metrics for Generation Open: a guide to getting credit for Open Science,” where she talked about how Generation Open’s needs are evolving beyond that of previous generations of scientists. Altmetrics are particularly well-suited to meet those needs. You can view her slides on Slideshare, and read a long-form blogpost based on the presentation here on our blog.

Tuesday saw Stacy talking with faculty and librarians at the University of Alberta and the University of Memphis, where she explained “Why Open Research is Critical to your career”. (tl;dr: Change is coming in scholarly communication–so you should get on board and start making the most of the great opportunities that Open Science and altmetrics can offer you.) Check out her slides on Google Docs.

On Wednesday, Stacy had the pleasure of hangin’ virtually with librarians and library students at the University of Wisconsin, where they talked about the fact that “Altmetrics are here: are you ready to help your faculty?” After all, who’s a better neutral third-party to help faculty navigate this new world of altmetrics than librarians? Slides from that presentation are available on Google Docs.

Jason gave his popular talk, “Altmetrics & Revolutions: How the web is transforming the measure and practice of science” to researchers at the University of New Brunswick on Thursday. His slides are available on Google Docs.

Stacy rounded out the week by chatting with researchers and librarians at the University of Cape Town on Friday. Her presentation on the basics of altmetrics and how to use them–”Altmetrics 101: How to make the most of supplementary impact metrics”–is available for viewing on Google Docs.

Heather’s going to be the one in need of a nap over the next two weeks–she’ll be presenting on open data and altmetrics throughout Australia. Here are the events ahead:

  • Melbourne, Mon, 27 Oct, 9am–12.30pm:  Creating your research impact story:  Workshop at eResearch Australasia. Also featuring: Pat Loria, CSU, and Natasha Simons, ANDS. (sold out)

  • Melbourne, Wed, 29 Oct, 10–11am: Keynote presentation at eResearch Australasia. Register for conference

  • Brisbane, Mon, 3 Nov, 1–4.30pm: Uncovering the Impact Story of Open Research and Data. Also featuring Paula Callan, QUT, and Ginny Barbour, PLOS. QUT: Owen J Wordsworth Room. Level 12, S Block, QUT Gardens Point. (sold out)

  • Sydney, Wed, 5 Nov, 1.30–4.30pm: An afternoon of talks featuring Heather Piwowar. Also featuring Maude Frances, UNSW, and Susan Robbins, UWS. ACU MacKillop Campus, North Sydney: The Peter Cosgrove Centre, Tenison Woods House, 8-20 Napier Street, North Sydney. (sold out)

Are you ready, Oz?!

UuIom9saJP5eg.gif

The Right Metrics for Generation Open: a guide to getting credit for Open Science

You’re not getting all the credit you should be for your research.

As an early career researcher, you’re likely publishing open access journal articles, sharing your research data and software code on GitHub, posting slides and figures on Slideshare and Figshare, and “opening up” your research in many other ways.

Yet these Open Science products and their impacts (on other scholars, the public, policymakers, and other stakeholders) are rarely mentioned when applying for jobs, tenure and promotion, and grants.

The traditional means of sharing your impact–citation counts–don’t meet the needs of today’s researchers. What you and the rest of Generation Open need is altmetrics.

In this post, I’ll describe what altmetrics are and the types of altmetrics you can expect to receive as someone who practices Open Science. We’ll also cover real life examples of scientists who used altmetrics to get grants and tenure–and how you can do the same.

Altmetrics 101

Altmetrics measure the attention your scholarly work receives online, from a variety of audiences.

As a scientist, you create research data, analyses, research narratives, and scholarly conversations on a daily basis. Altmetrics–measures of use sourced from the social web– can account for the uses of all of these varied output types.

Nearly everything that can be measured online has the potential to be an altmetric indicator. Here are just a few examples of the types of information that can be tracked for research articles alone:

scholarly

public

recommended

faculty of 1000

popular press

cited

traditional  citation

wikipedia

discussed

scholarly blogs

blogs, twitter

saved

mendeley, citeulike

delicious

read

pdf views

html views

 

When you add research software, data, slides, posters, and other scholarly outputs to the equation, the list of metrics you can use to understand the reception to your work grows exponentially.

And altmetrics can also help you understand the interest in your work from those both inside and outside of the Ivory Tower. For example, what are members of the public saying about your climate change research? How has it affected the decisions and debates among policy makers? Has it led to the adoption of new technologies in the private sector?

The days when your research only mattered to other academics are gone. And with them also goes the idea that there’s only one type of impact.

Flavors of impact

There are many flavors of impact that altmetrics can illuminate for you, beyond the traditional scholarly impact that’s measured by citations.

This 2012 study was the first to showcase the concept of flavors of impact via altmetrics. These flavors are found by examining the correlations between different altmetric indicators; how does a Mendeley bookmark correlate to a citation, or to a Facebook share? (And so on.) What can groups of correlations tell us about the uses of scholarship?

Among the flavors the researchers identified were a “popular hit” flavor (where scholarship is highly tweeted and shared on Facebook, but not seen much on scholarly sites like Mendeley or in citations) and an “expert pick” flavor (evidenced by F1000 Prime ratings and later citations, but few social shares or mentions). Lutz Bornmann’s 2014 study built upon that work, documenting that articles that are tagged on F1000 Prime as being “good for teaching” had more shares on Twitter–uncovering possible uses among educational audiences.

The correlation that’s on everyone’s mind? How do social media (and other indicators) correlate with citations? Mendeley bookmarks are found to have the most correlations with citations; this points to Mendeley’s use as a leading indicator (that is, if something is bookmarked on Mendeley today, it’s got better chance of being cited down the road than something that’s not bookmarked).

Correlations with citations aren’t the only correlations we should pay attention to, though. They only tell one part of an impact story–an important part, to be sure, but not the only part.

Altmetrics data includes qualitative data, too

Many don’t realize that altmetrics data isn’t only about the numbers. An important function of altmetrics aggregators like Altmetric.com and Impactstory (which we describe in more detail below) is to gather qualitative data from across the web into a single place, making it easy to read exactly what others are saying about your scholarship. Altmetric.com does this by including snippets of the blogs, tweets, and other mentions your work receives online. Impactstory links out to the data providers themselves, allowing you to more easily find and read the full-length mentions from across the web.

Altmetrics for Open Science

Now that you have an understanding of how altmetrics work in general, let’s talk about how they work for you as an Open Scientist. Below, we’ve listed some of the basic metrics you can expect to see on the scholarship that you make Open Access. We’ll discuss how to find these metrics in the next section.

Metrics for all products

Any scholarly object that’s got a URL or other permanent identifier like a DOI–which, if you’re practicing Open Science, would be all of them–can be shared and discussed online.

So, for any of your scholarly outputs that have been discussed online, you can expect to find Twitter mentions, blog posts and blog comments, Facebook and Google+ shares and comments, mainstream media mentions, and Wikipedia mentions.

Open Access Publications

Your open access publications will likely accrue citations same as your publications that appear in subscription journals, with two key differences: you can track citations to work that isn’t formally published (but has instead been shared on a preprint server like ArXiv or other such repository) and you can track citations to work that appear in non-peer reviewed literature. Citation indices like Scopus and Web of Science can help you track the former. Google Scholar is a good way to find citations in the non-peer reviewed literature.

Views and downloads can be found on some journal websites, and often on repositories–whether your university’s institutional repository, a subject repository like BioRXiv, or a general purpose repository like Figshare.

Screen Shot 2014-10-22 at 4.16.36 PM.png

Bookmarks on reference management services like Mendeley and CiteULike can give you a sense of how widely your work is being read, and by what audiences. Mendeley, in particular, offers excellent demographic information for publications bookmarked in the service.

Software & code

Software & code, like other non-paper scholarly products, are often shared on specialized platforms. On these platforms, the type of metrics your work receives is often linked to the platform itself.

SourceForge blazed the trail for data metrics by allowing others to review and rate code–useful, crowd-sourced quality indicators.

On GitHub, you can expect for your work to receive forks (which signal adaptations of your code), stars (a bookmark or virtual fistbump that lets others tell you, “I like this”), pull requests (which can get at others’ engagement with your work, as well as the degree to which you tend to collaborate), and downloads (which may signal software installations or code use). One big advantage to using GitHub to share your code is that it allows you to mint DOIs–making it much easier to track mentions and shares of your code in the scholarly literature and across general purpose platforms, like those outlined above.

Data

Data is often cited in one of two ways: citations to data packages (the dataset itself, stored on a website or repository) and citations to data papers (publications that describe the dataset in detail, and that link out to the dataset). You can often track the former using an altmetrics aggregator (more on that in a moment) or the Data Citation Index, a database that’s similar to Web of Science which searches for mentions of your dataset in the scholarly literature. Citations to data papers can sometimes be found in traditional citation indices like Scopus and Web of Science.

Interest in datasets can also be measured by tracking views and downloads. Often, these metrics are shared on repositories where datasets are stored.

Where data is shared on GitHub, forks and stars (described above) can give an indication of that data’s reuse.

More info on metrics for data can be found on my post for the e-Science Portal Blog, “Tracking the Impacts of Data–Beyond Citations”.

Videos

Videos are created by many researchers to summarize a study for generalist audiences. Other times, videos are a type of data.

YouTube tracks the most varied metrics: views, likes, dislikes, and comments are all reported. On Vimeo and other video sharing sites, likes and views are the most often reported metrics.

Slide decks & posters

Slide decks and posters are among the scholarly outputs that get the least amount of love. Once you’ve returned from your conference, you tend to shelve and forget about the poster that you (or your grad students) have put hours worth of work into–and the same goes for the slide decks you use when presenting.

If you make these “forgotten” products available online, on the other hand, you can expect to see some of the following indicators of interest in your work: views, favorites (sometimes used as a bookmark, other times as a way of saying “job well done!”), downloads, comments, and embeds (which can show you how often–and by whom–your work is being shared and in some cases blogged about).

How to collect your metrics from across the Web

We just covered a heck of a lot of metrics, huh? Luckily, altmetrics aggregators are designed to collect these far-flung data points from across the web and deliver them to you in a single report.

There are three main independent altmetrics aggregators: Impactstory.org, PlumX, and Altmetric.com. Here’s the scoop:

  • Impactstory.org: we’re a non-profit altmetrics service that collects metrics for all scholarly outputs. Impactstory profiles are designed to meet the needs of individual scientists. We regularly introduce new features based on user demand. You can sign up for a 30-day free trial on our website; after that, subscriptions are $10/month or $60/year.

  • PlumX: a commercial service that is designed to meet the needs of administrators and funding agencies. Like Impactstory, PlumX also collects metrics for all scholarly outputs. PlumX boasts the largest data coverage of all altmetrics aggregators.

  • Altmetric.com: a commercial service that collects metrics primarily for publishers and institutions. Altmetric can track any scholarly output with a DOI, PubMed ID, ArXiv ID, or Handle, but it does publications the best. Uniquely, they can find mentions to your scholarship in the mainstream media mentions and policy documents–two notoriously hard to mine locations.

Once you’ve collected your metrics from across the web, what do you do with them? We suggest experimenting with using them in your CV, year-end reporting, grant applications, and even tenure & promotion dossiers.

Skeptical? You needn’t be. An increasing number of scientists are using altmetrics for these purposes.

Researchers who have used altmetrics for tenure & grants

Each of the following researchers used altmetrics, alongside traditional metrics like citation counts and journal impact factors, to document the impact of their work.

Tenure: Dr. Steven Roberts, University of Washington

Steven-Roberts1-528x528.jpgSteven is an Associate Professor in the School of Aquatic & Fishery Sciences at the University of Washington. He decided to use altmetrics data in his tenure dossier to two ends: to showcase his public engagement and to document interest in his work.

To showcase public engagement, Steven included this table in the Education and Outreach section of his dossier, illustrating the effects his various outreach channels (blog, Facebook, Flickr, etc) have had to date:

Screen Shot 2014-10-20 at 2.19.52 PM.png

For evidence of the impact of specific products, he incorporated metrics into his CV like this:

Screen Shot 2014-10-20 at 2.24.04 PM.png

Screen Shot 2014-10-20 at 2.25.35 PM.png

Steven’s bid for tenure was successful.

Want to see more? You can download Steven’s full tenure dossier here.

Tenure: Dr. Ahmed Moustafa, American University in Cairo

ahmed.jpgAhmed’s an Associate Professor in the Department of Biology at American University in Cairo, Egypt.

He used altmetrics data in his tenure dossier in two interesting ways. First, he included a screenshot of his most important scholarly products, as they appear on his Impactstory profile, to summarize the overall impacts of his work:

Screen Shot 2014-10-20 at 2.52.15 PM.png

Note the badges that summarize in a glance the relative impacts of his work among both the public and other scholars. Ahmed also includes a link to his full profile, so his reviewers can drill down into the impact details of all his works, and also review them for themselves.

Ahmed also showcased the impact of a particular software package he created, JAligner, by including a link to a Google Scholar search that showcases all the scholarship that cites his software:

As of August 2013, JAligner has been cited in more than 150 publications, including journal articles, books, and patents, (http://tinyurl.com/jalignercitations) covering a wide range of topics in biomedical and computational research areas and downloaded almost 20,000 times (Figure 6). It is somehow noteworthy that JAligner has claimed its own Wikipedia entry (http://en.wikipedia.org/wiki/JAligner)!

Ahmed received tenure with AUC in 2013.

Grant Reporting: Dr. Holly Bik, University of Birmingham

0167.pngHolly was awarded a major grant from the Alfred P. Sloan Foundation to develop a bioinformatics data visualization tool called Phinch.

When reporting back to Sloan on the success of her project, she included metrics like the Figshare views that related posters and talks received, Github statistics for the Phinch software, and other altmetrics related to the varied outputs that the project created over the last few years.

Holly’s hopeful that these metrics, in addition to the traditional metrics she’s reported to Sloan, will make a great case for renewal funding, so they can continue their work on Phinch.

Will altmetrics work for you?

The remarkable thing about each of these researchers is that their circumstances aren’t extraordinary. The organizations they work for and receive funding from are fairly traditional ones. It follows that you, too, may be able to use altmetrics to document the impacts of your Open Science, no matter where you work or are applying for funding. After all, more and more institutions are starting to incorporate recognition of non-traditional scholarship into their tenure & promotion guidelines. You’ll need non-traditional ways like altmetrics to showcase the impacts of that scholarship.