Science 2.0 study

Updates on progress and discussions on results of Science 2.0: implications for European policies on research and innovation study

New ways to evaluate scientists

The main bottleneck to the adoption of science 2.0 paradigm is the lack of recognition. The career of the scientists is determined by papers, articles and citations, and there is no recognition for releasing data, code, or laboratory notebook. Moreover, effective reputation management tools can have a key role in finding the right micro-expertise to involve in large scale collaborative efforts.

As GrrlScientist puts it:

If there is no way to ensure that scientists get credit for their ideas and intellectual contributions, then they will not contribute to the Open Science movement. Traditionally, the way that credit has been assigned to scientists has been through publication of their data in peer-reviewed journals and by citing their colleagues’ work in their papers.

Michal Nielsen recognizes this as well in his book.

In our paper, we point out to the possibility of creating new ways of managing reputation, such as the Open Source example of IBM.

In our study, we’re trying to look for actual implementation of reputation management for scientist. So far we’ve come across:

- PeerEvaluation, a service which helps scientists share their data and papers and thereby measure their reputation

- Altmetrics, a service which maps the reputation of scientists by monitoring how people use their papers on CiteUlike, Menedeley, Zotero

This is very much related and overlapping with alternative ways to do peer-review, such as F1000.

However, these services remain highly experimental and there is little data about how they are used. Do you have any evidence of uptake and impact of alternative ways to evaluate scientists?

About these ads

Single Post Navigation

5 thoughts on “New ways to evaluate scientists

  1. I also found this relevant video from Strata 2011
    http://shop.oreilly.com/product/0636920019145.do

  2. You might be interested in Wouters and Costas’ paper, “Users, narcissism and control – tracking the impact of scholarly publications in the 21st century,” which investigates a wide array of altmetrics tools. Also, if you’re interested in alternative peer review systems, check out this collaboratively-gathered list of such tools: https://docs.google.com/document/d/1HD-BEaVeDdFjjCNFkb0j3pvwe7MrP3PtE-bWHkkdq7Q/edit

  3. Jason this is VERY useful thanks.
    I am particularly interested in reputation system that reward openness in terms of data sharing, code sharing, blogging etc. Any idea?

  4. Very interesting blog entry on reputation management in science.
    http://scholarlykitchen.sspnet.org/2012/05/18/the-black-market-for-facebook-likes/
    Notably, it cites an idea of Arjen Wais, professor at Wageningen University in the Netherlands that writes about a possible online marketplaces for trading citation to upgrade hFactor.
    Also, in view of this blog entry, how would you comment, Jason, on Phil Davis review of altmetrics pointing out how simple may it be to game downloads stats or tweets?

  5. Pingback: The emerging institutional setting of Science 2.0 « Science 2.0 study

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: