“The massive impact of science on our collective and individual lives has decreased the willingness of many to accept the pronouncements of scientists unless they can verify the strength of the underlying evidence for themselves. [...] It is vital that science is not seen to hide behind closed laboratory doors, but engages seriously with the public.”—
Open your minds and share your results, says Geoffry Boulton, asking that scientists make data available to the public and to other researchers, because “Science’s capacity for self-correction comes from this openness to scrutiny and challenge”.
Science as an open enterprise is a report from the Royal Society that highlights 6 main changes needed to improve the openess of science:
- “a shift away from a research culture where data is viewed as a private preserve;
- expanding the criteria used to evaluate research to give credit for useful data communication and novel ways of collaborating;
- the development of common standards for communicating data;
- mandating intelligent openness for data relevant to published scientific papers;
- strengthening the cohort of data scientists needed to manage and support the use of digital data;
- the development and use of new software tools to automate and simplify the creation and exploitation of datasets.”
“If you set out to solve a problem, there's no guarantee you will succeed, [...] But different people have different aptitudes and they know different tricks… it turned out their combined efforts can be much quicker.”—
The concept of open science is really interesting.
I’ve been at Science Online 2011 in London this weekend. One hot topic of conversation during Day One was science blogging and how it relates to science publishing in the form of journals.
There was much hand-wringing yesterday, during a panel discussion on the ‘Arsenic Life’ story (see these links), where science bloggers seemed exasperated by the fact that what they write in blogs is not linked with the research they discuss. After all they write some great stuff and it would be great if anyone reading the paper could read their dire warnings about the reliability of the conclusions*. At moments some on the the panel and in the audience even seemed to get close to suggesting that their blog posts should be placed on a level with peer-reviewed publications.
The ‘Arsenic Life’ tale is one where several things than could go wrong, did go wrong. The results were over-hyped, the scientists were unresponsive to criticism and the peer-review system broke down. However the vast majority of scientific results are reported pretty well and without such catastrophic failures of the system. Did blogging help? Yes and it may have been instrumental in bringing the issues to light. The scientists who wrote about the story on their blogs, did so in a journalistic act, not a scientific one. They are free to publish rebuttal papers and get a peer-reviewed response into the literature in due course. I’d be keen to know if anyone is doing this.
I think what many science bloggers forget is that they represent the very thin end of the science-blogging wedge. There are many research scientists out there writing blogs about their work and that of others. They share that part of the blogosphere with many more science bloggers who are non-scientists. Many interested amateurs and members of the public are writing about science and some are doing a very good job of it. (Some researchers do a bad job it too, by the way).
Whilst I agree with many that we should move to a more open and transparent publication process in academia, I don’t believe that blogging should be part of it - certainly not in its current form. Blogging represents a free and liberating way to share ideas and thoughts. It is unencumbered by regulation and this is exactly why I think many scientists enjoy it and find it useful. Perhaps one tantalising aspect of science blogging is that it feels like scientific lab reporting for many people - but it isn’t. You may chose to write your blog with all the rigour and finesse of a publishable work but it is still a blog. I suppose it comes down to trust and verifiability.
One can imagine ways to legitimise and promote blogging into a state closer to the academic model (without turning it into journalism). Something more akin to social networking than peer-review seems like a good idea. I would point anyone thinking about these ideas to the Research Blogging network which is collecting blogs about peer-reviewed research. Perhaps a blog journal, with editors and peer-review would be viable - does such a thing exist?
Science blogging is growing but the credibility of the few should not be used to elevate the many non-professional science blogs to a recognised, academic status. Science bloggers are doing something great: they are providing insight into the way science works and telling a more narrative story about their results and their field of work.
Another of yesterday’s sessions was about storytelling (It was hosted by @BoraZ and @mistersugar) - one theme emerging was that more scientists need to tell stories to help engage people and this is a crucial point. The science bloggers are acting outside of the scientific process and telling their own stories. This is a great thing to do and it doesn’t need to get incorporated into that process. It is great because it is distinct and unrestrained. I say let the science blogging continue on all sides, and in all forms and leave it separate from the process of peer-review and publication. there is no need to further muddy those waters.
*The idea of using trackbacks to allow bloggers to connect with things they discuss is not new, and you can in fact trackback to papers (e.g. on the arXiv: http://arxiv.org/help/trackback) if you blog about them.
“During the introduction [at BioBarCamp] many people expressed an interest in “Open Science”, “Open Data”, or some other open stuff, yet it was already pretty clear that many people meant many different things by this. I think for me the most striking outcome of [a session to define it] was that not only is this a radically new concept for many people but that many people don’t have any background understanding of open source software either which can make the discussion totally impenetrable to them. This, in my view strengthens the need for having some clear brands, or standards, that are easy to point to and easy to sign up to (or not).”—Science Commons » Blog Archive » What’s open science?
A proposal for a really fast statistics journal
I know we need a new journal like we need a good poke in the eye. But I got fired up by the recent discussion of open science (by Paul Krugman and others) and the seriously misguided Research Works Act- that aimed to make it illegal to deposit published papers funded by the government in Pubmed central or other open access databases.
I also realized that I spend a huge amount of time/effort on the following things: (1) waiting for reviews (typically months), (2) addressing reviewer comments that are unrelated to the accuracy of my work - just adding citations to referees papers or doing additional simulations, and (3) resubmitting rejected papers to new journals - this is a huge time suck since I have to reformat, etc. Furthermore, If I want my papers to be published open-access I also realized I have to pay at minimum $1,000 per paper.
So I thought up my criteria for an ideal statistics journal. It would be accurate, have fast review times, and not discriminate based on how interesting an idea is. I have found that my most interesting ideas are the hardest ones to get published. This journal would:
- Be open-access and free to publish your papers there. You own the copyright on your work.
- The criteria for publication would be: (1) it has to do with statistics, computation, or data analysis, (2) is the work is technically correct.
- We would accept manuals, reports of new statistical software, and full length research articles.
- There would be no page limits/figure limits.
- The journal would be published exclusively online.
- We would guarantee reviews within 1 week and publication immediately upon review if criteria (1) and (2) are satisfied
- Papers would receive a star rating from the editor - 0-5 stars. There would be a place for readers to also review articles
- All articles would be published with a tweet/like button so they can be easily distributed
To achieve such a fast review time, here is how it would work. We would have a large group of Associate Editors (hopefully 30 or more). When a paper was received, it would be assigned to an AE. The AEs would agree to referee papers within 2 days. They would use a form like this:
- Review of: Jeff’s Paper
- Technically Correct: Yes
- About statistics/computation/data analysis: Yes
- Number of Stars: 3 stars
- 3 Strengths of Paper (1 required):
- This paper revolutionizes statistics
- 3 Weakness of Paper (1 required):
- * The proof that this paper revolutionizes statistics is pretty weak
- because he only includes one example.
That’s it, super quick, super simple, so it wouldn’t be hard to referee. As long as the answers to the first two questions were yes, it would be published.
So now here’s my questions:
- Would you ever consider submitting a paper to such a journal?
- Would you be willing to be one of the AEs for such a journal?
- Is there anything you would change?