Some post #MarchforScience thoughts.

See the bottom of this post for a collection of great #MarchForScience tweets, images, and my livestream of the London march!

Foremost, thanks to everyone who came out and stood up to show their support. I think it is hard not to look at the worldwide crowds and feel an up-welling of pride and hope. If nothing else the feeling of solidarity, and of sending a loud message that we will not accept a post-evidence society, is well worth the efforts of the organizers and marchers. I just wanted to try and write down a few thoughts I had about the marches, which I’m sure are shared by many others.

Yesterday I think many of us saw, first hand and for the first time, that Science has real people power. Like any other special interest group, we can band together and organize to amplify the reach and influence of our message. Ultimately science requires the creation of a space that is free from politics, and the creation of that space is itself a political act. It is my hope that yesterday planted the seeds of organization that can grow into a movement. We can’t except the general public to stand up for us; it is indeed time to work to ensure a society where science and evidence-based policy flourish.

That being said, I’m sure many of you are also wondering what, if anything yesterday will really achieve. I also have the worry that these marches may ultimately act as another form of ‘slacktivism’, exorcising our anxieties while ultimately achieving little. I can’t speak for the worldwide marches, but I did feel that more could have been done to try and carry the momentum forward. It is a bold first step for scientists to put aside their self-assumed neutrality and stand up for their own cause. At the London march you could feel an almost palpable unease or cautiousness in the march yesterday. It was perhaps the most quiet, calm, and reserved political march i’ve ever participated in – and of course, also a lot of fun. Ultimately if we are going to effect change, this can only be the first step. We need to begin to organize into effective political action communities that can lobby on our behalf.

This also means addressing some of the infighting that arose during the course of the organization of the march. Science cannot turn a blind eye to diversity, or our own issues therein. Effective political action requires building a broad based progressive movement that is inclusive and champions a set of values that does not exclude persons of color, LGBT, or other minorities. I recognize that there are already growing pains; many scientists feel science should inherently be apolitical. But what we’ve seen is that, our work will be politicized no matter what stance we take on it. My hope is that the marches yesterday will embolden us to reach out to community organizers, to build a strong and evidence based movement for political reform. Let yesterday be the planting of a seed, from which a thousand flowers may bloom.


Here are some fun tweets and links from the march:

Clearly DC scientists had a blast! Love this video.

Amazing turn out in Seattle:

Sine game on point:

DC:

Truth!

Amazing aerial shot:

20k marchers in Philly!

Hello, my name is Science!

A Researcher’s Guide to the #Resistance

Note; the bottom of this post will be continuously updated with resources and action links. Please add any useful resources in the comments, to be added to the list.

 

This topic needs no introduction; if you are not already aware of the crisis and political turmoil I’m not sure this document could reach you anyway. This is for the woke scientist, scholar, and other academics ready to fight fear with resistance. I’m not exactly sure how to best arrange this document but it must be written. My goal is less to review the state of affairs, of which I’m sure you are aware, but rather to provide concrete tips and guidelines so that you can break free from ‘oh-dearism’ and leap into action.

With that in mind, lets break this down into a few sections:

  1. No action is too small.

On the progressive left, particularly among intellectuals, we have a history of infighting over which action is the best action. While I think there is merit to our culture of critical thought and inquiry in a democratic society, we are now past the time where such a response is enough. Fascism is at our doorstep and we must organize together any coalition of those willing to rise up. This means that no act of resistance is too small. I know from my interactions that the majority of academics are seriously worried about what they see in the daily news. We know that democratic principles are under serious threat, but we’re unsure of how to respond. The academic life is a rat race, and few feel ready to dedicate hours of their day to a cause whose efficacy of which they are unsure.

But this is exactly the attitude that our opponents are counting on. They are happy for us to share the latest outrages within our filter bubble, knowing we are too caught up in our daily lives to translate that outrage into action. With this in mind, it is imperative that we embrace any action. Intellectuals and academics have important skills to contribute to resistance; as the tenders and growers of knowledge we have a social duty to speak out against fascism. What we face now is nothing less than an existential threat to our craft and culture.

It is with this in mind that I earnestly beseech my colleagues and collaborators to cast aside doubt and one-upmanship. In times like these, you can’t condition your voice and action on the probability of success. Instead we need to organize together and present a unified resistance to Trumpism.

Of course, we have all busy lives to attend to. Our professional and personal commitments do not pause as we attend to Democracy. So I urge everyone to seek out the causes and actions with lie closest to their home and heart. Don’t waste time asking if your action is likely to succeed. If you feel strongly about the gagging of scientists, then join a pro-science march. Write to your MPs and professional societies asking them to publicly denounce such activities. Whatever the causes – women’s rights, social justice, the abuse and discrimination of people of color – leap into action with whatever help you can give. These movements need your talents; they need your thought, your code, your data, your critical thought and skills at debate. Your ability to communicate complex ideas in useful packages. They need these things as much if not more than your time and money – although you should also not hesitate to join the common foot soldier in standing up. Everyone’s rights are under threat, which brings me to my next point – the resistance must blossom everywhere.

  1. Neo-fascism is a global movement; the resistance must also be

Make no mistake; the current wave of authoritarian politics is not constrained to any single nation. It neither begins nor ends with Trump. Certainly in Europe there are far right movements springing up like weeds in every corner of the continent. As such the movement to resist must also be global. We must show our politicians that an injustice anywhere is an injustice everywhere, and that we will not stand for appeasement.

In eras gone past, the academic ‘intelligentsia’ played a critical role in shaping democratic reform movements. Such movements require leadership, critical thought, writing, and other such skills, which are the tradecraft of the academic. Today globalism has spread our kind far and wide around the globe. Most of us have too few local ‘resistance’ contacts. Do you know who your local union leaders are? Or where to go to work with women’s rights groups? Many of us are expatriated from our home countries, and may feel unsure if it is our place to fight in local political movements. As an American, should I join ant-Brexit movements? Should I vote in local politics? Or is this an intrusion?

The global nature of today’s oppression means that our resistance must also be global. This is tricky because ultimately we can all have the most influence in our local communities. If we are to unite together and defend Democracy, we must overcome our isolation and build a global movement with one cause celebre’; the defense of freedom. We must connect our global ties with our local leaders, to rouse the slumbering giant of the concerned majority. To do so it is vitally important that academics, scientists, and researchers reach out to their local communities and work to build politically active networks.

  1. We must organize and unite.

To overcome this isolation, which breeds inaction, we must organize. Today’s academic is woefully isolated. Many of us have moved continuously from state to state. This means that our local networks are typically impoverished, but our global networks are quite rich. We must shore up this weakness, while also capitalizing on our strengths. This means we need to start talking amongst ourselves, in the workplace and outside it. Don’t just stand at the water cooler saying ‘oh dear, it’s really all quite terrible, isn’t it’? This only contributes to the feeling of paralysis. Instead, take literally any action that builds a community around you. Organize a local action group at your university. Build a Facebook group around your international network. Find a cause that excites you and dedicate an hour or two a week to working with them. Go to marches! Marches are an important way to build community. Once you find a group of people dedicated to the resistance, offer your services to them. They are likely to be desperate for people with professional skills.

  1. Any resistance is effective resistance

Now you have probably already asked, but what can I really do? Will it really matter? I know how easy it is to fall into despair. After Brexit and the Trump election, I felt a deep darkness as never before. The triumph of fascism and radical capitalism seemed inevitable. I wallowed in self-pity, watching the unending tide of bad news, shouting ‘I told you so!’. This is totally ineffective and will make you and your colleagues feel horrible.

Here is the thing. Resistance is not about winning and losing. It’s about standing up for a moral cause; about drawing a line in the sand and saying: “here I will go no futher”. To paraphrase MLK Jr; “if you do nothing out of fear, then you are already dead inside”. History will judge us for the action we take in the coming weeks and months. Are you going to wait until someone drags a friend or coworker out of bed? Would you be more likely to stand up and fight knowing that the boots of fascism have been just past your door?

Now is the time to stand. And as an academic you have many ways to fight. Chances are you’ve been through at least a decade of advanced training in skills which are vital to any democratic movement. So shake off the chains of defeatism and DO SOMETHING! Your resistance, no matter how small or focused, sends a message to your friends and colleagues. It tells the oppressed that no, you will not stand idly by as they are persecuted. Trust me; you will sleep better and breath more easily with each and every action you take.

  1. Self-Care

As a researcher/scientist/academic, it is likely that you were already on the edge of burn out before our world imploded. For your resistance to be sustained, it must be self-nurturing. While marching and acting can be an effective way to retain a feeling of control, it must also be moderated by self-care and practical constraints. This means regulating your information intake and being disciplined about how and when you resist.

A few practical tips; reward yourself for effective action. If you go to a march, or write an essay on the evils of fascism, also take time out to relax. Read a good book, play some videogames, go for a walk. Take time to remember what it is you are fighting for. This goes double for social media. By this point is probably clear that Twitter, Facebook, and similar outlets are going to be a never ending stream of bad news, as well as an organizing hub for the resistance. While it’s vital that you participate in these forums and remain well-informed, you can also easily burn yourself out. Set specific times of the day when you take in the latest news and social media, and other times when you turn off these inputs and work on your own things. Again, if you are engaged in concrete action, there is no reason to feel guilty about taking time for yourself and your work.

  1. Use your time wisely – do not feed the trolls

While it might be an effective way to blow off steam, I recommend avoiding the pro-Trump/Brexit/Le Pen trolls entirely. I know this isn’t easy for most of us, as we want to believe that free debate and the exchange of ideas can solve most of the world’s problems. The issue is that, we are also fighting an unprecedented information war. Remember that 2.5 million more people voted for Hillary than for Trump. We have the moral high ground here; we’re fighting against fascism, and they are fighting for it. Not only is it unnecessary to convince these people, it is almost certainly impossible. What we need now to is build an effective resistance; the authoritarians will either realize the error of their ways and join us, or be judged by history accordingly. And the sad truth is, many of these accounts are likely fake, ‘astroturfed’ trolls being paid to support the radical right agenda. It just isn’t worth your time and energy; research suggests arguing with these people may actually strengthen their resolve. This also applies to the far-left; those who voted for Jill Stein because Hillary was ‘the same as Trump’. We need to be focused on turning out the moderate, silent majority, who are appalled at what they see in the news but have no idea how to stop it.

  1. Concrete action

 Hopefully by now you are on your feet, ready to act. So what CAN you do? First, you need to choose a domain of resistance. The best thing you can do is to find forms of sustainable resistance. You can’t go out and lose your job; this only reduces the longevity and depth of your possible action. The first step to effective action is therefor to select a cause, which is geographically and morally closest to you and your heart. This will help you build local roots and a community from which to grow your action. It will keep you motivated and prevent the tendency towards defeatism. We are all only human; for a resistance to be sustained it must come from a wellspring of the heart. It should enrich and grow your wellbeing, not sacrifice it[1].

With that in mind, here are some concrete ideas for how you can best resist:

  • Write. As an academic, you likely have a talent for thoughtful and persuasive writing. Write letters to your local newspaper, your political representative, on your blog, on facebook. Don’t just spread alarmism; state with force your opposition to concrete policies. Advocate for clear and decisive action. Lobby your representatives frequently and let them know that you and your colleagues will be voting and donating in kind. If you are an academic of prestige or status, then don’t leave that part out. Use your voice to provide the movement with clear and concrete leadership.
  • Call. Right now, go find a phone number for your local representative. If you are in the US, it’s important that you call your specific reps even if they are not from your party. It is important not just to call once, but also to call repeatedly. Set a schedule to make a phone call once a week, to give your representative an earful. If they are democrats, insist that they refuse to give a single inch to the GOP. If they are GOP, let them know loudly that you will disagree with their actions. Before calling, consider reading this excellent guide which can help you understand how to make your voice heard most effectively. Calling and/or writing to your representative is an easy way you can make a difference, and it doesn’t need to take more than 15 minutes a week.
  • Organize. Reach out to your local movement of choice, and get out there and help them. It isn’t enough to just tweet and share. These groups are desperate for help in a variety of ways, at least some of which you are probably skilled at. They need slogans, leadership, debate, and good writing copy. You could for example dedicate one week a month to lending some expertise to these groups. At the very least; march. It shows solidarity and helps us all feel less alone. Don’t forget to also write and lobby your existing scientific organizations; if enough of us pressure our professional societies to take a stand, it can have a massive effect.
  • Code. Are you a data scientist? A web developer? A social media socialite? This is the 21st century. Our resistance doesn’t have to just take the form of just calls and letters. Chances are your technical skills are in high demand. Web apps to organize; OPSEC documents to protect activist privacy; data science to analyze and optimize resistance. Our opponents are winning in part because they are using data science and social media to overwhelm traditional outlets. Your ability to process, analyze, interpret, or communicate data can be invaluable.
  • Donate. Choose at lease one professional organization and consider making a monthly, recurring donation. My choice is the ACLU as they have already shown an ability to fight Trumpism in the judiciary. But there is no shortage of causes needing your help. Attenant to the above, consider also getting in contact with local organizations to see if you can offer concrete help.
  • Teach. As an academic, you have a lot of experience teaching to an audience. Within the boundaries of your university ethics, use that podium for good. Get your students to consider the ways they can become politically active. Chances are you university has local political clubs (e.g., anti-war, pro-privacy, environmental) who are in need of your sponsorship or leadership. If not, consider starting one. Your critical thought and rhetorical skills can help motivate the youth to the streets and polls.
  • Vote. There is still a chance to stop this at the ballot box. But only if we get out there and help opposition parties. We must stop infighting and start supporting politicians who resist. The tea party effectively stymied Obama, one of the most popular politicians in recent history, by implementing a simple, unified vision for resistance. Any politician who worked with Obama, they primaried. Any GOP member who voiced a strong opposition, they supported. We must adopt these techniques. Any politician who shows any hint of appeasement must be opposed. You should consider getting involved in your local political groups, to maximize your impact on your local MP/representative/etc. Start a facebook group of people you know, to make sure you are all voting in local and midterm elections. We must fight fire with fire, and the best way to do this is to democratically stymy pro-fascist movements from the ground up.
  • Science. As a scientist, just sharing your data with the public is an act of resistance. The authoritarians seek to control the very flow of information itself. Reaching out to share your data and scientific knowledge is thus a powerful form of resistance. Use whatever data you have to back up the movement. Remember that we must always keep the high ground of truth on our side.
  • Create. Use your creativity to write poetry, speeches, to paint pictures of resistance. Craft t-shirts and colourful poster art. A resistance is sustained by it’s art; in this time of need we must unleash our most creative instincts in the fight for democracy.

 

  1. The future, and closing thoughts.

Many of us likely feel some degree of guilt; how could we have let this happen? There is no doubt that we became so caught up on the daily machinery of productivity that we have become complacent. But it is never too late to act; so long as one remains free to do so. There will be a time for careful debate and self-incrimination. It speaks volumes that many of us are only now awakening to the dire situation of civilization. We owe people of color, gay and trans, immigrant and all oppressed an apology for our complacency. But do so standing side by side with them in the #resistance. Now we must act together, in hopes of retaining our freedom to dissent.

Ultimately, we must look with hope towards the future. No matter how dark the headlines come, we must know that we will stand together in solidarity. Each day look deep inside and stoke that fire of resistance; know that our goal must not only be to push back the darkness of fascism, but to stem the wound from which it arose. We must build a better society, together.


Resources for the Resistant Academic (continuously updated):

Organizations Worth Donating To – consider a recurring 5$ donation! That cup of coffee can help sustain the fight. 

https://www.aclu.org/

https://www.amnesty.org.uk/

https://www.rescue.org/

https://www.plannedparenthood.org/

http://earthjustice.org/

https://www.splcenter.org/

 

Staying safe when resisting:

https://greenandblackcross.org/guides/

https://www.resistancemanual.org/Resistance_Manual_Home

https://greenandblackcross.org/guides/key-advice/

https://www.vice.com/en_uk/article/a-practical-guide-to-political-protest

Surveillance Self-Defense:

https://ssd.eff.org/en

View story at Medium.com

Practical guides to resistance:

https://www.indivisibleguide.com/

http://www.salon.com/2016/11/23/robert-reich-how-to-resist-donald-trumps-first-100-days_partner/

https://www.resistancemanual.org/Resistance_Manual_Home

Weekly action Checklist:
http://jenniferhofmann.com/home/weekly-action-checklist-democrats-independents-republicans-conscience/

Excellent tool to organize daily calls to your representative:
https://5calls.org/

How to be your own light in an authoritarian crisis:
https://thecorrespondent.com/5696/were-heading-into-dark-times-this-is-how-to-be-your-own-light-in-the-age-of-trump/1611114266432-e23ea1a6

Rules for a Constitutional Crisis:

View story at Medium.com

A Trump Resistance Guide

http://olel.weebly.com/blog/trump-resistance-reference-guide

Impeaching the president – a primer:
http://scholarship.law.upenn.edu/cgi/viewcontent.cgi?article=3404&context=penn_law_review

An activists guide to exploiting the media

http://www.safecom.org.au/monbiot-media.htm

Useful media and tweets:

 

Data for Democracy – a great portal for data scientists who want to resist

https://twitter.com/data4democracy?lang=en

Upcoming Marches

The march for science:

https://www.facebook.com/marchforscience/

https://twitter.com/LDNsciencemarch

Huge anti-brexit march:

https://www.theguardian.com/politics/2017/jan/28/stop-brexit-campaign-biggest-uk-biggest-protest-march?CMP=share_btn_fb

Meet the scientists affected by the Muslim Travel Ban:
http://www.nature.com/news/meet-the-scientists-affected-by-trump-s-immigration-ban-1.21389

Resistance Leaders – must-follow voices:

https://twitter.com/sarahkendzior

https://twitter.com/JuddLegum

https://twitter.com/Kasparov63

Operational Security (OPSEC):

http://www.citylab.com/crime/2017/01/are-police-searching-inauguration-protesters-phones/514244/

 

A resistance playlist – for when you need some moral building resistance songs!

[1] Note that this only applies in some case. If the brown shirts are coming for your neighbor, you may have to choose between submission and self-sacrifice. It is important to decide now how you will act in this seemingly absurd, but not improbable scenario. But for now I urge you to take effective and sustainable action, for all is not yet lost.

Is Frontiers in Trouble?

Lately it seems like the rising tide is going against Frontiers. Originally hailed as a revolutionary open-access publishing model, the publishing group has been subject to intense criticism in recent years. Recent issues include being placed on Beall’s controversial ‘predatory publisher list‘, multiple high profile disputes at the editorial level, and controversy over HIV and vaccine denialist articles published in the journal seemingly without peer review. As a proud author of two Frontiers articles and former frequent reviewer, these issues compounded with a general poor perception of the journal recently led me to stop all publication activities at Frontiers outlets. Although the official response from Frontiers to these issues has been mixed, yesterday a mass-email from a section editor caught my eye:

Dear Review Editors, Dear friends and colleagues,

As some of you may know, Prof. Philippe Schyns recently stepped down from his role as Specialty Chief Editor in Frontiersin Perception Science, and I have been given the honor and responsibility of succeeding him into this function. I wish to extend to him my thanks and appreciation for the hard work he has put in building this journal from the ground up. I will strive to continue his work and maintain Frontiers in Perception Science as one of the primary journals of the field. This task cannot be achieved without the support of a dynamic team of Associate Editors, Review Editors and Reviewers, and I am grateful for all your past, and hopefully future efforts in promoting the journal.

It am aware that many scientists in our community have grown disappointed or even defiant of the Frontiers publishing model in general, and Frontiers in Perception Science is no exception here. Among the foremost concerns are the initial annoyance and ensuing disinterest produced by the automated editor/reviewer invitation system and its spam-like messages, the apparent difficulty in rejecting inappropriate manuscripts, and (perhaps as a corollary), the poor reputation of the journal, a journal to which many authors still hesitate before submitting their work. I have experienced these troubles myself, and it was only after being thoroughly reassured by the Editorial office on most of these counts that I accepted to get involved as Specialty Chief Editor. Frontiers is revising their system, which will now leave more time for Associate Editors to mandate Review Editors before sending out automated invitations. When they occur, automated RE invitations will be targeted to the most relevant people (based on keyword descriptors), rather than broadcast to the entire board. This implies that it is very important for each of you to spend a few minutes editing the Expertise keywords on your Loop profile page. Most of these keywords were automatically collected within your publications, and they may not reflect your true area of expertise. Inappropriate expertise keywords are one of the main reasons why you receive inappropriate reviewing invitations! In the new Frontiers system, article rejection options will be made more visible to the handling Associate Editor. Although my explicit approval is still required for any manuscript rejection, I personally vow to stand behind all Associate Editors who will be compelled to reject poor-quality submissions. (While perceived impact cannot be used as a rejection criterion, poor research or writing quality and objective errors in design, analysis or interpretation can and should be used as valid causes for rejection). I hope that these measures will help limit the demands on the reviewers’ time, and contribute to advancing the standards and reputation of Frontiers in Perception Science. Each of you can also play a part in this effort by continuing to review articles that fall into your area of expertise, and by submitting your own work to the journal.

I look forward to working with all of you towards establishing Frontiers in Perception Science as a high-standard journal for our community.

It seems Frontiers is indeed aware of the problems and is hoping to bring back wary reviewers and authors. But is it too little too late? Discussing the problems at Frontiers is often met with severe criticism or outright dismissal by proponents of the OA publishing system, but I felt these neglected a wider negative perception of the publisher that has steadily grown over the past 5 years. To get a better handle on this I asked my twitter followers what they thought. 152 persons responded as follows:

As some of you requested control questions, here are a few for comparison:

 

That is a stark difference between the two top open access journals – whereas only 19% said there was no problem at Frontiers, a full 50% say there is no problem at PLOS ONE. I think we can see that even accounting for general science skepticism, opinions of Frontiers are particularly negative.

Sam Schwarzkopf also lent some additional data, comparing the whole field of major open access outlets – Frontiers again comes out poorly, although strangely so does F1000:

These data confirm what I had already feared: public perception among scientists (insofar as we can infer anything from such a poll) is lukewarm at best. Frontiers has a serious perception problem. Only 19% of 121 respondents were willing to outright say there was no problem at the journal. A full 45% said there was a serious problem, and 36% were unsure. Of course to fully evaluate these numbers, we’d like to know the baserate of similiar responses for other journals, but I cannot imagine any Frontiers author, reviewer, or editor feeling joy at these numbers – I certainly do not. Furthermore they reflect a widespread negativity I hear frequently from colleagues across the UK and Denmark.

What underlies this negative perception? As many proponents point out, Frontiers has been actually quite diligent at responding to user complaints. Controversial papers have been put immediately under review, overly spammy-review invitations and special issue invites largely ceased, and so on. I would argue the issue is not any one single mistake on the part of Frontiers leadership, but a growing history of errors contributing to a perception that the journal is following a profit-led ‘publish anything’ model. At times the journal feels totally automated, within little human care given to publishing and extremely high fees. What are some of the specific complaints I regularly hear from colleagues?

  • Spammy special issue invites. An older issue, but at Frontier’s inception many authors were inundated with constant invites to special issues, many of which were only tangentially related to author’s specialties.
  • Spammy review invites. Colleagues who signed on to be ‘Review Editors’ (basically repeat reviewers) reported being hit with as many as 10 requests to review in a month, again many without relevance to their interest
  • Related to both of the above, a perception that special issues and articles are frequently reviewed by close colleagues with little oversight. Similiarly, many special issues were edited by junior researchers at the PhD level.
  • Endless review. I’ve heard numerous complaints that even fundamentally flawed or unpublishable papers are impossible or difficult to reject. Reviewers report going through multiple rounds of charitable review, finding the paper only gets worse and worse, only to be removed from the review by editors and the paper published without them.

Again, Frontiers has responded to each of these issues in various ways. For example, Frontiers originally defended the special issues, saying that they were intended to give junior researchers an outlet to publish their ideas. Fair enough, and the spam issues have largely ceased. Still, I would argue it is the build up and repetition of these issues that has made authors and readers wary of the journal. This coupled with the high fees and feeling of automation leads to a perception that the outlet is mostly junk. This is a shame as there are certainly many high-value articles in Frontiers outlets. Nevertheless, academics are extremely bloodshy, and negative press creates a vicious feedback loop. If researchers feel Frontiers is a low-quality, spam-generating publisher who relies on overly automated processes, they are unlikely to submit their best work or review there. The quality of both drops, and the cycle intensifies.

For my part, I don’t intend to return to Frontiers unless they begin publishing reviews. I think this would go a long way to stemming many of these issues and encourage authors to judge individual articles on their own merits.

What do you think? What can be done to stem the tide? Please add your own thoughts, and stories of positive or negative experiences at Frontiers, in the comments.

____

Edit:

A final comparison question

 

 

The Wild West of Publication Reform Is Now

It’s been a while since I’ve tried out my publication reform revolutionary hat (it comes in red!), but tonight as I was winding down I came across a post I simply could not resist. Titled “Post-publication peer review and the problem of privilege” by evolutionary ecologist Stephen Heard, the post argues that we should be cautious of post-publication review schemes insofar as they may bring about a new era of privilege in research consumption. Stephen writes:

“The packaging of papers into conventional journals, following pre-publication peer review, provides an important but under-recognized service: a signalling system that conveys information about quality and breath of relevance. I know, for instance, that I’ll be interested in almost any paper in The American Naturalist*. That the paper was judged (by peer reviewers and editors) suitable for that journal tells me two things: that it’s very good, and that it has broad implications beyond its particular topic (so I might want to read it even if it isn’t exactly in my own sub-sub-discipline). Take away that peer-review-provided signalling, and what’s left? A firehose of undifferentiated preprints, thousands of them, that are all equal candidates for my limited reading time (such that it exists). I can’t read them all (nobody can), so I have just two options: identify things to read by keyword alerts (which work only if very narrowly focused**), or identify them by author alerts. In other words, in the absence of other signals, I’ll read papers authored by people who I already know write interesting and important papers.”

In a nutshell, Stephen turns the entire argument for PPPR and publishing reform on its head. High impact[1] journals don’t represent elitism; rather they provide the no name rising young scientist a chance to have their work read and cited. This argument really made me pause for a second as it represents the polar opposite of almost my entire worldview on the scientific game and academic publishing. In my view, top-tier journals represent an entrenched system of elitism masquerading as meritocracy. They make arbitrary, journalistic decisions that exert intense power over career advancement. If anything the self-publication revolution represents the ability of a ‘nobody’ to shake the field with a powerful argument or study.

Needless to say I was at first shocked to see this argument supported by a number of other scientists on Twitter, who felt that it represented “everything wrong with the anti-journal rhetoric” spouted by loons such as myself. But then I remembered that in fact this is a version of an argument I hear almost weekly when similar discussions come up with colleagues. Ever since I wrote my pie-in-the sky self-publishing manifesto (don’t call it a manifesto!), I’ve been subjected (and rightly so!) to a kind of trial-by-peers as a de facto representative of the ‘revolution’. Most recently I was even cornered at a holiday party by a large and intimidating physicist who yelled at me that I was naïve and that “my system” would never work, for almost the exact reasons raised in Stephen’s post. So lets take a look at what these common worries are.

The Filter Problem

Bar none the first, most common complaint I hear when talking about various forms of publication reform is the “filter problem”. Stephen describes the fear quite succinctly; how will we ever find the stuff worth reading when the data deluge hits? How can we sort the wheat from the chaff, if journals don’t do it for us?

I used to take this problem seriously, and try to dream up all kinds of neato reddit-like schemes to solve it. But the truth is, it just represents a way of thinking that is rapidly becoming irrelevant. Journal based indexing isn’t a useful way to find papers. It is one signal in a sea of information and it isn’t at all clear what it actually represents. I feel like people who worry about the filter bubble tend to be more senior scientists who already struggle to keep up with the literature. For one thing, science is marked by an incessant march towards specialization. The notion that hundreds of people must read and cite our work for it to be meaningful is largely poppycock. The average paper is mostly technical, incremental, and obvious in nature. This is absolutely fine and necessary – not everything can be ground breaking and even the breakthroughs must be vetted in projects that are by definition less so. For the average paper then, being regularly cited by 20-50 people is damn good and likely represents the total target audience in that topic area. If you network to those people using social media and traditional conferences, it really isn’t hard to get your paper in their hands.

Moreover, the truly ground breaking stuff will find its audience no matter where it is published. We solve the filter problem every single day, by publically sharing and discussing papers that interest us. Arguing that we need journals to solve this problem ignores the fact that they obscure good papers behind meaningless brands, and more importantly, that scientists are perfectly capable of identifying excellent papers from content alone. You can smell a relevant paper from a mile away – regardless of where it is published! We don’t need to wait for some pie in the sky centralised service to solve this ‘problem’ (although someday once the dust settles i’m sure such things will be useful). Just go out and read some papers that interest you! Follow some interesting people on twitter. Develop a professional network worth having! And don’t buy into the idea that the whole world must read your paper for it to be worth it.

The Privilege Problem 

Ok, so lets say you agree with me to this point. Using some combination of email, social media, alerts, and RSS you feel fully capable of finding relevant stuff for your research (I do!). But your worried about this brave new world where people archive any old rubbish they like and embittered post-docs descend to sneer gleefully at it from the dark recesses of pubpeer. Won’t the new system be subject to favouritism, cults of personality, and the privilege of the elite? As Stephen says, isn’t it likely that popular persons will have their papers reviewed and promoted and all the rest will fade to the back?

The answer is yes and no. As I’ve said many times, there is no utopia. We can and must fight for a better system, but cheaters will always find away[2]. No matter how much transparency and rigor we implement, someone is going to find a loophole. And the oldest of all loopholes is good old human-corruption and hero worship. I’ve personally advocated for a day when data, code, and interpretation are all separate publishable, citable items that each contribute to ones CV. In this brave new world PPPRs would be performed by ‘review cliques’ who build up their reputation as reliable reviewers by consistently giving high marks to science objects that go on to garner acclaim, are rarely retracted, and perform well on various meta-analytic robustness indices (reproducibility, transparency, documentation, novelty, etc). They won’t replace or supplant pre-publication peer review. Rather we can ‘let a million flowers bloom’. I am all for a continuum of rigor, ranging from preregistered, confirmatory research with pre and post peer review, to fully exploratory, data driven science that is simply uploaded to a repository with a ‘use at your peril’ warning’. We don’t need to pit one reform tool against another; the brave new world will be a hybrid mixture of every tool we have at our disposal. Such a system would be massively transparent, but of course not perfect. We’d gain a cornucopia of new metrics by which to weight and reward scientists, but assuredly some clever folks would benefit more than others. We need to be ready when that day comes, aware of whatever pitfalls may bely our brave new science.

Welcome to the Wild West

Honestly though, all this kind of talk is just pointless. We all have our own opinions of what will be the best way to do science, or what will happen. For my own part I am sure some version of this sci-fi depiction is inevitable. But it doesn’t matter because the revolution is here, it’s now, it’s changing the way we consume and produce science right before our very eyes. Every day a new preprint lands on twitter with a massive splash. Just last week in my own field of cognitive neuroscience a preprint on problems in cluster inference for fMRI rocked the field, threatening to undermine thousands of existing papers while generating heated discussion in the majority of labs around the world. The week before that #cingulategate erupted when PNAS published a paper which was met with instant outcry and roundly debunked by an incredibly series of thorough post-publication reviews. A multitude of high-profile fraud cases have been exposed, and careers ended, via anonymous comments on pubpeer. People are out there, right now finding and sharing papers, discussing the ones that matter, and arguing about the ones that don’t. The future is now and we have almost no idea what shape it is taking, who the players are, or what it means for the future of funding and training. We need to stop acting like this is some fantasy future 10 years from now; we have entered the wild west and it is time to discuss what that means for science.

Authors note: In case it isn’t clear, i’m quite glad that Stephen raised the important issue of privilege. I am sure that there are problems to be rooted out and discussed along these lines, particularly in terms of the way PPPR and filtering is accomplished now in our wild west. What I object to is the idea that the future will look like it does now; we must imagine a future where science is radically improved!

[1] I’m not sure if Stephen meant high impact as I don’t know the IF of American Naturalist, maybe he just meant ‘journals I like’.

[2] Honestly this is where we need to discuss changing the hyper-capitalist system of funding and incentives surrounding publication but that is another post entirely! Maybe people wouldn’t cheat so much if we didn’t pit them against a thousand other scientists in a no-holds-barred cage match to the death.

Birth of a New School: PDF version and Scribus Template!

As promised, today we are releasing a copy-edited PDF of my “Birth of a New School” essay, as well as a Scribus template that anyone can use to quickly create their own professional quality PDF manuscripts. Apologies for the lengthy delay, as i’ve been in the middle of a move to the UK. We hope folks will iterate and optimize these templates for a variety of purposes, especially post-publication peer review, commentary, pre-registration, and more. Special thanks to collaborator Kate Mills, who used Scribus to create the initial layout. You might notice we deliberately styled the manuscript around the format of one of those Big Sexy Journals (see if you can guess which one). I’ve heard this elaborate process should cost somewhere in the tens of thousands of dollars per article, so I guess I owe Kate a few lunches! Seriously though, the entire copy-editing and formatting process only took about 3 or 4 hours total (most of which was just getting used to the Scribus interface), less than the time you would spend formatting and reformatting your article for a traditional publisher. With a little practice Scribus or similar tools can be used to quickly turn out a variety of high quality article types.

Here is the article on Figshare, and the direct download link:

Screen Shot 2013-12-12 at 11.50.42
The formatted manuscript. Easy!

What do you think? Personally, I’m really pleased with it! We’ve also gone ahead and uploaded the Scribus template to Figshare. You can use this to easily publish your own post-publication peer reviews, commentaries, and whatever else you like. Just copy-paste your own text into the text fields, replace the images, upload to Figshare or a similiar service, and you are good to go! In general Scribus is a really awesome open source tool for publishing, both easy to learn and cross platform. Another great alternative is Fidus. For now we’re still not exactly sure how to generate citations – in theory if you format your manuscripts according to these guidelines, Google Scholar will pick them up anywhere on the net and generate alerts. For now we are recommending everyone upload their self-publications to Figshare or a similar service, who are already working on a streamlined citation generation scheme. We hope you find these useful; now go out and publish some research!

The template:

An easy to use Scribus template for self-publishing
Our Scribus template, for quick creation of research proofs.

Birth of a New School: How Self-Publication can Improve Research

Edit: click here for a PDF version and citable figshare link!

Preface: What follows is my attempt to imagine a radically different future for research publishing. Apologies for any overlooked references – the following is meant to be speculative and purposely walks the line between paper and blog post. Here is to a productive discussion regarding the future of research.

Our current systems of producing, disseminating, and evaluating research could be substantially improved. For-profit publishers enjoy extremely high taxpayer-funded profit margins. Traditional closed-door peer review is creaking under the weight of an exponentially growing knowledge base, delaying important communications and often resulting in seemingly arbitrary publication decisions1–4. Today’s young researchers are frequently dismayed to find their pain-staking work producing quality reviews overlooked or discouraged by journalistic editorial practices. In response, the research community has risen to the challenge of reform, giving birth to an ever expanding multitude of publishing tools: statistical methods to detect p-hacking5, numerous open-source publication models6–8, and innovative platforms for data and knowledge sharing9,10.

While I applaud the arrival and intent of these tools, I suspect that ultimately publication reform must begin with publication culture – with the very way we think of what a publication is and can be. After all, how can we effectively create infrastructure for practices that do not yet exist? Last summer, shortly after igniting #pdftribute, I began to think more and more about the problems confronting the publication of results. After months of conversations with colleagues I am now convinced that real reform will come not in the shape of new tools or infrastructures, but rather in the culture surrounding academic publishing itself. In many ways our current publishing infrastructure is the product of a paper-based society keen to produce lasting artifacts of scholarly research. In parallel, the exponential arrival of networked society has lead to an open-source software community in which knowledge is not a static artifact but rather an ever-expanding living document of intelligent productivity. We must move towards “research 2.0” and beyond11.

From Wikipedia to Github, open-source communities are changing the way knowledge is produced and disseminated. Already this movement has begun reach academia, with researchers across disciplines flocking to social media, blogs, and novel communication infrastructures to create a new movement of post-publication peer review4,12,13. In math and physics, researchers have already embraced self-publication, uploading preprints to the online repository arXiv, with more and more disciplines using the site to archive their research. I believe that the inevitable future of research communication is in this open-source metaphor, in the form of pervasive self-publication of scholarly knowledge. The question is thus not where are we going, but rather how do we prepare for this radical change in publication culture. In asking these questions I would like to imagine what research will look like 10, 15, or even 20 years from today. This post is intended as a first step towards bringing to light specific ideas for how this transition might be facilitated. Rather than this being a prescriptive essay, here I am merely attempting to imagine what that future may look like. I invite you to treat what follows as an ‘open beta’ for these ideas.

Part 1: Why self-publication?

I believe the essential metaphor is within the open-source software community. To this end over the past few months I have  feverishly discussed the merits and risks of self-publishing scholarly knowledge with my colleagues and peers. While at first I was worried many would find the notion of self-publication utterly absurd, I have been astonished at the responses – many have been excitedly optimistic! I was surprised to find that some of my most critical and stoic colleagues have lost so much faith in traditional publication and peer review that they are ready to consider more radical options.

The basic motivation for research self-publication is pretty simple: research papers cannot be properly evaluated without first being read. Now, by evaluation, I don’t mean for the purposes of hiring or grant giving committees. These are essentially financial decisions, e.g. “how do I effectively spend my money without reading the papers of the 200+ applicants for this position?” Such decisions will always rely on heuristics and metrics that must necessarily sacrifice accuracy for efficiency. However, I believe that self-publication culture will provide a finer grain of metrics than ever dreamed of under our current system. By documenting each step of the research process, self-publication and open science can yield rich information that can be mined for increasingly useful impact measures – but more on that later.

When it comes to evaluating research, many admit that there is no substitute for opening up an article and reading its content – regardless of journal. My prediction is, as post-publication peer review gains acceptance, some tenured researcher or brave young scholar will eventually decide to simply self-publish her research directly onto the internet, and when that research goes viral, the resulting deluge of self-publications will be overwhelming. Of course, busy lives require heuristic decisions and it’s arguable that publishers provide this editorial service. While I will address this issue specifically in Part 3, for now I want to point out that growing empirical evidence suggests that our current publisher/impact-based system provides an unreliable heuristic at best14–16. Thus, my essential reason for supporting self-publication is that in the worst-case scenario, self-publications must be accompanied by the disclaimer: “read the contents and decide for yourself.” As self-publishing practices are established, it is easy to imagine that these difficulties will be largely mitigated by self-published peer reviews and novel infrastructures supporting these interactions.

Indeed, with a little imagination we can picture plenty of potential benefits of self-publication to offset the risk that we might read poor papers. Researchers spend exorbitant amounts of their time reviewing, commenting on, and discussing articles – most of that rich content and meta-data is lost under the current system. In documenting the research practice more thoroughly, the ensuing flood of self-published data can support new quantitative metrics of reviewer trust, and be further utlized in the development of rich information about new ideas and data in near real-time. To give just one example, we might calculate how many subsequent citations or retractions a particular reviewer generates, generating a reviewer impact factor and reliability index. The more aspects of research we publish, the greater the data-mining potential. Incentivizing in-depth reviews that add clarity and conceptual content to research, rather than merely knocking down or propping up equally imperfect artifacts, will ultimately improve research quality. By self-publishing well-documented, open-sourced pilot data and accompanying digital reagents (e.g. scripts, stimulus materials, protocols, etc), researchers can get instant feedback from peers, preventing uncounted research dollars from being wasted. Previously closed-door conferences can become live records of new ideas and conceptual developments as they unfold. The metaphor here is research as open-source – an ever evolving, living record of knowledge as it is created.

Now, let’s contrast this model to the current publishing system. Every publisher (including open-access) obliges researchers to adhere to randomly varied formatting constraints, presentation rules, submission and acceptance fees, and review cultures. Researchers perform reviews for free for often publically subsidized work, so that publishers can then turn around and sell the finished product back to those same researchers (and the public) at an exorbitant mark-up. These constraints introduce lengthy delays – ranging from 6+ months in the sciences all the way up to two years in some humanities disciplines. By contrast, how you self-publish your research is entirely up to you – where, when, how, the formatting, and the openness. Put simply, if you could publish your research how and when you wanted, and have it generate the same “impact” as traditional venues, why would you use a publisher at all?

One obvious reason to use publishers is copy-editing, i.e. the creation of pretty manuscripts. Another is the guarantee of high-profile distribution. Indeed, under the current system these are legitimate worries. While it is possible to produce reasonably formatted papers, ideally the creation of an open-source, easy to use copy-editing software is needed to facilitate mainstream self-publication. Innovators like figshare are already leading the way in this area. In the next section, I will try to theorize some different ways in which self-publication can overcome these and other potential limitations, in terms of specific applications and guidelines for maximizing the utility of self-published research. To do so, I will outline a few specific cases with the most potential for self-publication to make a positive impact on research right away, and hopefully illuminate the ‘why’ question a bit further with some concrete examples.

 Part 2: Where to begin self-publishing

What follows is the “how-to” part of this document. I must preface by saying that although I have written so far with researchers across the sciences and humanities in mind, I will now focus primarily on the scientific examples with which I am more experienced.  The transition to self-publication is already happening in the forms of academic tweets, self-archives, and blogs, at a seemingly exponential growth rate. To be clear, I do not believe that the new publication culture will be utopian. As in many human endeavors the usual brandism3, politics, and corruption can be expected to appear in this new culture. Accordingly, the transition is likely to be a bit wild and woolly around the edges. Like any generational culture shift, new practices must first emerge before infrastructures can be put in place to support them. My hope is to contribute to that cultural shift from artifact to process-based research, outlining particularly promising early venues for self-publication. Once these practices become more common, there will be huge opportunities for those ready and willing to step in and provide rich informational architectures to support and enhance self-publication – but for now we can only step into that wild frontier.

In my discussions with others I have identified three particularly promising areas where self-publication is either already contributing or can begin contributing to research. These are: the publication of exploratory pilot-data, post-publication peer reviews, and trial pre-registration. I will cover each in turn, attempting to provide examples and templates where possible. Finally, Part 3 will examine some common concerns with self-publication. In general, I think that successful reforms should resemble existing research practices as much as possible: publication solutions are most effective when they resemble daily practices that are already in place, rather than forcing individuals into novel practices or infrastructures with an unclear time-commitment. A frequent criticism of current solutions such as the comments section on Frontiers, PLOS One, or the newly developed PubPeer, is that they are rarely used by the general academic population. It is reasonable to conclude that this is because already over-worked academics currently see little plausible benefit from contributing to these discussions given the current publishing culture (worse still, they may fear other negative repercussions, discussed in Part 3). Thus a central theme of the following examples is that they attempt to mirror practices in which many academics are already engaged, with complementary incentive structures (e.g. citations).

Example 1: Exploratory Pilot Data 

This previous summer witnessed a fascinating clash of research cultures, with the eruption of intense debate between pre-registration advocates and pre-registration skeptics. I derived some useful insights from both sides of that discussion. Many were concerned about what would happen to exploratory data under these new publication regimes. Indeed, a general worry with existing reform movements is that they appear to emphasize a highly conservative and somewhat cynical “perfect papers” culture. I do not believe in perfect papers – the scientific model is driven by replication and discovery. No paper can ever be 100% flawless – otherwise there would be no reason for further research! Inevitably, some will find ways to cheat the system. Accordingly, reform must incentivize better reporting practices over stricter control, or at least balance between the two extremes.

Exploratory pilot data is an excellent avenue for this. By their very nature such data are not confirmatory – they are exciting in that they do not conform well to prior predictions. Such data benefit from rapid communication and feedback. Imagine an intuition-based project – a side or pet project conducted on the fly for example. The researcher might feel that the project has potential, but also knows that there could be serious flaws. Most journals won’t publish these kinds of data. Under the current system these data are lost, hidden, obscured, or otherwise forgotten.

Compare to a self-publication world: the researcher can upload the data, document all the protocols, make the presentation and analysis scripts open-source, and provide some well-written documentation explaining why she thinks the data are of interest. Some intrepid graduate student might find it, and follow up with a valuable control analysis, pointing out an excellent feature or fatal flaw, which he can then upload as a direct citation to the original data. Both publications are citable, giving credit to originator and reviewer alike. Armed with this new knowledge, the original researcher could now pre-register an altered protocol and conduct a full study on the subject (or alternatively, abandon the project entirely). In this exchange, it is likely that hundreds of hours and research dollars will have been saved. Additionally, the entire process will have been documented, making it both citable and minable for impact metrics. Tools already exist for each of these steps – but largely cultural fears prevent it from happening. How would it be perceived? Would anyone read it? Will someone steal my idea? To better frame these issues, I will now examine a self-publication practice that has already emerged in force.

 Example 2: Post-publication peer review

This is a particularly easy case, precisely because high-profile scholars are already regularly engaged in the practice. As I’ve frequently joked on twitter, we’re rapidly entering an era where publishing in a glam-mag has no impact guarantee if the paper itself isn’t worthwhile – you may as well hang a target on your head for post-publication peer reviewers. However, I want to emphasize the positive benefits and not just the conservative controls. Post-publication peer review (PPPR) has already begun to change the way we view research, with reviewers adding lasting content to papers, enriching the conclusions one can draw, and pointing out novel connections that were not extrapolated upon by the authors themselves. Here I like to draw an analogy to the open source movement, where code (and its documentation) is forkable, versioned, and open to constant revision – never static but always evolving.

Indeed, just last week PubMed launched their new “PubMed Commons” system, an innovative PPPR comment system, whereby any registered person (with at least one paper on PubMed) can leave scientific comments on articles.  Inevitably, the reception on twitter and Facebook mirrored previous attempts to introduce infrastructure-based solutions – mixed excitement followed by a lot of bemused cynicism – bring out the trolls many joked. To wit, a brief scan of the average comment on another platform, PubPeer, revealed a generally (but not entirely) poor level of comment quality. While many comments seem to be on topic, most had little to no formatting and were given with little context. At times comments can seem trollish, pointing out minor flaws as if they render the paper worthless. In many disciplines like my own, few comments could be found at all. This compounds the central problem with PPPR; why would anyone acknowledge such a system if the primary result is poorly formed nitpicking of your research? The essential problem here is again incentive – for reviews to be quality there needs to be incentive. We need a culture of PPPR that values positive and negative comments equally. This is common to both traditional and self-publication practices.

To facilitate easy, incentivized self-publication of comments and PPPRs, my colleague Hauke Hillebrandt and I have attempted to create a simple template that researchers can use to quickly and easily publish these materials. The idea is that by using these templates and uploading them to figshare or similar services, Google Scholar will automatically index them as citations, provide citation alerts to the original authors, and even include the comments in its h-index calculation. This way researchers can begin to get credit for what they are already doing, in an easy to use and familiar format. While the template isn’t quite working yet (oddly enough, Scholar is counting citations from my blog, but not the template), you can take a look at it here and maybe help us figure out why it isn’t working! In the near future we plan to get this working, and will follow-up this post with the full template, ready for you to use.

Example 3: Pre-registration of experimental trials

As my final example, I suggest that for many researchers, self-publication of trial pre-registrations (PR) may be an excellent way to test the waters of PR in a format with a low barrier to entry. Replication attempts are a particularly promising venue for PR, and self-publication of such registrations is a way to quickly move from idea to registration to collection (as in the above pilot data example), while ensuring that credit for the original idea is embedded in the infamously hard to erase memory of the internet.

A few benefits of PR self-publication, rather than relying on for-profit publishers, is that PR templates can be easily open-sourced themselves, allowing various research fields to generate community-based specialized templates adhering to the needs of that field. Self-published PRs, as well as high quality templates, can be cited – incentivizing the creation and dissemination of both. I imagine the rapid emergence of specialized templates within each community, tailored to the needs of that research discipline.

Part 3: Criticism and limitations

Here I will close by considering some common concerns with self-publication:

Quality of data

A natural worry at this point is quality control. How can we be sure that what is published without the seal of peer review isn’t complete hooey? The primary response is that we cannot, just like we cannot be sure that peer reviewed materials are quality without first reading them ourselves. Still, it is for this reason that I tried to suggest a few particularly ripe venues for self-publication of research. The cultural zeitgeist supporting full-blown scholarly self-publication has not yet arrived, but we can already begin to prepare for it. With regards to filtering noise, I argue that by coupling post-publication peer review and social media, quality self-publications will rise to the top. Importantly, this issue points towards flaws in our current publication culture. In many research areas there are effects that are repeatedly published but that few believe, largely due to the presence of biases against null-findings. Self-publication aims to make as much of the research process publicly available as possible, preventing this kind of knowledge from slipping through the editorial cracks and improving our ability to evaluate the veracity of published effects. If such data are reported cleanly and completely, existing quantitative tools can further incorporate them to better estimate the likelihood of p-hacking within a literature. That leads to the next concern – quality of presentation.

Hemingway's thoughts on data.

Quality of presentation

Many ask: how in this brave new world will we separate signal from noise? I am sure that every published researcher already receives at least a few garbage citations a year from obscure places in obscure journals with little relevance to actual article contents. But, so the worry goes, what if we are deluged with a vast array of poorly written, poorly documented, self-published crud. How would we separate the signal from the noise?

 The answer is Content, Presentation, and Clarity. These must be treated as central guidelines for self-publication to be worth anyone’s time. The Internet memesphere has already generated one rule for ranking interest: content rules. Content floats and is upvoted, blogspam sinks and is downvoted. This is already true for published articles – twitter, reddit, facebook, and email circles help us separate the wheat from the chaff at least as much as impact factor if not more. But presentation and clarity are equally important. Poorly conducted research is not shared, or at least is shared with vehemence. Similarly, poorly written self-publications, or poorly documented data/reagents are unlikely to generate positive feedback, much less impact-generating eyeballs. I like to imagine a distant future in which self-publication has given rise to a new generation of well-regarded specialists: reviewers who are prized for their content, presentation, and clarity; coders who produce cleanly documented pipelines; behaviorists producing powerful and easily customized paradigm scripts; and data collection experts who produce the smoothest, cleanest data around. All of these future specialists will be able to garner impact for the things they already do, incentivizing each step of the research processes rather than only the end product.

Being scooped, intellectual credit

Another common concern is “what if my idea/data/pilot is scooped?” I acknowledge that particularly in these early days, the decision to self-publish must be weighted against this possibility. However, I must also point out that in the current system authors must also weight the decision to develop an idea in isolation against the benefits of communicating with peers and colleagues. Both have risks and benefits – an idea or project in isolation can easily over-estimate its own quality or impact. The decision to self-publish must similarly be weighted against the need for feedback. Furthermore, a self-publication culture would allow researchers to move more quickly from project to publication, ensuring that they are readily credited for their work. And again, as research culture continues to evolve, I believe this concern will increasingly fade. It is notoriously difficult to erase information from The Internet (see the “Streisand effect”) – there is no reason why self-published ideas and data cannot generate direct credit for the authors. Indeed, I envision a world in which these contributions can themselves be independently weighted and credited.

 Prevention of cheating, corruption, self-citations

To some, this will be an inevitable point of departure. Without our time-tested guardian of peer review, what is to prevent a flood of outright fabricated data? My response is: what prevents outright fabrication under the current system? To misquote Jeff Goldblum in Jurassic Park, cheaters will always find a way. No matter how much we tighten our grip, there will be those who respond to the pressures of publication by deliberate misconduct. I believe that the current publication system directly incentivizes such behavior by valuing end product over process. By creating incentives for low-barrier post-publication peer review, pre-registration, and rich pilot data publication, researchers are given the opportunity to generate impact for each step of the research process. When faced with the vast penalties of cheating due to a null finding, versus doing one’s best to turn those data into something useful for someone, I suspect most people will choose the honest and less risky option.

 Corruption and self-citations are perhaps a subtler, more sinister factor. In my discussions with colleagues, a frequent concern is that there is nothing to prevent high-impact “rich club” institutions from banding together to provide glossy post-publication reviews, citation farming, or promoting one another’s research to the top of the pile regardless of content. I again answer: how is this any different from our current system? Papers are submitted to an editor who makes a subjective evaluation of the paper’s quality and impact, before sending it to four out of a thousand possible reviewers who will make an obscure  decision about the content of the paper. Sometimes this system works well, but increasingly it does not2. Many have witnessed great papers rejected for political reasons, or poor ones accepted for the same. Lowering the barrier to post-publication peer review means that even when these factors drive a paper to the top, it will be far easier to contextualize that research with a heavy dose of reality. Over time, I believe self-publication will incentivize good research. Cheating will always be a factor – and this new frontier is unlikely to be a utopia. Rather, I hope to contribute to the development of a bridge between our traditional publishing models and a radically advanced not-too-distant future.

Conclusion

Our current systems of producing, disseminating, and evaluating research increasingly seem to be out of step with cultural and technological realities. To take back the research process and bolster the ailing standard of peer-review I believe research will ultimately adopt an open and largely publisher-free model. In my view, these new practices will be entirely complementary to existing solutions including such as the p-curve5, open-source publication models6–8, and innovative platforms for data and knowledge sharing such as PubPeer, PubMed Commons, and figshare9,10. The next step from here will be to produce useable templates for self-publication. You can expect to see a PDF version of this post in the coming weeks as a further example of self-publishing practices. In attempting to build a bridge to the coming technological and social revolution, I hope to inspire others to join in the conversation so that we can improve all aspects of research.

 Acknowledgments

Thanks to Hauke Hillebrandt, Kate Mills, and Francesca Fardo for invaluable discussion, comments, and edits of this work. Many of the ideas developed here were originally inspired by this post envisioning a self-publication future. Thanks also to PubPeer, PeerJ,  figshare, and others in this area for their pioneering work in providing some valuable tools and spaces to begin engaging with self-publication practices.

Addendum

Excellent resources already exist for the many of the ideas presented here. I want to give special notice to researchers who have already begun self-publishing their work either as preprints, archives, or as direct blog posts. Parallel publishing is an attractive transitional option where researchers can prepublish their work for immediate feedback before submitting it to a traditional publisher. Special notice should be given to Zen Faulkes whose excellent pioneering blog posts demonstrated that it is reasonably easy to self-produce well formatted publications. Here are a few pioneering self-published papers you can use as examples – feel free to add your own in the comments:

The distal leg motor neurons of slipper lobsters, Ibacus spp. (Decapoda, Scyllaridae), Zen Faulkes

http://neurodojo.blogspot.dk/2012/09/Ibacus.html

Eklund, Anders (2013): Multivariate fMRI Analysis using Canonical Correlation Analysis instead of Classifiers, Comment on Todd et al. figshare.

http://dx.doi.org/10.6084/m9.figshare.787696

Automated removal of independent components to reduce trial-by-trial variation in event-related potentials, Dorothy Bishop

http://bishoptechbits.blogspot.dk/2011_05_01_archive.html

Deep Impact: Unintended consequences of journal rank

Björn Brembs, Marcus Munafò

http://arxiv.org/abs/1301.3748

A novel platform for open peer to peer review and publication:

http://thewinnower.com/

A platform for open PPPRs:

https://pubpeer.com/

Another PPPR platform:

http://f1000.com/

References

1. Henderson, M. Problems with peer review. BMJ 340, c1409 (2010).

2. Ioannidis, J. P. A. Why Most Published Research Findings Are False. PLoS Med 2, e124 (2005).

3. Peters, D. P. & Ceci, S. J. Peer-review practices of psychological journals: The fate of published articles, submitted again. Behav. Brain Sci. 5, 187 (2010).

4. Hunter, J. Post-publication peer review: opening up scientific conversation. Front. Comput. Neurosci. 6, 63 (2012).

5. Simonsohn, U., Nelson, L. D. & Simmons, J. P. P-Curve: A Key to the File Drawer. (2013). at <http://papers.ssrn.com/abstract=2256237>

6.  MacCallum, C. J. ONE for All: The Next Step for PLoS. PLoS Biol. 4, e401 (2006).

7. Smith, K. A. The frontiers publishing paradigm. Front. Immunol. 3, 1 (2012).

8. Wets, K., Weedon, D. & Velterop, J. Post-publication filtering and evaluation: Faculty of 1000. Learn. Publ. 16, 249–258 (2003).

9. Allen, M. PubPeer – A universal comment and review layer for scholarly papers? | Neuroconscience on WordPress.com. Website/Blog (2013). at <http://neuroconscience.com/2013/01/25/pubpeer-a-universal-comment-and-review-layer-for-scholarly-papers/>

10. Hahnel, M. Exclusive: figshare a new open data project that wants to change the future of scholarly publishing. Impact Soc. Sci. blog (2012). at <http://eprints.lse.ac.uk/51893/1/blogs.lse.ac.uk-Exclusive_figshare_a_new_open_data_project_that_wants_to_change_the_future_of_scholarly_publishing.pdf>

11. Yarkoni, T., Poldrack, R. A., Van Essen, D. C. & Wager, T. D. Cognitive neuroscience 2.0: building a cumulative science of human brain function. Trends Cogn. Sci. 14, 489–496 (2010).

12. Bishop, D. BishopBlog: A gentle introduction to Twitter for the apprehensive academic. Blog/website (2013). at <http://deevybee.blogspot.dk/2011/06/gentle-introduction-to-twitter-for.html>

13. Hadibeenareviewer. Had I Been A Reviewer on WordPress.com. Blog/website (2013). at <http://hadibeenareviewer.wordpress.com/>

14. Tressoldi, P. E., Giofré, D., Sella, F. & Cumming, G. High Impact = High Statistical Standards? Not Necessarily So. PLoS One 8, e56180 (2013).

15.  Brembs, B. & Munafò, M. Deep Impact: Unintended consequences of journal rank. (2013). at <http://arxiv.org/abs/1301.3748>

16.  Eisen, J. A., Maccallum, C. J. & Neylon, C. Expert Failure: Re-evaluating Research Assessment. PLoS Biol. 11, e1001677 (2013).

http://wl.figshare.com/articles/875339/embed?show_title=1

How to reply to #icanhazpdf in 3 seconds

Yesterday my friend Hauke and I theorized about a kind of dream scenario- a totally distributed, easy to use, publication liberation system. This is perhaps not feasible at this point [1]. Today we’re going to present something that will be useful right now. The essential goal here is to make it so that anyone, anywhere, can access the papers they need in a timely manner. The idea is to take advantage of existing strategies and tools to streamline paper sharing as much as possible. Folks already do this- every day on twitter or in private, requests for papers are made and fulfilled. Our goal is to completely streamline this process down to a few clicks of your mouse. That way a small but dedicated group of folks – the Papester Collective – can ensure that #icanhazpdf requests are fulfilled almost instantly. This is a work in progress. Leave comments on how to improve and further streamline this system and join the collective!

SHORT VERSION: HOW TO GET A PAPER BEHIND A PAYWALL QUICKLY

Tweet (for example): “#icanhazpdf http://dx.doi.org/10.1523/JNEUROSCI.4568-12.2013

Click: Here you can find more detailed instructions.

HOW TO JOIN THE COLLECTIVE AND START SERVING REQUESTS

SHORT INSTRUCTIONS AND REQUIRED SOFTWARE:

  1. Twitter: Monitor #icanhazpdf #requests
  2. Zotero and zotero browser plugin: after clicking on DOI link or abstract page just click on ‘Save to Zotero’ button to auto-grabs PDFs

  3. Zotfile: automatically copies new Zotero pdfs files saved to public Dropbox folder

  4. Dropbox: Cloud storage system to seamlessly share files with anyone without login.

  5. Dropbox linker: automatically adds links from public folder to your clipboard

  6. Reply to request tweets: paste URL from clipboard and if you want #papester

That’s it! Now you can just click request links, click the Zotero get PDF button, and CTRL+V a dropbox direct download link in response!

Click: Here you can find more detailed instructions.

1.The fundamental problem: uploading huge repositories of scientific papers is not sensible for now. It’s too much data (50 million papers * 0.5-1.5 megabytes together make up ~ 25-75 Terrabytes) and the likelihood for every paper to be downloaded is more uniformly distributed than with files traditionally shared like music. For instance, there are 100 million songs x 3.5 mb songs, and it is difficult to find exotic songs online – some songs have decent availability now because there are only a few favourites – not so with favourite papers. Also, fewer people will share papers than songs, so this makes it more even more difficult to sustain a complete repository. Thus, we need a system that fufills requests individually.

Disclaimer: Please make sure you only share papers with friends who also have the copyrights to the papers you share.