Our current publication system should be redesigned to maximize the rate of progress in our field. This means accelerating the speed at which new ideas and results are exchanged, disseminated, and evaluated. This also means minimizing the amount of time each of us spends evaluating other people's work through reviewing and sifting through the literature. A major issue is that our current system, with its emphasis on highly-selective conferences, is highly biased against innovative ideas and favors incremental tweaks on well-established methods. Ideas that turn out to be highly influential are sometimes held up for months (if not years) in reviewing purgatory, particularly if they require several years to come to maturity (there are a few famous examples, mentioned). The friction in our publication system is slowing the progress of our field. It makes progress incremental. And it makes our conferences somewhat boring.
Our current publication system is the result of history, where the dissemination of scientific information was limited by printing and shipping capacity and cost. Paradoxically, Computer Science has not fully taken advantage of the Web as a communication medium to the same extent that other fields have (such as Physics and Mathematics). In an attempt to maximize the efficiency of our scientific communication system, ICLR 2013 is adopting a new publication model that dissociates dissemination from evaluation.
The reviewing process will proceed as follows: - Authors post their submissions on arXiv and send us a link to the paper. A separate, permanent website will be setup to handle the reviewing process, to publish the reviews and comments, and to maintain links to the papers.
- The ICLR program committee designates anonymous reviewers as usual.
- The submitted reviews are published without the name of the reviewer, but with an indication that they are the designated reviews. Anyone can write and publish comments on the paper (non anonymously). Anyone can ask the program chairs for permission to become an anonymous designated reviewer (open bidding). The program chairs have ultimate control over the publication of each anonymous review. Open commenters will have to use their real name, linked with their Google Scholar profile.
- Authors can post comments in response to reviews and comments. They can revise the paper as many time as they want, possibly citing some of the reviews.
- At a given date, the ICLR program committee will consider all submitted papers, comments, and reviews and decides which papers are to be presented at the conference as oral or poster. Although papers can be modified after that date, there is no guarantee that the modifications will be taken into account by the committee.
- The best of the accepted papers, (the top 25%-50%) will be given oral presentations at the conference. We have made arrangements for revised versions of selected papers from the conference to be published in a JMLR special topic issue.
- The other papers will be considered non-archival (like workshop presentations), and could be submitted elsewhere (modified or not), although the ICLR site will maintain the reviews, the comments, and the links to the arXiv versions.
|
|
