Frequently Asked Questions (and their answers)

On this page:

1. General

What is Review Quality Collector (RQC)?

Review Quality Collector (RQC) is an initiative for improving the quality of scientific peer review. Its core is a mechanism that supplies a reviewer with a receipt for their work for each conference instance or journal year.

Why do we need RQC?

Reviewing is important for the scientific process, but so far does produce very little benefit for the reviewer in the currency that counts in research: reputation. This makes many reviewers invest less care into their reviewing than would be necessary for reviewing to function properly, which hurts the quality of the scientific process.

How does RQC work?

Please see the description of the general mechanism on the homepage and specifically of the grading system on the grading system page. Please see the description of usage procedures on the how-to page.

Where did the idea come from?

See the description of the history of RQC.

What are the advantages and disadvantages?

Read the list of advantages and the list of disadvantages

I don't like RQC. Can I escape it?

As a reviewer, you will always be able to opt out of RQC processes and even if you don't, you can keep some or all of your receipts private. As a conference chair or journal editor, you may have to resist some pressure from your reviewers (and then risk losing them).

What does the logo mean?

The RQC logo symbolizes the crowded schedules of typical reviewers.

2. On using RQC

In which cases should I use RQC?

RQC is useful for conferences and workshops large or small. The higher quality you expect from the reviewing process, the more reason there is to use RQC. For a workshop that will accept almost everything that is on-topic, RQC might be an overkill.

At which time should I use RQC?

Ideally, you will announce the use of RQC when you hire your reviewers; if you announce it later, that's OK, too, as reviewers can still opt out. The actual review grading is best done during the discussion phase (after reviewing, before notification); one week is a good timeframe.

How much effort is required?

Reviewers will have to read and understand each co-review. Beyond that, it will take less than 5 minutes per co-review for the actual grading.

Organizers will have the following effort elements:

See the how-to page for the procedure description.

How about double-blind, triple-blind, or open reviewing?

RQC is compatible with these modes. Please see the description of the RQC reviewing model.

Is there a monetary cost involved?

RQC is and will remain free for reviewers, for conferences, and for research institutions.

Once it is established, it is intended to require a per-receipt fee from publishers of journals.

I may like RQC. Can I play with it somehow?

Yes, you can. See How to try out RQC with fake data.


3. Current status of introduction

Who can use RQC already?

So far, only there is functionality only for conference chairs (and their reviewers): review quality definition, grading, receipt creation; and only for conferences using Easychair.

The functionalty for research institutions will be coming next: collecting receipts, downloading receipt data databases for analysis.
(No, I will not promise any particular arrival time.)

Functionality for journals and publishers will arrive in phase 3.

How much has RQC been adopted?

RQC is still in its early adopter phase.


4. Technical issues

What software should I use to modify an RQdef file?

Review quality definitions (RQdefs) are represented as RTF (Rich Text Format) files.

On Windows, the natural choice for editing such files would be MS Word. Note that wordpad, although nominally it can edit RTF, can do so only in a very restricted manner. The resulting files may look alright, but they use an impoverished style of encoding and RQC will not be able to analyze them.
On Mac OS X, use MS Word or LibreOffice Writer.
On Linux, LibreOffice Writer is the most suitable choice.
On Android and iOS, various software might be able to produce suitable files, but I recommend to avoid these platforms.

Finally, Google Docs can handle RTF as well without any local software installation, just by using one of the major web browsers. It also allows a team to collaboratively edit the document at the same time, which is nice to get a good discussion e.g. during a Skype meeting.

Whichever software you use, make sure you store the result of your editing efforts as RTF again, even if the software suggests some other format.

Is RQC reliable?

The software is not yet very robust against various kinds of external problems, but the implementation is well-covered by automated tests and ought to be quite reliable.

Can I look at the development history?

Yes. There is a list of changes on the RQC history & future page.

Pitfalls when using Easychair


5. Infrastructure used by RQC

Without the following great software and services, RQC could not exist:

Software

In alphabetical order:

Services




Mon 2017-10-23 04:17 UTC

Review Quality Collector  © 2012-2017

Imprint / Impressum