The RQC API for manuscript handling systems

API version: 2022-10-20

This is a work-in-progress and not yet ready for use!

On this page:

Elsewhere:

1. What is an API and why is it needed?

For RQC to be practical, it needs to be integrated with the manuscript handling system (MHS) to which the reviews are submitted.

For conferences, RQC can play the active part: All reviews are present at a particular time (the reviewing deadline, or shortly later) and RQC can go and get all the submission, reviewer, and reviewing data from the manuscript handling system (after the PC chairs have permitted it to do so) and then do its job with them.

For journals, this mode of operation does not work: Reviews trickle in all the time and it makes no sense to wait with grading them until, say, the end of the year. So the manuscript handling system needs to actively transmit the data for reviews to RQC as they come in. RQC needs to provide a technical way to do that: The API (application programming interface).

2. API for manuscript handling systems (MHS)

2.1 Main use case

Overall, the process for handling one submission to journal X works roughly as follows:

Actors: The journal's manuscript handling system (MHS), Editor, Review Quality Collector system (RQC), Reviewers

  1. The MHS receives a submission S.
  2. The Editor invites Reviewers A, B, C to review S.
  3. The MHS receives review A.
  4. The MHS receives review B.
  5. The MHS receives review C, or the Editor becomes impatient to wait for it and wants to make a decision on S.
  6. The Editor triggers "Initiate RQC review grading" for S in the MHS.
  7. The MHS submits data about submission S, and reviews A, B, and (if the reviewer had agreed to provide a review) C to RQC via the API.
  8. RQC replies with a URL G (of the RQC grading page for S).
  9. The MHS redirects the Editor's browser to page G.
  10. The Editor uses RQC page G to assign the case to a subjournal, to grade reviews A, B, C, and to set notification parameters for the other participants (authors, reviewers, other editors).
  11. RQC redirects the Editor's browser back to the MHS.
  12. The Editor either makes an acceptance decision or first involves other editors, and the decision is then made in some later session.
  13. The MHS submits data about the decision for submission S to RQC. RQC initiates the grading process in case the editor has skipped steps 6 to 11.

2.2 MHS configuration use case

Before the above can happen, the MHS must be equipped with the journal-specific API key (like a password) that it must use to authenticate itself to RQC.

Actors: The journal's manuscript handling system (MHS), Publisherperson, RQguardian, Review Quality Collector system (RQC).

  1. The Publisherperson uses RQC to generate an API key for the journal.
  2. The Publisherperson enters the API key in the journal's MHS (or asks an RQguardian to do it).
  3. The MHS validates the API key with RQC and stores it.

Variant: 3b: If RQC rejects the API key, the MHS shows the error message provided by RQC and does not store the key.

The above functionality of use case 2.1 is available only after a successful step 3, not in the case of 3b.

2.3 API overview

So far, the RQC API has only two endpoints: mhs_submission and mhs_apikeycheck.

mhs_apikeycheck must be used first, using HTTP GET, for use case 2.2.

mhs_submission is used, using HTTP POST, for use case 2.1. It can also be used with HTTP GET for testing purposes to validate that the data has arrived at RQC as expected. HTTP GET is not required otherwise. For each submission that undergoes review at the MHS, submission is usually called twice (and in rare cases more): First to deposit the submission, editors, and reviewing data. This triggers the grading process at RQC. Later to add information about the editorial decision and possibly any additional reviews that may have arrived in the meantime.

Responses are JSON objects. If there is any semantic problem with a call, the response will contain a detailed error message.

The remainder of Section 2 will explain how to implement an RQC adapter for an MHS that makes use of the API.

2.4 Requirements for the MHS database schema

2.4.1 What you probably have already

RQC requires the following data for each submission (which thus needs to be stored in the MHS database somewhere), as far as available:

(The above are not the actual names used by RQC, see further down.)

For some of these, reasonable defaults are available; in particular, firstname can be left empty if unavailable and the full name supplied in the lastname field; if ORCID IDs are not known, that field can be left empty. Other fields are important; in particular, RQC uses email addresses as keys for identifying people, so if several authors share an email address, there is trouble and only one of them can be reported to RQC.
This is described in more detail in the request descriptions below.

2.4.2 What you need to add

Two fields will have to be added to each journal's master data in the MHS:

Beyond that, it may be needed to store an extra mapping of the codes for editorial decisions; see under decision below. This might also require an extension of the MHS GUI to set those values.

Further fields
will have to be added to the journal transaction data in the MHS; they describe which reviewers have opted in to or opted out of participating in RQC for this year for this journal:

Of course you need not realize it exactly like this. You must merely be able to represent opting status with respect to the current year per reviewer per journal. (If the journal has subdivisions, whether to represent them or not depends on whether they are represented at RQC as a single journal or as multiple journals. Opting status is per RQC-side journal.)

2.5 Requirements for MHS functionality

An RQC adapter for a given MHS must provide the following functionality:

2.5.1 One-time preparation step: Registration

The MHS must allow RQguardians or Publisherpersons to enter the rqc_journal_id and rqc_journal_apikey they received from their publisher for the given journal, and the MHS must validate these data upon entry by issuing an mhs_apikeycheck call. Invalid data must not be accepted and must never be used for RQC calls.

2.5.2 Potential per-review step: Reviewer opts in/out

The dialog by which a reviewer submits a review must be extended: If there is no entry yet for this reviewer, journal, and year in the table described in Section 2.4.2 above, the MHS must ask the reviewer if they want to opt into participating in RQC for this journal for this year or want to opt out of participating in RQC for this journal for this year.

Use the following wording for the question:

I want to participate in Review Quality Collector (RQC) for all my reviews for this journal this year. My reviewing data will be submitted to RQC and RQC will provide me with a reviewing receipt in exchange. I understand the purpose of RQC and its use of my data as described at https://reviewqualitycollector.org/t/terms-and-privacy. I can make a separate decision on this for other journals. I can make a separate decision on this for the present journal next year.

Yes / No

Answering yes creates a contract between the reviewer and RQC where the reviewer provides a review and RQC provides a reviewing receipt. This contract is the basis on which RQC is allowed (by privacy regulations such as the EU-GDPR) to process the reviewer's data. Different wording would provide a shakier basis and must not be used.

After the reviewer has answered the above dialog, create an entry for it and do not ask this reviewer again for reviews pertaining to the same year and journal.

(Don't worry: This was the most complicated aspect of what the MHS needs to do.)

2.5.3 Per-submission steps: Submit to RQC

2.6 API call URLs

mhs_apikeycheck URLs have the following form: https://reviewqualitycollector.org/api/mhs_apikeycheck/<journal_id>

mhs_submission URLs have the following form: https://reviewqualitycollector.org/api/mhs_submission/<journal_id>/<external_uid>

The API key that is required for both calls is transmitted in an HTTP header as follows:

2.7 API call HTTP Headers

Each call to the API must provide the following HTTP headers:

2.7 mhs_apikeycheck API endpoint

2.7.1 What and when

A GET call to this endpoint validates a pair of rqc_journal_id and rqc_journal_apikey. The call must be made by the MHS when (and only when) a user attempts to save initial or modified settings of these two values. The call must include the headers as described in Section 2.7.

If the validation is successful, calls to the submission endpoint may subsequently be made. If the validation fails, calls to the submission endpoint must not be made, because no such calls would succeed and the editors would be confused by the resulting error messages.

2.7.3 RQC response: status codes and body

RQC will respond to such a GET request with one of the following status codes:

2.8 mhs_submission API endpoint

2.8.1 Explicit vs. implicit vs. delayed calls

Explicit calls: The MHS should provide a button or link "RQC-grade reviews" on the editors' submission administration page as soon as at least one review has been received. The editor can use this GUI element to submit the currently existing reviews for this submission to RQC, will be immediately redirected to RQC for grading these reviews, and then be redirected back to the MHS. Using this functionality is called an "explicit call". Explicit calls are optional: Grading the review can be a useful preparatory step for making the editorial decision, but it is not necessary to perform the grading at this point -- and possibly this particular editor is not involved in the grading at all. More than one editor may make an explicit call.

Implicit calls: Whenever an editorial decision for a submission is made (or changed) in the MHS, the MHS must make an "implicit call" to RQC and submit the review data. The MHS may or may not redirect the user to RQC at this point as it sees fit, but the recommended behavior it to not redirect to RQC as that might be confusing to the editor. Implicit calls are mandatory, they must be made for each submission each time a new editorial decision is stored, because the behavior of RQC may depend on the decision.

Delayed calls: A call to RQC may fail due to network problems or unavailability of the RQC service. If that happens, the call should be put into a backlog of calls to retry. This backlog should be revisited once per day and the first of the calls in the queue submitted again. If a call succeeds, remove it from the queue. If it fails, count the failure, move the call to the end of the queue, stop retrying any calls for that day, and retry the next call on the queue the next day. Delete calls after 10 failures. Again: If any call fails on a given day, stop all retrying for that day in order to avoid useless calls.
Note that this behavior applies only to unavailability failures; it must not be applied if the authentication token is wrong.

See the response status codes below for a discussion of the redirection mechanism and the discussion of consequences for design considerations.

2.8.2 Example JSON body

Here is an example what that JSON may look like; an explanation of each field follows below:

{
    "interactive_user": "myeditor@anywhwere.org"
    "mhs_submissionpage": "https://mymhs.example.com/journal17/user29?submission=submission31"
    "title": "Rather Altogether Modestly Long-ish Submission Title 21",
    "external_uid": "MANU-10021/R2",
    "visible_uid": "MANU-10021/R2",
    "submitted": "2022-02-01T21:07:42Z",
    "author_set": [
        {
            "email": "author31@sm.wh",
            "firstname": "A.N.",
            "lastname": "Author",
            "orcid_id": "0000-0001-5592-0005",
            "order_number": 1
        },
        {
            "email": "other_author@sm.wh",
            "firstname": "Brubobolob",
            "lastname": "Authoress",
            "orcid_id": "0000-0001-5592-0006",
            "order_number": 3
        }
    ],
    "editorassignment_set": [
        {
            "email": "editor1@sm.wh",
            "firstname": "Keen",
            "lastname": "Editorus",
            "orcid_id": "0000-0001-5592-0003",
            "level": 1                
        },
        {
            "email": "editor2@sm.wh",
            "firstname": "Keener",
            "lastname": "Editora",
            "orcid_id": "0000-0001-5592-0004",
            "level": 3      
        },
        {
            "email": "editor3@sm.wh",
            "firstname": "Kim",
            "lastname": "Nguyen",
            "orcid_id": "0000-0001-5592-0007",
            "level": 3      
        }
    ],
    "review_set": [
        {
            "visible_id": "1",
            "invited": "2022-02-08T14:30:10Z",
            "agreed": "2022-02-09T07:22:03Z",
            "expected": "2022-02-28T14:30:10Z",
            "submitted": "2022-02-15T09:09:46Z",
            "text": "This is the text of review 1.",
            "is_html": false,
            "suggested_decision": "MINORREVISION",
            "reviewer": {
                "email": "reviewer1@sm.wh",
                "firstname": "J.",
                "lastname": "Reviewer 1",
                "orcid_id": "0000-0001-5592-0001"
            }
        },
        {
            "visible_id": "2",
            "invited": "2022-02-15T00:00:00Z",
            "agreed": "",
            "expected": "2022-02-28T00:00:00Z",
            "submitted": "2022-03-01T00:00:00Z",
            "text": "This is the text of review 2.\nIt is multiple lines long, some of which are themselves a bit longer (like this one, for instance; at least somewhat -- truly long would be longer, of course)\n\nLine 4, the last one.",
            "is_html": false,
            "suggested_decision": "ACCEPT",
            "reviewer": {
                "email": "reviewer2@sm.wh",
                "firstname": "J.",
                "lastname": "Reviewer 2",
                "orcid_id": "0000-0001-5592-0002"
            }
        }
    ]
    "decision": "ACCEPT",
}

(Please make sure you correctly discriminate integer literals from corresponding strings (1 vs "1") and the three special JSON literals true, false, and null from their unrelated string look-alikes "true", "false", and "null". null is never used in the RQC API.)

2.8.4 Size limits

Cut off longer values at the end for the POST request, or the request will fail.

2.8.5 POST call: Body and semantics

This section describes the meaning of the data shown in the example in Section 2.8.1. The field ordering is not relevant.

This is what each field means and how to use it:

If a journal uses a different set of decision types, match those to their closest equivalent in the above set. Uppercasing is relevant. If an MHS allows each journal to define an arbitrary list of decision codes, and the MHS cannot know their meaning, the RQC adapter for that MHS must make the editors define a mapping from the journal's internal codes to the above codes for RQC. (Not fun, sure, but there is no way around it. Possibly an Excel upload is a suitable compromise between easy-to-build and easy-to-use.) No API calls can be made until that mapping is known and can be used.

2.8.6 RQC response: status codes and body

RQC will respond to such a POST request with one of the following status codes:

2.8.7 Semantics of subsequent POST calls

Once a submission has been sent to RQC, it should normally be updated by additional calls only in the following respects:

If other changes occur, e.g. in the authors list or the set of editor assignments, RQC may or may not react properly (or at all) to those.

No further updates are possible at all once the respective journal year is no longer active (which is after February of the next year). Such calls will result in a 400 Bad request response.

2.8.8 GET call

You can also call the submission API endpoint with the HTTP GET method. If the headers are correct, you will get either a response with status code 200 OK and a JSON body equivalent to that previously used for POST or get a 404 Not found response if no successful POST had previously occurred.

3. RQC API implementation status

The RQC functionality for journals (including the API) and one corresponding MHS adapter (for Open Journal Systems, OJS) are currently nearing the completion of their first version. Until both are finished, the API is considered provisional and is subject to sudden change.

If you want to start implementing the API for your manuscript handling system now, please contact Lutz Prechelt.

4. Testing your MHS/RQC adapter

5. API change history

6. TODO


Feb. 2, 2023, 8:44 a.m.

Review Quality Collector  © 2012-2023

Contact/Imprint/Impressum