last change: 2021-05-20
This is a work-in-progress and not yet ready for use!
Elsewhere:
For RQC to be practical, it needs to be integrated with the manuscript handling system (MHS) to which the reviews are submitted.
For conferences, RQC can play the active part: All reviews are present at a particular time (the reviewing deadline, or shortly later) and RQC can go and get all the submission, reviewer, and reviewing data from the manuscript handling system (after the PC chairs have permitted it to do so) and then do its job with them.
For journals, this mode of operation does not work: Reviews trickle in all the time and it makes no sense to wait with grading them until, say, the end of the year. So the manuscript handling system needs to actively transmit the data for reviews to RQC as they come in. RQC needs to provide a technical way to do that: The API (application programming interface).
Overall, the process for handling one submission to journal X works roughly as follows:
Actors: The journal's manuscript handling system (MHS), Editor, Review Quality Collector system (RQC), Reviewers
So far, the RQC API has only one single endpoint.
The URLs to reach it are described in a separate section below.
The endpoint is normally used twice for each submission, both
times using the POST HTTP verb:
First to deposit the submission, editors, and reviewing data.
Later to add information about the editorial decision and possibly
any additional reviews that may have arrived in the meantime.
Review text supplied in the initial PUT can be modified by the later call.
GET is supported for checking the validity of a pair of journal ID
and journal key.
Other than that, GET is supported solely for testing purposes; it is not needed
for normal operation.
DELETE is not supported because RQC attempts to never remove gradings.
All URLs have the following form:
https://reviewqualitycollector.org/j/mhsapi/gp/<journal_id>/<submission_url_id>/<token>
where
journal_id is an integer value assigned to the journal by RQC.
When registering a journal for use with RQC, RQC will show this value.
It must be manually transfered to the MHS.submission_url_id is the URL-encoded version of the submission_id,
the string assigned to a given submission by the MHS.
MHSs often call it the submission number.submission_id has been transformed according
to the rules of RFC 3986 by means of percent-encoding ("quoting") such that
it contains only characters valid in a URL and in particular no longer
contains the '/' (slash) character.token is an authentication token as described further down.RQC requires the following data for each submission (which thus needs to be stored in the MHS database somewhere):
(These are not the actual names used by RQC, see further down.)
For some of these, reasonable defaults are available; in particular,
firstname can be left empty if unavailable and the full name supplied
in the lastname field; if ORCID IDs are not known, that field can be
left empty.
Other fields are very important; in particular, RQC uses email addresses
as keys for identifying people, so if several authors share an email
address, there is trouble and only one of them can be reported to RQC.
This is described in more detail in the request descriptions below.
Two fields will have to be added to each journal's master data in the MHS:
rqc_journal_id: an integer supplied by RQC when registering the journal.
This value is public.rqc_journal_key: an ASCII string (maximum length 64) created by RQC
upon request. This value is secret and is used for authentication
as described below.Beyond that, it may be needed to store an extra mapping of the codes
for editorial decisions; see under decision below.
This might also require an extension of the MHS GUI to set those values.
Further fields
will have to be added to the journal transaction data
in the MHS; they describe which reviewers have
opted in to or opted out of participating in RQC for this year for this journal:
journal_id: The primary key of this journal within the MHS. reviewer_email: The email address that represents the reviewerreviewer_pseudonym: A random string of 20 lowercase letters in range a-z
that is created when the entry is created and then kept fixed.opting status: True for "reviewer has opted in",
False for "reviewer has opted out".year: The year to which the decision pertains.
(This is not what you probably think it is;
see the discussion of opting below.) The MHS must allow editors to enter the journal_id and
journal_key they received from their publisher for the given
journal, and the MHS must validate these data upon entry by issuing a
HEAD call as described below (and get a '200 OK' HTTP response status).
Invalid data must not be accepted and must never be used for RQC calls.
The dialog by which a reviewer submits a review must be extended: If there is no entry yet for this reviewer, journal, and year in the table described in Section 2.4.2 above, the MHS must ask the reviewer if they want to opt into participating in RQC for this journal for this year or want to opt out of participating in RQC for this journal for this year.
Use the following wording for the question:
I want to participate in Review Quality Collector (RQC) for all my reviews for this journal this year. My reviewing data will be submitted to RQC and RQC will provide me with a reviewing receipt in exchange. I understand the purpose of RQC and its use of my data as described at https://reviewqualitycollector.org/t/terms-and-privacy. I can make a separate decision on this for other journals. I can make a separate decision on this for the present journal next year.
Yes / No
Answering yes creates a contract between the reviewer and RQC where the reviewer provides a review and RQC provides a reviewing receipt. This contract is the basis on which RQC is allowed (by privacy regulations such as the EU-GDPR) to process the reviewer's data. Different wording would provide a shakier basis and should not be used.
After the reviewer has answered the above dialog, create an entry for it and do not ask this reviewer again for reviews pertaining to the same year and journal.
Which year? "This year" in the above text does not always mean the year in which the statement is made; it can sometimes refer to the previous year. "This year" means the calendar year in which the first review for the present submission was received. (This is needed because RQC needs to handle all reviews for the same submission as one unit.)
(Don't worry: This was the most complicated aspect of what the MHS needs to do.)
POST call to the API endpoint.
If one or more of the invited reviews have not yet arrived, the
GUI should ideally show a warning dialog
"You are about to initiate the review grading process for this
submission, but not all reviews have yet arrived.
Reviews arriving only after this step will likely not be included in
the grading process." (Options: Proceed, Cancel).POST call to the API endpoint to notify
RQC of the decision. This (usually second) call also includes
all submission and reviewing data and will step in if the first
call was never made.
It is important, because missing the grading of a review would be
damaging to the journal's reputation.Authentication is performed via the token submitted in the URL
with each call.
A different token must be supplied for each combination of
journal ID and submission ID as follows:
submission_id.
Let us assume the simplified ID is EASE-2022-012.R1.journal_key corresponding to the journal_id.
Let us assume the key is abcdefg.submission_id and journal_key, giving
EASE-2022-012.R1abcdefg.cbf23c4c7cda2fc86b2b5aafd2f073e13623c03566c9f8eee3815a07b9a9a1a1.token for the given pair of
journal ID and submission ID.The token is required for all calls.
If it is wrong (and only then), the response will have
HTTP status code 401 Unauthorized.
This fact will not be repeated in the response code listings below.
A GET request must be used by the MHS to make sure a pair of
journal_id and journal_key is indeed valid.
This is useful for validating the data input by an editor in a
journal configuration form, because if journal_id or journal_key are
wrong, none of the later RQC calls would succeed, and so the corresponding
reviews would never reach RQC evaluation.
For this purpose, the URL is formed differently, as follows:
https://reviewqualitycollector.org/j/mhsapi/<journal_id>/<datetime>/<token>/?check=1
where
journal_id is the same as described above.datetime is an ISO-8601 datetime string for the current time
in timezone UTC ("zero-offset time", "Zulu time").yyyy-mm-ddThh:mm:ssZ, for example:
2021-12-02T13:56:01Z;
see articles ISO 8601
and UTC
in Wikipedia.token is like described above,
except that the datetime replaces the submission_id
in the computation.check=1 indicates the checking-only mode of request processing. The purpose of submitting current time in the URL is to make sure the clocks of MHS and RQC are properly aligned.
RQC will respond to such a GET request with one of the following
status codes:
200 OK: The journal_id exists, the token correctly reflects the
corresponding journal key, and the datetime was within 60 seconds of
RQC's time. The response body will contain a JSON OK.400 Bad request: The datetime string was malformed, or
the datetime is off by more than 60 seconds compared
to RQC's clock.
The response body will contain details for human debugging.403 Forbidden: The token is wrong. It was either computed in the
wrong fashion or the underlying journal_key is invalid for the given
journal_id.
The response body will contain details for human debugging,
including the first few characters of the correct token.404 Not found: The whole URL was syntactically malformed or
the journal_id does not exist at RQC.POSTExplicit calls: The MHS should provide a button or link "RQC-grade reviews" on the editors' submission administration page as soon as at least one review has been received. The editor can use this GUI element to submit the currently existing reviews for this submission to RQC, will be immediately redirected to RQC for grading these reviews, and then be redirected back to the MHS. Using this functionality is called an "explicit call". Explicit calls are optional: Grading the review is a nice preparatory step for making the editorial decision, but it is not necessary to perform the grading at this point -- and possibly this particular editor is not involved in the grading at all. More than one editor may make an explicit call.
Implicit calls: Whenever an editorial decision for a submission is made (or changed) in the MHS, the MHS must make an "implicit call" to RQC and submit the review data. The MHS may or may not redirect the user to RQC at this point as it sees fit. Implicit calls are mandatory, they must be made for each submission each time a new editorial decision is stored, because the behavior of RQC may depend on the decision.
Delayed calls: A call to RQC may fail due to network problems or
unavailability of the RQC service.
If that happens, the call should be put into a backlog of calls to retry.
This backlog should be revisited once per day and the first of the calls in the
queue submitted again.
If a call succeeds, remove it from the queue.
If it fails, count the failure, move the call to the end of the queue,
and retry the next call on the queue the next day. Delete calls after
10 failures.
If any call fails on a given day, stop all retrying for that day in order
to avoid useless calls.
Note that this behavior applies only to unavailability failures; it should not
be applied if the authentication token is wrong.
See the response status codes below for a discussion of the redirection mechanism and the discussion of consequences for design considerations.
An API call must have a Content-Type: application/json HTTP header
and provide its data as UTF-8-encoded
JSON in the body.
See json.org
or JSON at Wikipedia.
Here is an example what that JSON may look like; an explanation of each field follows below:
{
"api_version": "1.0alpha", // GET only, not needed in POST
"mhs_software": "Open Journal Systems 3.5.1"
"mhs_adapter_version": "RQC plugin 1.1.17"
"interactive_user": "myeditor@anywhwere.org"
"mhs_submissionpage": "https://mymhs.example.com/journal17/user29?submission=submission31"
"title": "Rather Altogether Modestly Long-ish Submission Title 21",
"visible_id": "MANU-10021/R2",
"submitted": "2022-02-01T11:07:42Z",
"author_set": [
{
"email": "author31@sm.wh",
"firstname": "A.N.",
"lastname": "Author",
"orcid_id": "0000-0001-5592-0005"
"is_corresponding": true,
"order_number": 1
},
{
"email": "other_author@sm.wh",
"firstname": "Brubobolob",
"lastname": "Authoress",
"orcid_id": "0000-0001-5592-0006"
"is_corresponding": true,
"order_number": 2
}
],
"editorassignment_set": [
{
"email": "editor1@sm.wh",
"firstname": "Keen",
"lastname": "Editorus",
"orcid_id": "0000-0001-5592-0003"
"level": 1
},
{
"email": "editor2@sm.wh",
"firstname": "Keener",
"lastname": "Editora",
"orcid_id": "0000-0001-5592-0004"
"level": 3
},
{
"email": "editor3@sm.wh",
"firstname": "Kim",
"lastname": "Kong",
"orcid_id": "0000-0001-5592-0007"
"level": 3
}
],
"review_set": [
{
"visible_id": "0",
"invited": "2022-02-08T14:30:10Z",
"agreed": "2022-02-09T07:22:03Z",
"expected": "2022-02-28T14:30:10Z",
"submitted": "2022-02-15T09:09:46Z",
"text": "This is the text of review 1.",
"is_html": false,
"suggested_decision": "minorrevision",
"reviewer": {
"email": "reviewer1@sm.wh",
"firstname": "J.",
"lastname": "Reviewer 1",
"orcid_id": "0000-0001-5592-0001"
}
},
{
"visible_id": "1",
"invited": "2022-02-15T00:00:00Z",
"agreed": "",
"expected": "2022-02-28T00:00:00Z",
"submitted": "2022-03-01T00:00:00Z",
"text": "This is the text of review 2.\It is multiple lines long, some of which are themselves a bit longer (like this one, for instance; at least somewhat -- truly long would be longer, of course)\n\nLine 4, the last one.",
"is_html": false,
"suggested_decision": "accept",
"reviewer": {
"email": "reviewer2@sm.wh",
"firstname": "J.",
"lastname": "Reviewer 2",
"orcid_id": "0000-0001-5592-0002"
}
}
]
"decision": "accept",
}
(Please make sure you correctly
discriminate integer literals from corresponding strings (1 vs "1")
and the three special JSON literals true, false, and null from
their unrelated string look-alikes "true", "false",
and "null". null is never used in the RQC API.)
Cut off longer values at the end for the POST request,
or the request will fail.
The field ordering is not relevant.
This is what each field means and how to use it:
api_version: The fixed string "1.0alpha".
You will find this in all GET responses from RQC
but must not include it in your POST requests.mhs_software: The name and version number of the
MHS software which has issued the call.
This is used by human beings on the RQC side for support and debugging.mhs_adapter_version: The name and version number of the small part
of the overall MHS software which has actually issued the call.
This may be a plugin or extension or other separate part and can hence
be versioned independently of the MHS itself.
The information is used by human beings on the RQC side for support and debugging.interactive_user: An email address, describing who has just triggered
this request interactively (see steps 6 and 12 of the use case).
If the call was produced such that no user would consider themselves to
have made it, submit "".mhs_submissionpage: In the response to your POST request, RQC will include
a URL to which you should redirect the interactive user's browser to
bring them to an appropriate RQC grading page;
see the discussion of the response status codes below.
After grading is done, RQC should redirect the user back to the MHS.
The mhs_submissionpage describes where this latter redirection should go
(perhaps the same page from which the POST request was triggered).
If interactive_user is "", this can be "" as well.title: The title of the submission.
(If subtitles are stored separately in the MHS, feel free to leave them out).visible_id: The submission ID of the submission as it should be
shown to users on the RQC GUI.submission_id that is part of
the URL, except when submission_id is simplified because
visible_id contains characters not allowed in the URL,
such as slashes or blanks.
In that case,
visible_id is the true submission ID from the MHS' point of view
whereas
the submission_id is the true submission ID from RQC's point of view.
Ideally, they are identical.submitted: ISO-8601 UTC datetime string for the time when the
submission was received at the MHS.author_set: An unordered array of author objects,
each describing one author of the submission.
Each author object has the following structure:email: The author's email address.
Within one submission, each author must have a different email address.
If subsequent authors have the same email address, include only
the firsst of them that has is_corresponding in the API call.firstname: The author's firstname (given name);
or "" if only a fullname is stored in the MHS. lastname: The author's lastname (family name);
or the author's fullname if only that is stored in the MHS.orcid_id: The author's ORCID ID if known or "" otherwise.is_corresponding: true if this is a corresponding author or
false if not.
Depending on the RQC configuration chosen by a journal,
corresponding authors may be integrated into
the review grading process.
(For non-corresponding authors, you could submit empty strings
for email, firstname, lastname, and orcid_id.) order_number: an integer indicating author ordering.
The first author has a 1, the second author a 2, and so on.
All order numbers must be different. editor_set: An unordered array of editor objects,
each describing one editor who is somehow concerned with the
editorial process for this particular submission.
Each editor object has the following structure:
email, firstname, lastname, orcid_id: as for authors.
However, the same editor is allowed to occur more than once
in the same submission
if s/he is assigned to the submission with more than one editor level.level: An integer with value 1, 2, or 3.For each submission, there must be one or more level-1 editors and
zero or more level-2 and level-3 editors.
- review_set: An unordered array of review objects,
each describing one review requested (but not necessarily received)
for this submission.
Each review object has the following structure:
- visible_id: A "name" for the review within the given submission.
Typically simply a number string like "1", "2" etc.
- invited: When the reviewer was invited to the review.
Or "" if this is unknown.
- agreed: When the reviewer agreed to provide the review.
Or "" when this is unknown.
If a reviewer has never responded to the invitation, they are not a
reviewer in the RQC sense and must be left out of the request.
- expected: The deadline agreed for providing the review.
Or "" if no deadline was set.
RQC needs at least either invited/agreed or expected to work
and will reject submissions where none of the three is supplied.
- submitted: When the review was actually supplied.
Or "" if the review was agreed to but has not yet arrived.
- text: The text (body) of the review.
- is_html: true if text is an HTML fragement,
false if it is plain text (which will then be HTML-escaped when shown).
HTML will be sanitized and allows a limited set of tags only (mostly
bold, italics, paragraphs, lists, headings, and anchors/hyperlinks).
- suggested_decision:
What the review suggests the editorial decision should be.
Only five different values are allowed:
"" means the review makes no suggestion.
"accept" means "please accept the submission as is".
"minorrevision" means "probably accept, but minor changes are needed".
"majorrevision" means "major changes are needed, resubmit for another review then".
"reject" means "please reject the submission".
- reviewer: A nested object describing the reviewer:
- email, firstname, lastname, orcid_id: as for authors.
- Opt-out: If a reviewer has opted out (see Sections 2.4.2 and 2.5.2),
submit the reviewer's reviewer_pseudonym as the value of email,
and submit empty strings as the values of
firstname, lastname, orcid_id, and the review's text,
- decision: The editorial decision made for this submission.
Works much like suggested_decision above.
The empty string means the decision has not yet been made.
A decision must eventually be submitted for each submission.
Allowed non-empty values:
- "accept": The submission is accepted.
- "minorrevision": The submission is provisionally accepted.
The authors are asked to prepare a reworked version of the
submission, but no full round of further reviewing is expected.
- "majorrevision": The authors are asked to prepare a
reworked version of the submission, which will then undergo
a fresh round of reviewing with the same or different reviewers.
- "reject": The submission is rejected.
If a journal uses a different set of decision types, match those to their
closest equivalent in the above set.
If an MHS allows each journal to define an arbitrary list of decision codes,
and the MHS cannot know their meaning, the RQC adapter for that MHS must
allow the editors to define a mapping from the journal's internal codes
to the above codes for RQC. (Not fun, sure, but there is no way around it.
Possibly an Excel upload is a suitable compromise between easy-to-build and
easy-to-use.)
RQC will respond to such a POST request with one of the following
status codes:
200 OK: The call was valid and has been accepted.
No redirect URL has been supplied.303 See Other: The call was valid and has been accepted.
The MHS should redirect the interactive user's browser to the URL
provided in the response body.400 Bad request: The request was semantically invalid.
The response body will contain suitable error messages to understand
the problem or problems.403 Forbidden: The token is wrong.
If the MHS had properly validated the API key, this presumably means
that the API key has changed at the RQC side.
In that case, the journal editors should be alerted because no subsequent
API call is going to be successful.
The response body will contain an informative message meant to be
displayed to the users.404 Not found: The whole URL was syntactically malformed or
the journal_id does not exist at RQC.POST callsOnce a submission has been POSTed to RQC, it can be updated
by additional calls only in the following respects:
GETTO DO.!!!
The RQC functionality for journals and hence the API are not yet implemented. Implementation is under way.
Until both the RQC-side API implementation and the first MHS adapter using the API are finished, the API is considered provisional and is subject to sudden change.
If you want to start implementing the API for your manuscript handling system now, please contact Lutz Prechelt.