API version: 2022-10-20
This is a work-in-progress and not yet ready for use!
Elsewhere:
For RQC to be practical, it needs to be integrated with the manuscript handling system (MHS) to which the reviews are submitted.
For conferences, RQC can play the active part: All reviews are present at a particular time (the reviewing deadline, or shortly later) and RQC can go and get all the submission, reviewer, and reviewing data from the manuscript handling system (after the PC chairs have permitted it to do so) and then do its job with them.
For journals, this mode of operation does not work: Reviews trickle in all the time and it makes no sense to wait with grading them until, say, the end of the year. So the manuscript handling system needs to actively transmit the data for reviews to RQC as they come in. RQC needs to provide a technical way to do that: The API (application programming interface).
Overall, the process for handling one submission to journal X works roughly as follows:
Actors: The journal's manuscript handling system (MHS), Editor, Review Quality Collector system (RQC), Reviewers
Before the above can happen, the MHS must be equipped with the journal-specific API key (like a password) that it must use to authenticate itself to RQC.
Actors: The journal's manuscript handling system (MHS), Publisherperson, RQguardian, Review Quality Collector system (RQC).
Variant: 3b: If RQC rejects the API key, the MHS shows the error message provided by RQC and does not store the key.
The above functionality of use case 2.1 is available only after a successful step 3, not in the case of 3b.
So far, the RQC API has only two endpoints: mhs_submission
and mhs_apikeycheck
.
mhs_apikeycheck
must be used first, using HTTP GET, for use case 2.2.
mhs_submission
is used, using HTTP POST, for use case 2.1.
It can also be used with HTTP GET for testing purposes to validate that
the data has arrived at RQC as expected. HTTP GET is not required otherwise.
For each submission that undergoes review at the MHS, submission
is
usually called twice (and in rare cases more):
First to deposit the submission, editors, and reviewing data.
This triggers the grading process at RQC.
Later to add information about the editorial decision and possibly
any additional reviews that may have arrived in the meantime.
Responses are JSON objects. If there is any semantic problem with a call, the response will contain a detailed error message.
The remainder of Section 2 will explain how to implement an RQC adapter for an MHS that makes use of the API.
RQC requires the following data for each submission (which thus needs to be stored in the MHS database somewhere), as far as available:
(The above are not the actual names used by RQC, see further down.)
For some of these, reasonable defaults are available; in particular,
firstname can be left empty if unavailable and the full name supplied
in the lastname field; if ORCID IDs are not known, that field can be
left empty.
Other fields are important; in particular, RQC uses email addresses
as keys for identifying people, so if several authors share an email
address, there is trouble and only one of them can be reported to RQC.
This is described in more detail in the request descriptions below.
Two fields will have to be added to each journal's master data in the MHS:
rqc_journal_id
: an integer supplied by RQC when registering the journal.
This value is public.rqc_journal_apikey
: an ASCII string (maximum length 64) created by RQC
upon request. This value is secret and is used for authentication
as described below.Beyond that, it may be needed to store an extra mapping of the codes
for editorial decisions; see under decision
below.
This might also require an extension of the MHS GUI to set those values.
Further fields
will have to be added to the journal transaction data
in the MHS; they describe which reviewers have
opted in to or opted out of participating in RQC for this year for this journal:
journal_id
: The primary key of this journal within the MHS. reviewer_email
: The email address that represents the revieweropting status
: True for "reviewer has opted in",
False for "reviewer has opted out".year
: The year to which the decision pertains.
(This is not what you probably think it is;
see the discussion of opting below.) Of course you need not realize it exactly like this. You must merely be able to represent opting status with respect to the current year per reviewer per journal. (If the journal has subdivisions, whether to represent them or not depends on whether they are represented at RQC as a single journal or as multiple journals. Opting status is per RQC-side journal.)
An RQC adapter for a given MHS must provide the following functionality:
The MHS must allow RQguardians or Publisherpersons to enter the rqc_journal_id
and
rqc_journal_apikey
they received from their publisher for the given
journal, and the MHS must validate these data upon entry by issuing an
mhs_apikeycheck
call.
Invalid data must not be accepted and must never be used for RQC calls.
The dialog by which a reviewer submits a review must be extended: If there is no entry yet for this reviewer, journal, and year in the table described in Section 2.4.2 above, the MHS must ask the reviewer if they want to opt into participating in RQC for this journal for this year or want to opt out of participating in RQC for this journal for this year.
Use the following wording for the question:
I want to participate in Review Quality Collector (RQC) for all my reviews for this journal this year. My reviewing data will be submitted to RQC and RQC will provide me with a reviewing receipt in exchange. I understand the purpose of RQC and its use of my data as described at https://reviewqualitycollector.org/t/terms-and-privacy. I can make a separate decision on this for other journals. I can make a separate decision on this for the present journal next year.
Yes / No
Answering yes creates a contract between the reviewer and RQC where the reviewer provides a review and RQC provides a reviewing receipt. This contract is the basis on which RQC is allowed (by privacy regulations such as the EU-GDPR) to process the reviewer's data. Different wording would provide a shakier basis and must not be used.
After the reviewer has answered the above dialog, create an entry for it and do not ask this reviewer again for reviews pertaining to the same year and journal.
(Don't worry: This was the most complicated aspect of what the MHS needs to do.)
submission
API endpoint.
If one or more of the invited reviews have not yet arrived, the
GUI should ideally show a warning dialog
"You are about to initiate the review grading process for this
submission, but not all reviews have yet arrived.
Reviews arriving only after this step will likely not be included in
the grading process." (Options: Proceed, Cancel).mhs_apikeycheck
URLs have the following form:
https://reviewqualitycollector.org/api/mhs_apikeycheck/<journal_id>
mhs_submission
URLs have the following form:
https://reviewqualitycollector.org/api/mhs_submission/<journal_id>/<external_uid>
journal_id
is an integer value assigned to the journal by RQC.
When registering a journal for use with RQC, RQC will show this value.
It must be manually transfered to the MHS.external_uid
is the URL-encoded version of the unique
identifier string assigned to a given submission by the MHS.
MHSs often call it the submission number.external_uid
has been transformed according
to the rules of RFC 3986 by means of percent-encoding ("quoting") such that
it contains only characters valid in a URL and in particular no longer
contains the '/' (slash) character.The API key that is required for both calls is transmitted in an HTTP header as follows:
Each call to the API must provide the following HTTP headers:
X-RQC-API-Version
: The value of "API version" given at the top of the
present document at the time when the MHS adapter for RQC was developed.
Example value: "2022-10-19"
.X-RQC-MHS-Version
: A free-form string. The name and version number of the
MHS software which has issued the call.
Example value: "Open Journal Systems 3.5.1"
X-RQC-MHS-Adapter
: A free-form string. The version number (and perhaps name) of the small part
of the overall MHS software which has actually issued the call.
Example value: "RQC plugin 1.1.17"
X-RQC-MHS-Version
header again.
The information is used by human beings on the RQC side for support and debugging.X-RQC-Time
: The MHS server's current time in UTC timezone
in ISO 8601 format: YYYY-MM-DDThh:mm:ssZ, e.g. 2022-10-18T21:02:58Z.
Such datetime timestamps are important for RQC's work and so
RQC will compare this time to its own and will reject the call if
the MHS is more than 60 seconds off.Authorization
: A string in RFC 6750 "Bearer" authorization format.
Provides the journal-specific API key as deposited by use case 2.2.Bearer qpwoeirutzalskdjfhgy
Content-Type: application/json
(POST calls only).
All POST calls provide their data as UTF-8-encoded JSON in the body.
See json.org
or JSON at Wikipedia.mhs_apikeycheck
API endpointA GET call to this endpoint validates a pair of
rqc_journal_id
and rqc_journal_apikey
.
The call must be made by the MHS when (and only when) a user attempts to save
initial or modified settings of these two values.
The call must include the headers as described in Section 2.7.
If the validation is successful, calls to the submission
endpoint may subsequently be made.
If the validation fails, calls to the submission
endpoint must not be made,
because no such calls would succeed and
the editors would be confused by the resulting error messages.
RQC will respond to such a GET request with one of the following status codes:
200 OK
: The validation was successful.400 Bad request
: The request was semantically invalid; the validation was unsuccessful.
The JSON response body will contain suitable error messages to understand
the problem or problems. This is an unusual case.403 Forbidden
: The API key is wrong; the validation was unsuccessful.
The JSON response body will contain an informative message meant to be
displayed to the user.404 Not found
: The whole URL was malformed or
no journal with the given journal_id
exists at RQC; the validation was unsuccessful.mhs_submission
API endpointExplicit calls: The MHS should provide a button or link "RQC-grade reviews" on the editors' submission administration page as soon as at least one review has been received. The editor can use this GUI element to submit the currently existing reviews for this submission to RQC, will be immediately redirected to RQC for grading these reviews, and then be redirected back to the MHS. Using this functionality is called an "explicit call". Explicit calls are optional: Grading the review can be a useful preparatory step for making the editorial decision, but it is not necessary to perform the grading at this point -- and possibly this particular editor is not involved in the grading at all. More than one editor may make an explicit call.
Implicit calls: Whenever an editorial decision for a submission is made (or changed) in the MHS, the MHS must make an "implicit call" to RQC and submit the review data. The MHS may or may not redirect the user to RQC at this point as it sees fit, but the recommended behavior it to not redirect to RQC as that might be confusing to the editor. Implicit calls are mandatory, they must be made for each submission each time a new editorial decision is stored, because the behavior of RQC may depend on the decision.
Delayed calls: A call to RQC may fail due to network problems or
unavailability of the RQC service.
If that happens, the call should be put into a backlog of calls to retry.
This backlog should be revisited once per day and the first of the calls in the
queue submitted again.
If a call succeeds, remove it from the queue.
If it fails, count the failure, move the call to the end of the queue,
stop retrying any calls for that day,
and retry the next call on the queue the next day. Delete calls after
10 failures.
Again: If any call fails on a given day, stop all retrying for that day in order
to avoid useless calls.
Note that this behavior applies only to unavailability failures; it must not
be applied if the authentication token is wrong.
See the response status codes below for a discussion of the redirection mechanism and the discussion of consequences for design considerations.
Here is an example what that JSON may look like; an explanation of each field follows below:
{
"interactive_user": "myeditor@anywhwere.org"
"mhs_submissionpage": "https://mymhs.example.com/journal17/user29?submission=submission31"
"title": "Rather Altogether Modestly Long-ish Submission Title 21",
"external_uid": "MANU-10021/R2",
"visible_uid": "MANU-10021/R2",
"submitted": "2022-02-01T21:07:42Z",
"author_set": [
{
"email": "author31@sm.wh",
"firstname": "A.N.",
"lastname": "Author",
"orcid_id": "0000-0001-5592-0005",
"order_number": 1
},
{
"email": "other_author@sm.wh",
"firstname": "Brubobolob",
"lastname": "Authoress",
"orcid_id": "0000-0001-5592-0006",
"order_number": 3
}
],
"editorassignment_set": [
{
"email": "editor1@sm.wh",
"firstname": "Keen",
"lastname": "Editorus",
"orcid_id": "0000-0001-5592-0003",
"level": 1
},
{
"email": "editor2@sm.wh",
"firstname": "Keener",
"lastname": "Editora",
"orcid_id": "0000-0001-5592-0004",
"level": 3
},
{
"email": "editor3@sm.wh",
"firstname": "Kim",
"lastname": "Nguyen",
"orcid_id": "0000-0001-5592-0007",
"level": 3
}
],
"review_set": [
{
"visible_id": "1",
"invited": "2022-02-08T14:30:10Z",
"agreed": "2022-02-09T07:22:03Z",
"expected": "2022-02-28T14:30:10Z",
"submitted": "2022-02-15T09:09:46Z",
"text": "This is the text of review 1.",
"is_html": false,
"suggested_decision": "MINORREVISION",
"reviewer": {
"email": "reviewer1@sm.wh",
"firstname": "J.",
"lastname": "Reviewer 1",
"orcid_id": "0000-0001-5592-0001"
}
},
{
"visible_id": "2",
"invited": "2022-02-15T00:00:00Z",
"agreed": "",
"expected": "2022-02-28T00:00:00Z",
"submitted": "2022-03-01T00:00:00Z",
"text": "This is the text of review 2.\nIt is multiple lines long, some of which are themselves a bit longer (like this one, for instance; at least somewhat -- truly long would be longer, of course)\n\nLine 4, the last one.",
"is_html": false,
"suggested_decision": "ACCEPT",
"reviewer": {
"email": "reviewer2@sm.wh",
"firstname": "J.",
"lastname": "Reviewer 2",
"orcid_id": "0000-0001-5592-0002"
}
}
]
"decision": "ACCEPT",
}
(Please make sure you correctly
discriminate integer literals from corresponding strings (1
vs "1"
)
and the three special JSON literals true
, false
, and null
from
their unrelated string look-alikes "true"
, "false"
,
and "null"
. null
is never used in the RQC API.)
Cut off longer values at the end for the POST request, or the request will fail.
This section describes the meaning of the data shown in the example in Section 2.8.1. The field ordering is not relevant.
This is what each field means and how to use it:
interactive_user
: An email address, describing who has just triggered
this request interactively (direct call, see steps 6 and 12 of the use case).
If the call was produced such that no user would consider themselves to
have made it (implicit calls and delayed calls), submit ""
.mhs_submissionpage
: In the response to your POST request, RQC will include
a URL to which you should redirect the interactive user's browser to
bring them to an appropriate RQC grading page;
see the discussion of the response status codes below.
After grading is done, RQC should redirect the user back to the MHS.
The mhs_submissionpage
describes where this latter redirection should go
(perhaps the same page from which the POST request was triggered).
If interactive_user
is ""
, this must be ""
as well.title
: The title of the submission.
(If subtitles are stored separately in the MHS, feel free to leave them out).visible_uid
: The submission ID of the submission as it should be
shown to users on the RQC GUI.external_uid
that is part of
the URL, except that it must not contain blanks and must contain only printable latin characters.
Like external_uid
, it should be unique within this journal throughout the entire lifetime
of the journal, but for visible_uid
this is only a 'should' condition, not a 'must' condition.submitted
: ISO 8601 UTC datetime string for the time when the
submission was received at the MHS.author_set
: An unordered array of author objects,
each describing one corresponding(!) author of the submission.
Non-corresponding authors are simply left out and will be reflected only in
potential gaps in author order_number
values.
Each author object has the following structure:email
: The author's email address.
Within one submission, each author must have a different email address.
If several authors have the same email address, include in the API call only
the first of those.firstname
: The author's firstname (given name);
or ""
if only a fullname is stored in the MHS. lastname
: The author's lastname (family name);
or the author's fullname if only that is stored in the MHS.orcid_id
: The author's ORCID ID if known or ""
otherwise.order_number
: an integer indicating author ordering.
The first author has a 1
, the second author a 2
, and so on.
All order numbers must be different.
Use real author numbers, also counting the non-corresponding authors, if any.
For instance, if only the second and last author of a five-author submission
are corresponding authors, there will be only two author entries (not five),
one with order_number: 2
and one with order_number: 5
.editor_set
: An unordered array of editor objects,
each describing one editor who is somehow concerned with the
editorial process for this particular submission.
Each editor object has the following structure:
email
, firstname
, lastname
, orcid_id
: as for authors.
However, the same editor is allowed to occur more than once
in the same submission
if s/he is assigned to the submission with more than one editor level.level
: An integer with value 1
, 2
, or 3
.For each submission, there must be one or more level-1 editors and
zero or more level-2 and level-3 editors.
- review_set
: An unordered array of review objects,
each describing one review requested (but not necessarily received)
for this submission.
Each review object has the following structure:
- visible_id
: A "name" for the review within the given submission.
Typically simply a number string (not an integer) like "1"
, "2"
etc.
- invited
: Datetime when the reviewer was invited to the review.
Or ""
if this is unknown.
- agreed
: Datetime when the reviewer agreed to provide the review.
Or ""
when this is unknown.
If a reviewer has never responded to the invitation and never sent a review,
they are not a reviewer in the RQC sense and must be left out of the call.
- expected
: The deadline datetime agreed for providing the review.
Or ""
if no deadline was set.
RQC needs at least one of invited
, agreed
, or expected
and will reject submissions where none of the three is supplied.
- submitted
: Datetime when the review was actually supplied.
Or ""
if the review was agreed to but has not yet arrived.
- text
: The text (body) of the review.
- is_html
: true
if text
is an HTML fragement,
false
if it is plain text (which will then be HTML-escaped when shown).
HTML will be sanitized and allows a limited set of tags only (mostly
bold, italics, paragraphs, lists, headings, and anchors/hyperlinks).
- suggested_decision
:
What the review suggests the editorial decision should be.
Only five different values are allowed and uppercasing is relevant:
""
means the review makes no suggestion.
"ACCEPT"
means "please accept the submission as is".
"MINORREVISION"
means "probably accept, but minor changes are needed".
"MAJORREVISION"
means "major changes are needed, resubmit for another review then".
"REJECT"
means "please reject the submission".
- reviewer
: A nested object describing the reviewer:
- email
, firstname
, lastname
, orcid_id
: as for authors.
- Opt-out: If a reviewer has opted out (see Section 2.5.2),
submit a pseudo-address (see below) as the value of email
instead of the real email address,
and submit empty strings as the values of
firstname
, lastname
, orcid_id
, and the review's text
.
The pseudo-address is formed as follows (pseudocode):
hash = sha1_hex(email + salt)
pseudo_address = hash + "@example.edu"
where salt
is an arbitrary fixed(!) secret value.
Example: Assume our reviewer has email address reviewer1@some.where
and your MHS's salt is dmeugqpglkw
.
Then you compute the SHA-1 hash of reviewer1@some.wheredmeugqpglkw
which is 79d3c1099f8d7ebaa6afc3095b2f5fb677650880
and get
79d3c1099f8d7ebaa6afc3095b2f5fb677650880@example.edu
.
It is not important whether you use SHA-1 or some other hash function;
it is not important what your salt is or that it be kept super-secret.
What is important is this
- that the same emailaddress always maps to the same pseudo-address,
- that all pseudo-addresses be different, and
- that the pseudo-address ends with @example.edu
, so that RQC can recognize it is a pseudo-address.
- decision
: The editorial decision made for this submission.
The values are like for suggested_decision
above.
The empty string means the decision has not yet been made.
A decision must eventually be submitted for each submission.
Allowed non-empty values:
- "ACCEPT"
: The submission is accepted.
- "MINORREVISION"
: The submission is provisionally accepted.
The authors are asked to prepare a reworked version of the
submission, but no full round of further reviewing is expected.
- "MAJORREVISION"
: The authors are asked to prepare a
reworked version of the submission, which will then undergo
a fresh round of reviewing with the same or different reviewers.
- "REJECT"
: The submission is rejected.
If a journal uses a different set of decision types, match those to their closest equivalent in the above set. Uppercasing is relevant. If an MHS allows each journal to define an arbitrary list of decision codes, and the MHS cannot know their meaning, the RQC adapter for that MHS must make the editors define a mapping from the journal's internal codes to the above codes for RQC. (Not fun, sure, but there is no way around it. Possibly an Excel upload is a suitable compromise between easy-to-build and easy-to-use.) No API calls can be made until that mapping is known and can be used.
RQC will respond to such a POST request with one of the following status codes:
200 OK
: The call was valid and has been accepted.
There are two cases:redirect_url
field.
This happens when interactive_user
and mhs_submissionpage
were set in the call.
The MHS should redirect the interactive user's browser to the URL given in redirect_url
.
This will be a page in RQC where the submission can be assigned to a subjournal and then
graded. If the user logged into RQC (or who now logs into RQC) in that browser
is not assigned to the submission as an editor or has already graded the submission before,
RQC will redirect back to the mhs_submissionpage
rightaway.
Otherwise, RQC will redirect back to the mhs_submissionpage
automatically right after the grading.redirect_url
field.
The MHS should simply carry on as usual.400 Bad request
: The request was semantically invalid.
The response body will contain suitable error messages to understand
the problem or problems.403 Forbidden
: The API key is wrong.
If the MHS had previously validated the API key, this presumably means
that the API key has changed at the RQC side.
In that case, the journal editors should be alerted because no subsequent
API call is going to be successful.
The response body will contain an informative message meant to be
displayed to the user.404 Not found
: The whole URL was malformed or
no journal with the given journal_id
exists at RQC.Once a submission has been sent to RQC, it should normally be updated by additional calls only in the following respects:
decision
value addedreview_set
If other changes occur, e.g. in the authors list or the set of editor assignments, RQC may or may not react properly (or at all) to those.
No further updates are possible at all once the respective journal year
is no longer active (which is after February of the next year).
Such calls will result in a 400 Bad request
response.
You can also call the submission
API endpoint with the HTTP GET method.
If the headers are correct, you will get either a response with status code 200 OK
and
a JSON body equivalent to that previously used for POST
or get a 404 Not found
response if no successful POST had previously occurred.
The RQC functionality for journals (including the API) and one corresponding MHS adapter (for Open Journal Systems, OJS) are currently nearing the completion of their first version. Until both are finished, the API is considered provisional and is subject to sudden change.
If you want to start implementing the API for your manuscript handling system now, please contact Lutz Prechelt.
EdAssgmentSY
inherit from pid.Person
(firstname, lastname and properties)
and from TimeStampedModel
.subm.computed_token
if possible