last change: 2019-10-01
This is a work-in-progress and not yet ready for use!
For RQC to be practical, it needs to be integrated with the manuscript handling system (MHS) to which the reviews are submitted.
For conferences, RQC can play the active part: All reviews are present at a particular time (the reviewing deadline, or shortly later) and RQC can go and get all the submission, reviewer, and reviewing data from the manuscript handling system (after the PC chairs have permitted it to do so) and then do its job with them.
For journals, this mode of operation does not work: Reviews trickle in all the time and it makes no sense to wait with grading them until, say, the end of the year. So the manuscript handling system needs to actively transmit the data for reviews to RQC as they come in. RQC needs to provide a technical way to do that: The API (application programming interface).
Overall, the process for handling one submission to journal X works roughly as follows:
Actors: The journal's manuscript handling system (MHS), Editor, Review Quality Collector system (RQC), Reviewers
So far, the RQC API has only one single endpoint.
The URLs to reach it are described in a separate section below.
The endpoint is normally used twice for each submission, both
times using the PUT HTTP verb:
First to deposit the submission, editors, and reviewing data.
Later to add information about the editorial decision (and possibly
any additional reviews that may have arrived in the meantime).
Data supplied in the initial PUT will not be modified by later calls.
HEAD is supported for checking the validity of a pair of journal ID
and journal key.
GET is supported solely for testing purposes; it is not needed
for normal operation. (TODO: Report grading summary information
in the GET response so the MHS can show grading status information.)
DELETE is not supported because review grading is initiated once reviews
are received by RQC and RQC will not remove gradings.
All URLs have the following form:
https://reviewqualitycollector.org/j/mhsapi/<journal_id>/<submission_id>/?token=<token>
where
journal_id is an integer value assigned to the journal by RQC.
When registering a journal for use with RQC, RQC will show this value
and it must be manually transfered to the MHS.submission_id is a (possibly simplified version of a) string
assigned to a given submission by
the Editor or the MHS.token is an authentication token as described further down.RQC requires the following data for each submission (which thus needs to be stored in the MHS database somewhere):
(These are not the actual names used by RQC, see further down.)
For some of these, reasonable defaults are available; in particular,
firstname can be left empty if unavailable and the full name supplied
in the lastname field; if ORCID IDs are not known, that field can be
left empty.
Other fields are very important; in particular, RQC uses email addresses
as keys for identifying people, so if several authors share an email
address, there is trouble and only one of them can be reported to RQC.
This is described in more detail in the request descriptions below.
Two fields will have to be added to each journal's master data in the MHS:
journal_id: an integer supplied by RQC when registering the journal.
This value is public.journal_key: an ASCII string (maximum length 64) created by RQC
upon request. This value is secret and is used for authentication
as described below.Beyond that, it may be needed to store an extra mapping of the codes
for editorial decisions; see under decision below.
This would also require an extension of the MHS GUI to set those values.
journal_id and
journal_key they received from their publisher for the given
journal and the MHS must validate these data upon entry by issuing a
HEAD call as described below (and get a 200_OK response status).
Invalid data must not be accepted, in order to avoid making lots of
invalid RQC API calls.PUT call to the API endpoint.
If one or more of the invited reviews have not yet arrived, the
GUI should ideally show a warning dialog
"You are about to initiate the review grading process for this
submission, but not all reviews have yet arrived.
Reviews arriving only after this step will likely not be included in
the grading process." (Options: Proceed, Cancel).PUT call to the API endpoint to notify
RQC of the decision. This (usually second) call also includes
all submission and reviewing data and will step in if the first
call was never made.Authentication is performed via the token submitted in the URL
with each call.
A different token must be supplied for each combination of
journal ID and submission ID as follows:
submission_id.
Let us assume the ID is EASE-2017-012.R1.journal_key corresponding to the journal_id.
Let us assume the key is abcdefg.submission_id and journal_key, giving
EASE-2017-012.R1abcdefg.b880b7969c78c5e5c71963f6130ae2204ab8b5d0!!!.token for the given pair of
journal ID and submission ID.The token is required for all calls.
If it is wrong (and only then), the response will have
HTTP status code 401 Unauthorized.
This fact will not be repeated in the response code listings below.
HEADA HEAD request must be used by the MHS to make sure a pair of
journal_id and journal_key is indeed valid.
This is useful for validating the data input by an editor in a
journal configuration form, because if journal_id or journal_key are
wrong, none of the later RQC calls would succeed and the corresponding
reviews would never reach RQC evaluation.
For this purpose, the URL is formed differently, as follows:
https://reviewqualitycollector.org/j/mhsapi/<journal_id>/<datetime>/?token=<token>
where
journal_id is the same as described above.datetime is an ISO-8601 datetime string for the current time
in timezone UTC ("zero-offset time", "zulu time").yyyy-mm-ddThh:mm:ssZ, for example:
2017-12-02T13:56:01Z;
see articles ISO 8601
and UTC
in Wikipedia.token is like described above,
except that the datetime replaces the submission_id
in the computation.The purpose of submitting current time in the URL is to make sure the clocks of MHS and RQC are properly aligned.
RQC will respond to such a HEAD request with one of the following
status codes:
200 OK: The journal_id exists, the token correctly reflects the
corresponding journal key, and the datetime was within 60 seconds of
RQC's time. 400 Bad request: The datetime string was malformed or
the datetime is off by more than 60 seconds compared
to RQC's clock. The response body will contain details for human debugging.403 Forbidden: The token is wrong. It was either computed in the
wrong fashion or the underlying journal_key is invalid for the given
journal_id.
The response body will contain details for human debugging.404 Not found: The whole URL was syntactically malformed or
the journal_id does not exist at RQC.PUTExplicit calls: The MHS should provide a button or link "RQC-grade reviews" on the editors' submission administration page as soon as at least one review has been received. The editor can use this GUI element to submit the currently existing reviews for this submission to RQC, will be immediately redirected to RQC for grading these reviews, and then be redirected back to the MHS. Using this functionality is called an "explicit call". Explicit calls are optional: Grading the review is a nice preparatory step for making the editorial decision, but it is not necessary to perform the grading at this point -- and possibly this particular editor is not involved in the grading at all. More than one editor may make an explicit call.
Implicit calls: Whenever an editorial decision for a submission is made (or changed) in the MHS, the MHS must make an "implicit call" to RQC and submit the review data. The MHS may or may not redirect the user to RQC at this point as it sees fit. Implicit calls are mandatory, they must be made for each submission each time a new editorial decision is stored.
Delayed calls: A call to RQC may fail due to network problems or
unavailability of the RQC service.
If that happens, the call should be put into a backlog of calls to retry.
This backlog should be revisited once per day and each of the calls in the
queue submitted again.
If a call succeeds, remove it from the queue.
If it fails, count the failure and retry the next day. Delete calls after
10 failures.
If any call fails on a given day, stop all retrying for that day in order
to avoid useless calls.
Note that this behavior applies only to unavailability failures; it should not
be applied if the authentication token is wrong or no
configuration has deposited in RQC for a given submission type.
See the response status codes below for a discussion of the redirection mechanism and the discussion of consequences for design considerations.
The calls must have a Content-Type: application/json HTTP header
and provide their data as UTF-8-encoded
JSON in the body.
See json.org
or JSON at Wikipedia.
(If you absolutely need to provide your data as XML, speak up;
I may be willing to support it.)
Here is an example what that JSON may look like; an explanation of each field follows below:
{
"api_version": "1.0alpha", // GET only, not needed in PUT
"interactive_user": "myeditor@anywhwere.org"
"mhs_redirect_url": "https://mymhs.example.com/journal17/user29?submission=submission31" // PUT only, not present in GET
"title": "Rather Altogether Modestly Long-ish Submission Title 21",
"visible_id": "MANU-10021/R2",
"submitted": "2017-02-01T00:00:00Z",
"predecessor_submission_id": "MANU-10021-R1",
"predecessor_visible_id": "MANU-10021/R1",
"author_set": [
{
"email": "author31@sm.wh",
"firstname": "A.N.",
"lastname": "Author",
"orcid_id": "0000-0001-5592-0005"
"is_corresponding": true,
"order_number": 1
},
{
"email": "other_author@sm.wh",
"firstname": "Brubobolob",
"lastname": "Authoress",
"orcid_id": "0000-0001-5592-0006"
"is_corresponding": true,
"order_number": 2
}
],
"editorassignment_set": [
{
"email": "editor1@sm.wh",
"firstname": "Keen",
"lastname": "Editorus",
"orcid_id": "0000-0001-5592-0003"
"level": 1
},
{
"email": "editor2@sm.wh",
"firstname": "Keener",
"lastname": "Editora",
"orcid_id": "0000-0001-5592-0004"
"level": 3
}
],
"review_set": [
{
"visible_id": "0",
"invited": "2017-02-08T00:00:00Z",
"agreed": "2017-02-09T00:00:00Z",
"expected": "2017-02-28T00:00:00Z",
"submitted": "2017-02-15T00:00:00Z",
"text": "This is the text of review 1.",
"is_html": false,
"suggested_decision": "minorrevision",
"reviewer": {
"email": "reviewer1@sm.wh",
"firstname": "J.",
"lastname": "Reviewer 1",
"orcid_id": "0000-0001-5592-0001"
}
},
{
"visible_id": "1",
"invited": "2017-02-15T00:00:00Z",
"agreed": "2017-02-15T00:10:00Z",
"expected": "2017-02-28T00:00:00Z",
"submitted": "2017-03-01T00:00:00Z",
"text": "This is the text of review 2.",
"is_html": false,
"suggested_decision": "accept",
"reviewer": {
"email": "reviewer2@sm.wh",
"firstname": "J.",
"lastname": "Reviewer 2",
"orcid_id": "0000-0001-5592-0002"
}
}
]
"decision": "accept",
}
(Please make sure you correctly
discriminate integer literals from corresponding strings (1 vs "1")
and the three special JSON literals true, false, and null from
their unrelated string look-alikes "true", "false",
and "null". null is never used in the RQC API.)
The field ordering is not relevant.
This is what each field means and how to use it:
api_version: The fixed string "1.0alpha".
You will find this in all GET/HEAD/PUT responses from RQC
but must not include it in your PUT requests.interactive_user: An email address, describing who has just triggered
this request interactively (see steps 6 and 12 of the use case).mhs_redirect_url: In the response to your PUT request, RQC will include
a URL to which you should redirect the interactive user's browser to
bring them to an appropriate RQC grading page;
see the discussion of the response status codes below.
After grading is done, RQC should redirect the user back to the MHS.
The mhs_redicrect_url describes where this latter redirection should go
(e.g. the same page from which the PUT request was triggered).title: The title of the submission.
(For brevity, leave out subtitles if those are stored separately in the MHS).visible_id: The submission ID of the submission as it should be
shown to users on the RQC GUI.submission_id that is part of
the URL, except when the latter may be simplified because
visible_id contains characters not allowed in the URL,
such as slashes or blanks.
In that case,
visible_id is the true submission ID from the MHS' point of view
whereas
the submission_id is the true submission ID from RQC's point of view.
Ideally and usually, they are identical.submitted: ISO-8601 UTC datetime string for the time when the
submission was received at the journal.predecessor_submission_id: If the data record describes the first
round of reviewing of the given work, this value must be null.
Otherwise (for a revised version of the work),
it must contain the submission_id used for the previous
submission of the work.predecessor_visible_id: If the data record describes the first
round of reviewing of the given work, this value must be null.
Otherwise (for a revised version of the work),
it must contain the visible_id used for the previous
submission of the work.author_set: An unordered array of author objects,
each describing one author of the submission.
Each author object has the following structure:email: The author's email address.
Within one submission, each author must have a different email address. firstname: The author's firstname (given name);
or "" if only a fullname is stored in the MHS. lastname: The author's lastname (family name);
or the author's fullname if only that is stored in the MHS.orcid_id: The author's ORCID ID if known or the empty string otherwise.is_corresponding: true if this is a corresponding author or
false if not.
Depending on the RQC configuration chosen by a journal,
corresponding authors may be integrated into
the review grading process.
Non-corresponding authors will never be contacted.order_number: an integer indicating author ordering.
The first author has a 1, the second author a 2, and so on.
All order numbers must be different. editor_set: An unordered array of editor objects,
each describing one editor who is somehow concerned with the
editorial process for this particular submission.
Each editor object has the following structure:
email, firstname, lastname, orcid_id: as for authors.
However, the same editor is allowed to occur more than once
in the same submission
if s/he is assigned to the submission with more than one editor level.level: An integer with value 1, 2, or 3.For each submission, there must be one or more level-1 editors and
zero or more level-2 and level-3 editors.
- review_set: An unordered array of review objects,
each describing one review requested (but not necessarily received)
for this submission.
Each review object has the following structure:
- visible_id: A "name" for the review within the given submission.
Typically simply a number string like "1", "2" etc.
- invited: When the reviewer was invited to the review.
Or the empty string if this is unknown.
- agreed: When the reviewer agreed to provide the review.
Or the empty string when this is unknown.
If a reviewer has never responded to the invitation, they are not a
reviewer in the RQC sense and must be left out of the request.
- expected: The deadline agreed for providing the review.
Or the empty string if no deadline was set.
RQC needs at least either invited/agreed or expected to work
and will reject submissions where none of the three is supplied.
- submitted: When the review was actually supplied.
Or the empty string if the review has not yet arrived.
- text: The text (body) of the review.
- is_html: true if text is an HTML fragement,
false if it is plain text (which will then be HTML-escaped when shown).
HTML will be sanitized and allows a limited set of tags only (mostly
bold, italics, paragraphs, lists, headings, and anchors/hyperlinks).
- suggested_decision:
What the review suggests the editorial decision should be.
Only five different values are allowed:
"" (empty string) means the review makes no suggestion.
"accept" means "please accept the submission as is".
"minorrevision" means "probably accept, but minor changes are needed".
"majorrevision" means "major changes are needed, resubmit for another review then".
"reject" means "please reject the submission".
- reviewer: A nested object describing the reviewer:
- email, firstname, lastname, orcid_id: as for authors.
- decision: The editorial decision made for this submission.
Works much like suggested_decision above.
The empty string means the decision has not yet been made.
A decision must eventually be submitted for each submission.
Allowed non-empty values:
- "accept": The submission is accepted.
- "minorrevision": The submission is provisionally accepted.
The authors are asked to prepare a reworked version of the
submission, but no full round of further reviewing is expected.
- "majorrevision": The authors are asked to prepare a
reworked version of the submission, which will then undergo
a fresh round of reviewing with the same or different reviewers.
- "reject": The submission is rejected.
If a journal uses a different set of decision types, match those to their
closest equivalent in the above set.
If an MHS allows each journal to define an arbitrary list of decision codes
and the MHS cannot know their meaning, the RQC adapter for that MHS must
allow the editors to define a mapping from the journal's internal codes
to the above codes for RQC. (Not fun, sure, but there is no way around it.)
TO DO.
GETTO DO.
The RQC functionality for journals and hence the API are not yet implemented. Implementation is under way.
Until both the RQC-side API implementation and the first MHS adapter using the API are finished, the API is considered provisional and is subject to sudden change.
If you want to start implementing the API for your manuscript handling system now, please contact Lutz Prechelt.
TO DO.
Eventually, RQC will support test journals that do not show up in official journal lists and from which reviews will automatically be deleted after some time.