Define review criteria

The review criteria in Converia determine the standards by which the reviewers evaluate submissions. The program committee will also make the final evaluation based on many of these criteria. There is a wide range of evaluation options: between individual feedback and strict grading, there are many other gradations.

A classic paper review is shown in Fig. 1 in the frontend and Fig. 2 with the corresponding configuration in the backend.

Criteria groups

The review criteria can be arranged in different criteria groups. In the frontend, the names of the groups are displayed in the heading. These are clearly visible in Fig. 1 (Quality/Decision/Topics).

Creating a new review criterion

Two review criteria are already set up by default in the copy template. If additional criteria are needed, they can be added using the Create new entry button.

1. Determines the name of the criterion for differentiation in the backend.

2. The name that the criterion should have in the frontend (Fig. 1).

3. Additional information for the criterion can be added in the description.

4. Determines the reference of the criterion. If the same criteria are to be used in the abstract and manuscript review, the reference must be adjusted before the manuscript review begins. The following references are possible: a) Abstract b) Manuscript

5. Determines the type of criterion. The following types are available: a) Grade/Quality b) Classification c) Presentation d) Internal comment

6. Decide whether the criterion should be a mandatory field. The following options are available: a) Always mandatory b) Optional c) Mandatory if paper is not accepted d) Optional if paper is not accepted

7. Whether any points should be taken into account.

8. Assignment to a criteria group.

9. Short title for the criterion, used as a column title in the final review.

Grade/Quality

If a criterion of the type Grade/Quality has been created, the answers displayed in the frontend can be configured in the Answers tab.

A name, an effect and a value can be defined here. For the effect, a distinction is made between no effect and point score not applicable. The former is the default for a criterion of the type Grade/Quality. If, for example, there is an additional option to abstain from the review, the effect point score not applicable would be necessary so that the assigned value is not included in the review and distorts averages.

Classification

If a criterion of the type Classification has been created, the answers output in the frontend can be configured in the Answers tab.

The name and effect are assigned to the selection options here. The effects of the evaluation criterion are also required for step 3 in the workflows, since the classification of the abstracts is based on the review criterion.

The effects of the review criterion of the type classification control the colored display of the reviewed papers in the frontend:

Presentation

In order for the program committee to be able to adjust the presentation form in the final review, a criterion of the type presentation must be created.

Here, appropriate effects can be assigned to the presentation forms. The effects refer to the presentation forms created in the configuration of the submission. In most cases, the effect of the same name is assigned to the presentation form.

Internal comment

An internal comment can be used to record messages or assessments from the reviewers to the program committee. It is essential for an internal comment criterion to configure the input field type to single-line or multi-line on the Answers tab.

Power session reviewing criteria