Dual Two-Pass screening is a sequential, quality controlled screening process that has two steps. In the first step, two users sequentially Advance or Exclude articles at the Abstract level. Any disagreements at this step are adjudicated by an Admin. In the second step, two users conduct a Full Text Review and Include or Exclude articles. All disagreements this second step must also be adjudicated by an Admin.
Only those with Admin privileges can serve as Adjudicators, but any user can serve as a Screener.
Configure Exclusion Reasons #
You will need to Configuring Exclusion Reasons before screening underlying studies.
Configure Dual Two-Pass Screening #
Navigate to Settings and select “Dual” and “Two-Pass” #
Note: Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, so all records without an adjudicated Include or Exclude decision will be reverted to Unscreened and all data associated with individual users’ decisions will be lost!
Dual Two-Pass Screening Steps: #
1. Screen each study twice at the abstract level. #
Two independent reviewers will need to review the abstract of every study and screen the abstracts using the same approach as Standard Screening Mode with the exception that studies are only advanced to full-text screening at this stage instead of included. AutoLit automatically queues the abstracts to all users until two screening decisions are made; then, the abstracts are sent forward for adjudication.
In Dual modes, it can be useful to view the number of prior reviewers for the current record. This is displayed to the right of the advance button (see below). It is also displayed next to the include button in Full Text screening. 0 means no decisions have been made about the current record, 1 means 1 reviewer has made a decision, and so on.
Even in Abstract Screening, you may want to view the full text, to do so go to the Full Text tab. In Dual Screening types, the status of whether the full text has been uploaded or not is hidden. This is to avoid bias as the knowledge that the other user has uploaded the record’s full text may influence your screening decision. You still have the option to show the full text upload status as well as the full text regardless by clicking “Show Anyways.” This action does not affect your screening decisions and is not shown in the Full Text Screening module since all full texts will be uploaded.
2. Adjudicate decision for abstracts #
There is an option to auto-adjudicate. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision on the abstracts. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option. Once adjudicated, the studies will either be excluded or advanced and sent forward to Full Text Screening.
Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can Blind Adjudication in Settings.
If you have Robot Screener turned on, its decisions will be shown in Abstract Adjudication and not Full Text Adjudication. This is because, in Dual Two Pass mode, Robot Screener only runs on abstracts. Learn more here.
4. Screen the full-text of each study. #
Two independent reviewers will then need to review the full-text of every study and screen the abstracts using the same approach as Standard Screening Mode. AutoLit automatically queues the full-texts to all users until two screening decisions are made; then, the articles are sent forward for adjudication.
5. Adjudicate decisions for full-texts #
There is an option to auto-adjudicate. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision on the full-texts. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option. Once adjudicated, the studies will either be excluded or included. Included studies move forward to Tagging.
Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can Blind Adjudication in Settings.
Kappa Statistics for Interrater Reliability #
After you finish Dual Screening, you can view the Kappa statistics in Activity.
Guidance on Robot Screener in Dual Two Pass Screening #
The recommended practice is to use Robot Screener as a reviewer in Abstract Screening when in Dual Two Pass mode, and use fully manual Full Text screening.
Abstract Screening:
- Abstract Screening requires 50 abstract adjudicated records, either advanced or excluded under Abstract Screening –> Adjudicate Screening, and 10 advanced records
- These actions must be performed before Robot Screener can be turned on for Abstract Screening
Full Text Screening:
- Because fewer records are available, more intensive assessment of the full text is needed, and final decisions on inclusion are made at the Full Text stage, AutoLit does not currently offer Robot Screener at the Full Text Screening stage.
- You can keep Robot Screener toggled ‘on’ throughout your screening process, but once records are Advanced from Abstract Screening, each record must be screened by two independent screeners before proceeding to adjudication.
Learn more about Robot Screener.
