Criteria-Based Screening (CBS) offers an alternative, comprehensive screening workflow in Nested Knowledge that structures your screening process around question-based form instead of a list of exclusion reasons. By creating simple Yes/No questions, it allows reviewers or an AI to answer these questions in the screening process before finalizing a screening decision to encourage a more effective screening process.
CBS can be used for fully manual screening or paired with Smart Screener, an AI tool that assesses and answers your screening questions with annotations. This tool can speed up manual screening, or completely bypass human screening and apply screening decisions based on a threshold of correct answers.
This brings the first model to our software that enables immediate screening by AI without any human screening! See below for a video demo:
Overview #
By default, Screening in Nested Knowledge is Exclusion Reason–based. Criteria-Based Screening offers a more structured approach:
- Enable Criteria mode in Nest Settings.
- Define your inclusion criteria as Yes/No questions.
- Answer these questions for each record (abstract or full-text).
- Include/exclude based on which or how many criteria are met.
- Optionally let AI pre-fill answers via Smart Screener.
This method is ideal for protocol-driven reviews where consistency and traceability are important.
Steps for Configuring Criteria-Based Screening #
1. Turn on CBS in Nest Settings #
Since exclusion reason creation is the default configuration, navigate to Nest Settings and toggle on “Criteria”.
If you’d prefer to remain in the list of exclusion reasons paradigm, leave Exclusion reasons toggled and see the main Configure Screening page for guidance.
2. Navigate to Configure Screening #
On the Configure Screening page (red), the layout prompts you to name and create questions to be answered during the Screening stage.
3. Create Criteria/Questions #
Based on your protocol, create yes/no questions as part of the Abstract criteria (for Abstract Screening) and Full Text criteria (for Full Text Screening). Questions must be created where answering yes is an indicator that the study should be included, and no is an indicator that the study should be excluded.
For example, for a review of studies only in China, you might create a question named: “Study Location”, and the question would be “Does this study take place in China?”. Since the review will only include studies in China, answering yes = inclusion, and no = exclusion.
For more complex questions, it is better to provide this detail in the question for clarity. For example, for Study Type, the question could be: “Is this an epidemiological, population-based, or observational study (cross-sectional, cohort, case-control, surveillance reports, registry study)?”. This provides more detail for what is encompassed when answering yes = inclusion, anything else would be a no = exclusion.
Ensure you provide a name, question, select Abstract or Full Text and hit Save. New questions will appear in the corresponding criteria as they’re saved. Note: as you’ll see next in the Screening stage, after the screening criteria has been assessed for each study and exclusion is determined, an exclusion reason will need to be applied. The list of reasons displays the names of criteria created here, so keep this in mind when naming criterion.
At this stage, questions for each must be created separately and questions cannot be exactly the same for ease of tracking. So you may wish to get creative with question names e.g. Study Population (Abstract) vs Study Population (FT).
4. Editing, Ordering and Deleting Criteria #
To edit a question, select the pencil icon (red). To reorder criteria, select the 6 dots to drag and drop (blue); the order displayed here is the order shown in the form when screening. To delete a criteria, select the trash icon (pink); deleting criteria will remove any answers applied to studies as well as exclusion reasons if applied.
PRISMA Settings #
By selecting PRISMA Settings (red below) you can:
Assign an existing exclusion reason to represent “No Full Text Available” and will be reflected as such in the auto-generated PRISMA diagram.
Keywords #
To learn how to add User Keywords for Abstract Highlighting, see here. This is available in either Configure Screening mode.
Screening in Criteria-Based Mode #
After setting up your criteria, you can now screen manually or with the assistance of Smart Screener.
Screening Manually: Overview #
When screening manually in CBS mode:
- Each record displays your criteria questions in the right panel.
- Click Yes or No for each.
- The system will display the current criteria met count and your pre-set inclusion rule (e.g., must meet all criteria).
- Click Include or Exclude to finalize.
This structured approach makes it easier to justify decisions and maintain reviewer alignment.
Steps for Screening Manually #
1. Navigate to Screening #
Head to the main screening page (red) and ensure the Screening Criteria is open on the right-hand side (blue). Your configured criteria/questions will be displayed here for Abstract and Full Text. This form is designed for a reviewer to go through one-by-one and answer each question alongside evidence from the study.
2. Answer Questions #
Go through and answer the yes/no questions and the criteria met count will keep track of the number of questions answered yes, no and unanswered, respectively (red). Below I have 1 answered yes, 0 answered no and 1 unanswered question.
3. Assign a Screening Decision #
Based on the criteria met count, and answers to the questions, either Include or Exclude for a specific reason. This will move you onto the next study.
Screening with Smart Screener: Overview #
Smart Screener brings AI extraction into Criteria-Based Screening:
- After configuring your Screening Criteria, generate Smart Screener.
- The AI will auto-suggest Yes/No answers for each criterion without prior training.
- Suggestions appear highlighted in the criteria panel—click to confirm or adjust and follow the manual screening process highlighted above.
- Use the AI Confidence score (displayed per criterion) to guide your screening decisions.
Note: Smart Screener works independently to our inclusion probabilities in our screening model. Smart Screener is based on the answers to your criteria/questions, and the screening model is trained on your applied screening decisions. They can be used in combination.
Steps for Using Smart Screener #
Generate Smart Screener #
On the Configure Screening page, after you have created criteria, select “Smart Screener” in the top left. Here, you have a few settings to configure before generating:
- Abstract & Full Text: Select whether you want the AI to answer the questions for study Abstracts or Full Texts — to do both, you’ll have to run two separate generations. Ensure you have criteria created in the corresponding section before generating.
- Refresh: Check Refresh if you change the Screening Criteria and wish to re-generate smart screener for all records.
- Unscreened Records Only: Check to have Smart Screener run only on studies with no screening decision applied/that are unscreened.
- Annotations Required: Checked by default to only answer questions when an annotation is attached. You may wish to uncheck if annotations are less important for your review.
Note: the study limit for generating Smart Screener for Abstracts is 5000 studies and for Full Texts is 1000 studies. Then hit “Generate”.
Manual Screening with Smart Screener Assistance #
Once Smart Screener has finished, navigate back to the Screening page. You’ll see each question in the form has been answered (but still editable, of course). Typically an annotation is provided if the question was answered yes, with no annotation is the answer is no. This form of answered questions allowing a reviewer to scroll through, view the total count at the end and assign a screening decision to advance/include or exclude.
Bulk Decisions with Thresholding #
If you’d prefer to avoid manual screening and apply the AI suggestions without review, you can do so via Thresholding.
- After generating Smart Screener, navigate back to Configure Screening and select “Thresholding” in the top left (red below)
- Select the type of criteria you’d like to apply this threshold to, Abstract, Full Text or Both
- Select a threshold number: the number refers to the number of questions answered “Yes”
- Select the action to apply threshold of the options provided
For example, if you select Type of Criteria: Abstract, Threshold: 2, Include at or above threshold, Exclude below, then all study abstracts that have been assessed by Smart Screener that have answered “Yes” to 2 or more questions in the criteria will be included. All other study abstracts will be excluded. This is customisable as the criteria differ in number and importance from review to review.
- You may also wish to preview the set of records that this action will apply to, studies that will be Included and those that will be Excluded.
- Once you are happy with your threshold set, select “Apply Threshold”.
As with manual screening, applying this threshold will result in advanced/included records moving to the next stage of screening or data extraction (depending on your Screening mode) and excluded records will remain excluded. You can filter to applied screening decisions in Study Inspector.
Next Steps #
To view the studies that have been affected by applying the threshold, navigate to Study Inspector (red)–> Bulk Actions (blue)–> History (orange)–> View Records (pink):
Filtering to Screening Criteria and/or Criteria Count #
After conducting Criteria-Based Screening, you may wish to filter to and view studies with a certain Screening Criteria that have been answered Yes, No or unanswered or a Criteria Count. To do this navigate to Study Inspector, select the “Add Filter” dropdown, under Workflow there are filters for Screening Criteria and Criteria Count to utilise to view subsets of studies.
See more on how to use Study Inspector filters.
Then onto Tagging! #
After completing Screening, the next stage is Tagging your included studies. See here for more.
