Objective #
The objective of Critical Appraisal is to evaluate the quality of evidence of included cohort studies or controlled trials and we offer commonly used systems of assessment within your nest. You can complete Critical Appraisal in Single or Dual mode.
Configure Critical Appraisal #
Head to Nest Settings and select the Critical Appraisal system of choice. After selection it will be added to the left hand menu.
We currently offer the following Critical Appraisal systems
- System: SIGN, Version: 2011
- System: Cochrane RoB, Version: 2
- System: JBI, Version: 2020
- System: Newcastle-Ottawa Scale, Version: 1
- System: ROBINS-I, Version: 19 September 2016
- System: SIGN, Version: 2019
- System: QUADAS, Version: 2
If you don’t see a Critical Appraisal system you wish to use, we recommend exploring our public tag hierarchy templates to see if we offer it there or feel free to replicate it yourself. While there won’t be any Critical Appraisal-associated outputs, you will still be able to export extracted appraisal data in a spreadsheet format.
Choose Scope #
You may choose a scope. Note: this is only applicable if you have configured data elements for MA Extraction, otherwise leave the scope as “Entire Study”.
If you have configured outcomes and wish to only assess specific ones, they will be displayed for selection:
Navigate to Critical Appraisal #
Navigate to Critical Appraisal Module #
After selecting the system and scope, reviewers can begin performing critical appraisal. Note: only included studies will be shown in this module.
Read study and select study type #
Depending on the selected Critical Appraisal system, each assessment will be different but you will answer the same set of questions for every study.
Assess study bias #
Fill out the questions as you read through the uploaded study.
You can monitor your progress, skip studies (and return to them later), and leave comments!
Smart Critical Appraisal #
The first step to using smart critical appraisal is to navigate to the Nest Settings and choose a Critical Appraisal system that fits your needs. As of now, the only systems that are compatible with SCA are: only for JBI and SIGN (2011/2019)
Then from there, navigate to the gear icon next to the Critical or Dual Critical Appraisal module.
That will bring you to the page where you can generate answers for your current system using Smart Critical Appraisal:
From there go ahead and generate by clicking the button. Each Nest comes with 1000 full text generations.
As it runs there are a few things to keep in mind. It will only run on included records with a full text upload, meaning that no abstracts are reviewed. In addition, the Smart system annotates all of its work while not overwriting or answering any questions that have already been done by humans. You can skip “Overall” generation when you want to make the call yourself, on your own standards, otherwise we will decide based on “typical practice” If you want it to work on the final and the reviewer level, run it in dual mode.
For clearing the Smart CA system’s answers, know that it will never clear decisions made by a human. As always, the features of Smart Critical Appraisal can be done manually by a human reviewer.
Critical Appraisal Visuals #
By completing Critical Appraisal, you automatically generate Critical Appraisal visuals: Domain Distribution and Stoplight diagrams on Synthesis. Additionally, you will be able to export all data as a spreadsheet from Inspector.
