In systematic review, critical appraisal is needed for validation, but it’s also one of the most time-consuming steps in the entire workflow. Reviewing every included study for bias, methodological rigor, and quality signals can take days, even weeks.
Nested Knowledge’s Smart Critical Appraisal (SCA) changes that equation. By combining AI-generated appraisal answers with a granular audit workflow, SCA lets your team move faster without compromising the standards that make your evidence synthesis credible and publishable.
Critical appraisal is the systematic evaluation of the quality and reliability of clinical studies included in your review. It answers a fundamental question: How much should we trust this evidence? By identifying sources of bias, assessing study design, and scoring methodological rigor, critical appraisal directly shapes the strength of your conclusions.
Within NK’s AutoLit platform, Critical Appraisal is fully integrated into your review workflow. You can run it in Single or Dual mode––with Dual mode enabling two independent reviewers to assess the same study, the gold standard for reducing reviewer subjectivity.
The platform supports seven industry-leading appraisal systems:
*currently compatible with Smart Critical Appraisal
The AI annotates all of its work, never overwrites human decisions, and can be cleared selectively which gives your team complete transparency and authority over the final appraisal record.
Here’s what makes Smart CA genuinely different from a generic AI assistant: it’s built directly into your structured appraisal workflow, understands your chosen bias framework (JBI or SIGN), and operates only on included studies that have a full-text upload.
Getting Smart Critical Appraisal running in your Nest takes just a few minutes. Here’s the path from setup to AI-generated appraisals:
1: Configure Your Appraisal System
Head to Nest Settings and select a Critical Appraisal system. Choose JBI (2020) or SIGN (2011/2019) to unlock Smart CA compatibility. Once selected, the module appears in your left-hand navigation.
2: Set Your Scope
If you’ve configured Meta-Analytical Extraction data elements, you can narrow the appraisal to specific outcomes. Otherwise, leave scope as “Entire Study” and move forward.
3: Navigate to the Smart CA Gear Icon
From the Critical Appraisal module, click the gear icon next to Critical or Dual Critical Appraisal. This opens the Smart CA generation panel.
4: Generate & Review
Click Generate. The AI will process all included studies with full texts uploaded, producing annotated appraisal answers. Human-completed questions are untouched. Review, adjust, and finalize at your discretion.
5: Run in Dual Mode for Maximum Rigor
For the highest-quality evidence synthesis, run Smart CA in Dual Critical Appraisal mode. This applies the AI at both the reviewer and final decision level, creating a complete auditable record.
Completing Critical Appraisal, whether manually, with AI assistance, or both, automatically generates two powerful visualization outputs on your Synthesis page:
Domain Distribution Diagrams show how bias is distributed across appraisal domains in your study set, letting you quickly see where the evidence is strongest and where uncertainty lies.
Stoplight Diagrams provide an at-a-glance quality signal for each included study — the kind of visual that reviewers, regulators, and journal editors expect in high-quality systematic reviews.
And when you need raw data, every appraisal answer is exportable as a spreadsheet directly from Study Inspector — ready for supplementary materials, regulatory submissions, or internal reporting.
Appraisal questions answered, visuals generated, and data exported, everything your systematic review or CER submission needs all in one workflow.
For Medical Affairs, HEOR, and Regulatory teams operating under tight timelines, Smart Critical Appraisal offers something rare: a way to accelerate a methodologically sensitive process without cutting corners.
Whether you’re running a Clinical Evaluation Report for a medical device submission, preparing a Health Technology Assessment dossier, or publishing a systematic review, the rigor of your critical appraisal process is visible to every reviewer. Smart CA helps you demonstrate that rigor while dramatically compressing the time it takes to achieve it.
Yep, you read that right. We started making software for conducting systematic reviews because we like doing systematic reviews. And we bet you do too.
If you do, check out this featured post and come back often! We post all the time about best practices, new software features, and upcoming collaborations (that you can join!).
Better yet, subscribe to our blog, and get each new post straight to your inbox.
One of the features of Core Smart Tags is Smart Study Type – this refers to our AI system that automatically categorises the study type
