At Nested Knowledge, we are committed to transparency and rigorous evaluation of our AI-powered review tools. This page compiles validation studies across the core stages of the evidence synthesis workflow: Search, Screening, Data Extraction.
For each stage, we provide internal assessments and external, third-party evaluations, including academic reviews, NICE compliance alignment, and independent comparative analyses. These studies demonstrate both the feature evolution of our platform and the real-world performance of our tools in clinical and regulatory research contexts.
Notably, recent external benchmarking placed Nested Knowledge as outperforming three other automation platforms. See the linked abstract and poster in Tool Reviews and Feature Comparisons for full details.
1. Overview and Critical Links #
- AI in Nested Knowledge – how AI powers our platform
- NICE Compliance Guide – how Nested Knowledge alignswith UK standards for clinical evidence
- Use Cases and Case Studies – real-world applications across all domains in the evidence synthesis life-cycle
2. Validation Studies by Workflow Stage #
Literature Search #
- Smart Search is a human-in-the-loop reasoning agent in Nested Knowledge that uses LLM-driven chain-of-thought logic to build Boolean search strings after input of a research question, achieving over 75% recall in validation against Cochrane and in-system SLRs. The tool significantly outperforming black-box LLM approaches. See published ISPOR USA 2025 Abstract and accompanying presentation.
Screening #
- Robot Screener is a machine learning-based AI tool in Nested Knowledge that replaces the second reviewer in a Dual Screening workflow. The tool prioritises high recall to ensure no relevant studies are missed during abstract screening, achieving up to 97% recall in internal studies and comparable performance to human reviewers in external HEOR-focused validations. This makes Robot Screener a powerful, time-saving asset for high-quality, comprehensive SLRs and HTAs.
- See published ISPOR USA 2024 Abstract (internal) and accompanying poster presentation.
- See published ISPOR USA 2024 Abstract (external) and accompanying poster presentation.
- View summary of validation statistics and explanation of how the statistics work.
Tagging (Data Extraction) #
- Core Smart Tags (CSTs) are a specialised AI tool in Nested Knowledge that combine machine learning, NLP, and heuristics to extract and hierarchically structure key clinical data from a research question. This includes extracting PICOs, study type, location, and size with validated accuracy and human-in-the-loop oversight, enabling faster, reliable evidence extraction for clinical SLRs.
- See published ISPOR USA 2025 Abstract and accompanying poster.
- Adaptive Smart Tags (ASTs) leverages AI to automatically highlight and extract user-defined variables across abstracts and full texts, achieving up to 80% match to manual extraction in validation studies enabling faster, intuitive and audit-tracked data structuring for complex clinical reviews.
- Version 1: See published GES Prague 2024 Abstract and accompanying poster.
- Version 2: See unpublished, internally conducted online statistics.
Meta-Analytical Extraction #
- Smart Meta-Analytical Extraction (SMAE) is an AI tool that generates a rapid meta-analytical outputs, such as forest plots, from chosen studies and their accompanying full texts.
- Released May 2025, it is currently in beta. Stay tuned for accompanying validation studies!
3. Tool Reviews and Feature Comparisons #
- Feature Comparisons Table
- Function-by-function match-up across systems conducted by Nested Knowledge prior to feature-completeness
- Updated in 2023
- External Reviews of entire tool
- For academic research
- For clinical reviews in regulatory research: Abstract and Full Text
- Living evidence map: position of NK in automation ecosystem
- Nested Knowledge vs three other systems
- Nested Knowledge was concluded as the most comprehensive, effective tool referred to by “T1”. See published ISPOR USA 2025 Abstract and accompanying poster presentation.
Do you have further questions about how our AI tools work? Let us know!
