June 17, 2022
Recently, we sat down with Stanford physicians Dr. Jeremy Heit and his protégé Dr. Guatam Adusumilli to learn how they published four systematic literature reviews in under a year using the Nested Knowledge software platform.
Dr. Jeremy Heit, MD, PhD is an Assistant Professor of Radiology and of Neurosurgery and the interim Chief of Neuroradiology at Stanford University. Dr. Heit is a neurointerventional surgeon (neurointerventional radiologist) who specializes in treating stroke, subdural hematoma, and brain aneurysms. Dr. Heit research interests include the use of mechanical thrombectomy in stroke treatment.
Dr. Gautam Adusumilli is a resident in Diagnostic Radiology at Massachusetts General Hospital and a continuing Research Associate in the Heit Lab at Stanford, where he completed his preliminary year in Surgery. Together with Dr. Heit, he conducts research into new, minimally-invasive, image-guided treatments for acute ischemic stroke.
The Nested Knowledge (NK) research collaboration with Stanford began in May 2021. A full year ago, and what a prolific year it has been. Working in collaboration with the research team at Nested Knowledge, these physicians came armed with excellent research questions in their field of study. Initially, these two physicians began by examining whether mechanical thrombectomy had an additional benefit to the patient when including tissue plasminogen activator or iv-tPA.
And the research questions continued to pour in, and in short order they had published systematic reviews investigating mechanical thrombectomy in basilar strokes, common data elements in middle meningeal artery embolization in chronic subdural hematoma, and differences in outcomes between mechanical thrombectomy in anterior and posterior circulation strokes.
Dr. Adusumilli – Jeremy told me about a new platform for conducting systematic reviews and meta-analyses. I’ve never published meta-analysis or systematic reviews, because I had this impression that it’s not as exciting as other research projects. My belief was that systematic reviews are an experience of being ‘in the weeds’, needing to sift through study by study. The inclusion criteria and exclusion criteria is a bit shifty and you have to adjust the criteria depending on what you find. In the past, I’ve found that you end up being more inclusive which leads to lower quality studies. A lot of studies have similar research questions that are lower quality because they’ve included retrospective cohorts or single-armed studies because they’re trying to be more inclusive.
Dr. Adusumilli – I came into my first training session and the first thing that stuck out to me was how user-friendly the software is. Within a 10-15 minute span of training, I was able to pick up the platform and felt comfortable enough to get my feet wet and jump in.
I also felt that the NK software acts as an umbrella for everything I need to conduct a systematic review or meta-analysis. If the user has any questions regarding the inclusion criteria, exclusion criteria, research questions, data elements, you can find it all in one space. You have all your data in one place; all your included studies, all the abstracts, and then an initial qualitative review of these studies. Which I think is the most unique feature of the software. The ease of organization, when everything is in one place. I think that organization is key to creating a high quality publication.
In the past, people have told me that conducting a meta-analysis can be disorganized, but that is not the case with Nested Knowledge. There have been many research questions that we have pursued that needed new literature searches, and it is very easy to go back into the software to re-screen studies and then extract the new data.
The NK software does most of the work for the researcher.
Try it: View an interactive figure from the first publication to come from this collaboration examining tissue plasminogen activator or iv-tPA. Filter by patient demographics, interventions, and outcome variables. Sunburst diagram generated by Nested Knowledge depicting the variables and treatment arms of this study, with organization hierarchy from inside to outside. For example, “Clinical Outcomes” and “Angiographic Outcomes” are two sub-categories of “Outcomes”.
Dr. Adusumilli – Based on my experience seeing the literature and talking to people in the field, I concluded that after the 2015 landmark trial, there was a consistent question from physicians. Is tissue plasminogen activator or tPA use in stroke patients futile or is it useful in a specific population of patients? There are only a few publications addressing this research question, and we felt like it was important to see what a systematic literature review would say about it.
Dr. Heit – in the era of mechanical thrombectomy, which is a procedure used to remove occlusions, the use of tPA in these patients is a very hotly debated topic. The field of neurointervention is a really fun field, and the fun part is that it is small. It has a lot of extremely smart, thoughtful people who are driven to advance the field quickly and accurately. This leads to many conversations about the literature…but, the sort of low-hanging fruit for thrombectomy has been picked, and if you talk about what is the role of an adjunctive therapy, you’re going to need larger numbers to really prove that something is or is not beneficial. When you’ve done the easier studies, answered questions that can be answered with a small number of patients, to see a true difference between two different treatment arms, suddenly you need a lot more patients to really make sure that the statistics are accurate. So if you’re looking at thrombectomy with tPA versus just thrombectomy alone, I would have expected a priori that you wouldn’t see a difference without further study, and in fact, that’s kind of what each individual studies showed. Then, that makes it a very good question to ask with a meta-analysis, following studies and saying, did you see a difference? …and that pattern suddenly makes a lot of sense. Then, the speed and efficiency of Nested Knowledge facilities this kind of research collaboration between many different physicians.
Dr. Adusumilli – there is a low activation energy to use the platform. As Jeremy mentioned, when you have a group of people coming up with a protocol and common data elements, for example, the software is very easy to use. In one of our projects, I was using the qualitative synthesis feature and I thought, “this is actually extremely easy to use and it wouldn’t be that hard to create a table from these outputs. So, let’s just do it, what are we waiting for?”
Dr. Heit – Yeah. I think that’s a good lesson in general, for Nested Knowledge and this sort of scientific approach too, is tackling these areas where there is tremendous interest and there are some high quality studies getting done, but being able to find out, “Hey, here are the deficiencies. Here are the studies that look to be high quality, but here are some design flaws that are really maybe limiting your conclusion.” And then when you’re able to demonstrate that and publish that, it becomes a call to the field to address those deficiencies. And to really say, “Hey, if you’re gonna go forward and do this, you need to design your study properly so that you don’t fall into this trap, and so that we actually learn the knowledge that lets us answer this question at hand.”
And I think that bootstraps into the study on common data elements for middle meningeal artery embolization, right?
Dr. Adusumilli – right!
Dr. Heit – Now you’re talking about studies that aren’t even necessarily published yet, but you’re kind of examining them and you say, “Well, okay when you guys all do this research, are we gonna be able to pool each of these studies together and have a really robust meta-analysis?” And then what we find is well, actually that could be a huge problem, because everyone is not quite looking at the same criteria which makes things hard to pool. There is a tendency for things to get siloed into individual efforts, and individual efforts could be still multi-center or large groups, but that may not be enough. We’re getting into the era of a big data increasingly, right? So if big data is going to work, we’ve got to be able to make sure that each study is able to contribute to the conglomerate data. An efficient way to assess what is actually being collected and what are the commonalities. Nested Knowledge is really helping you attack multiple different fronts in research and also informing the publication of it, which I think is pretty remarkable.
Dr. Adusumilli – Yeah, I totally agree. I think the common data elements for subdural hematomas was actually quite a jarring study result because you had 18 studies that we were able to compile, which again, had we not used the Nested Knowledge, would have taken significantly longer. Really, they were all just trials, with only a few published to date. So we had 18 studies that were so heterogeneous, not just in terms of treatment arms; they were also putting together surgery alone, versus MMA after surgery, versus without medical therapy surgery. It was such a heterogeneous group of studies that there was no way you can report on these in a homogenous manner so as to actually inform clinicians on how to treat this. And then you also had like eight or nine different outcome elements, and most of the studies only had one or two common ones. And you’re only reporting on one or two common data elements over 18 studies that are all actively happening and there’s hundreds of thousands of dollars being poured into this stuff. That’s the scary thing.
Dr. Heit – Super expensive. You want to just be thoughtful and think about it ahead of time. We should be collecting data that is amenable to that kind of research and that kind of prioritizing ahead of time. And I could see how you could use NK as a central source to inform you when you’re embarking on thinking about doing these studies and just kind of asking ahead of time, “Well, what do we need to do? How do we come up with common data elements? What’s the status of what’s out there?” And you can pretty darn quickly get that knowledge.
Dr. Heit – The other thing too that I think that NK has the potential to really be a defining platform for doing a living review, kind of like we’re talking about all these studies, because this is a very active space, right? Each research question is very active and new data that is constantly coming out. So for example, we were talking about the Basilar thrombectomy trial…at the ESSC meeting today, that trial is being presented. It another randomized trial. And so as of later today, the paper won’t be out yet, but there is new knowledge about something we wrote about within the last 12 months, and so do we write another paper? You absolutely could, but it would also be great to have some journal owning this and saying, “We’re going to update as we go.” You can then provide updated data in real-time that is synthesizing this high-quality evidence as it comes out, and you may suddenly say something like, “Well, we didn’t have evidence for it. Maybe this new trial tips the scale, and suddenly now we do have evidence for it.” I think that’s a really potentially unique use of this, and the number of papers just goes up and up; there’s so much knowledge being produced every year.
Yes, it is of varying degrees of quality, but regardless, there are some real gems out there, and I personally find it really hard to keep assimilating all this knowledge. There is so much to read, there is so much to understand. We are going to have to have new tools to help us consolidate and assimilate all this knowledge.
Creating your first systematic review and/or meta-analysis is easy and free with Nested Knowledge. Sign up right now to get started:
