Tags reflect the qualitative content of underlying studies and provide method for attaching text or images from these studies. After tags have been configured, and you have included studies, you can begin applying tags in the Tagging module. You can also apply tags during the Screening stage! Once a tag is applied, it is immediately viewable on Qualitative Synthesis and ready to be exported.
If you are in Standard Tagging mode, see our instructions on how to apply standard tags.
Steps for Tagging in Form-based Mode: #
1. Navigate to Tagging #
Navigate to Tagging in the left-hand menu.
This will enable you to apply tags to records sequentially. If you would prefer to search and find records to tag, or to view records that have already been tagged, use Study Inspector.
2. View the Full Text #
At this point, your included studies should have full texts uploaded and selecting the Tagging module will show you the first included study’s full text to begin extracting data. If no full text has yet been imported, learn how to upload it both individually and in bulk here.
Note: while extracting from Full Texts is best practice, you may also apply tags with excerpts from the Abstract and Supplements tabs.
3. Answering Questions #
Form-based tagging is designed to show the questions you configured side-by-side with the Full Text or Abstract or Supplements for ease of data extraction. Questions will be opened automatically and available for answer in the right panel (red); the Question under review has a light blue background, to apply the tag select “Apply” and you will be auto-scrolled to the next unanswered question. If your tags have text contents, you can annotate the Full Text or Abstract or enter text of your own. See below for more on in-text annotations!
All Questions should either be answered or marked “Not Relevant”.
Grouped Questions #
By default, questions are grouped by root tag. You can navigate between question groups using the dropdown (red) or left and right arrows (blue). If you’d prefer your questions to be shown as a single form, you can change this in Nest Settings.
The root tags/groups also display a key to indicate completion of the corresponding group of questions. No circle indicates questions are incomplete (Interventions/Comparators above), a hollow circle indicates partial completion (Patient Characteristics), and a full circle indicates full completion (Study Design).
Question Type-specific Answers #
For each Question in the list, complete the following actions based on the type of Question:
- Single Select: Apply one child tag that answers the pre-configured questions. To do so, select one of the tags from the drop-down, and then highlight or select an Excerpt.
- Multi-Select: Any of the child tags can be an answer, so you can apply as many tags from the drop-down as are applicable to the study. When all relevant child tags are added, select “Next” to mark the Question complete.
- Single Apply: The tag under review is either applied to the study (select “Apply”) or marked irrelevant. No child tags are added!
Whenever a Question has no relevant answers, select “Not Relevant” to move to the next Question
4. Add an Annotation #
When answering questions where the tag has text contents, you may wish to add an annotation. You have three options for how to identify the text excerpt that will be associated with that tag:
- Highlighting (Text Annotation): A traceable, exact quote from the text of the article.
- Selection (Area Annotation): A traceable, exact image extraction from a table, figure, or other area of the article.
- Manual entry (No Annotation): A non-traceable excerpt (that is, an excerpt that is not connected to a specific part of the article) that you type into the Tag Text box.
4a. Use the Highlighting Tool: #
The default Tag Text method is Highlighting. You can also manually select the Highlighting icon, if you need to toggle back to this option (red).
Click and drag over the text you would like to Highlight (blue). Highlighting will extract an exact text excerpt that is shown in light blue, and the text will be automatically populated to the Tag Text box (orange).
4b. Use the Select Tool: #
To switch from the default Highlighting tool to the Select tool (red). Create a box across the area you’d like to select for the tag (blue). This selection will be automatically saved in the tag text box (orange).
Selection / Area Annotation is best used on tables, figures, and images that are not amenable to exact text quotation.
4c. Manually type out in Tag text box: #
If you prefer to manually type the information from the text, you can do this by clicking your cursor in the tag text box and type what you’d like.
Manual text entry should be used whenever you want to associate customized text rather than quotation from the underlying article. Warning: manual entry will not maintain an exact location in the full text, so it may be difficult to find the exact contents of the article that support manually entered text excerpts.
To select text manually (without highlight) select the right-most cursor icon (red).
Clear Annotations #
If you need to redo your tag text annotation, you can either simply redo the action (Highlighting, Selecting, or Manually typing), or select “Clear Annotation” from the top of the Full Text (left-most x icon).
This will remove all tag text; next, choose the tag text type you would like to use, and redo the relevant Highlight, Selection, or Manual text entry.
Q: Why not leave the annotation / tag text blank?
A: It is possible to apply tags without filling in the tag text. However, doing so will mean that the only evidence that the tag is applicable to that specific study will be the fact that it was applied, and those who view your Qualitative Synthesis will have no context. If you fill in text content, you provide specific evidence of that tag’s applicability as well as presenting the specific information from that study to viewers of Qualitative Synthesis.
5. Click “Apply Tag” #
Once you have the content of interest into the tag text box, make sure that you have selected the relevant tag from the drop-down menu (red box). Once you have confirmed that both the Tag and the Tag Text Content are correct, click “Apply Tag.”
6. Select Complete #
When you have finished extracting data in the study, select “Complete” in the navigation panel to move onto the next study. This also removes it from the queue. If you wish to return to this study to make edits you can do so in Study Inspector.
Tag Recommendations #
Tag Recommendations is a tool we offer to speed up the process of data extraction in your nest. The tool searches the study full text and highlights specific evidence within the text to help answer your questions one by one. We offer two types of tag recommendations:
- Standard Tag Recommendations are available to all users and perform an automatic key word search of the tag name (and any applicable child tags) in your full texts. These require no user action.
- Smart Tag Recommendations utilize OpenAI’s GPT 3.5/4 to perform a smart search of the tag data. Smart Tag Recs must be switched on in Settings after the requirements are met and generates recommendations for Abstracts as well as Full Texts but not Supplements. Learn more about generating Smart Tag Recs before switching them on as there are limited regenerations.
Tag recommendations are displayed in the tab below Questions on the right-hand side (red). In Full Text view, it’s titled “Full Text Tag Recs” (always displayed); in Abstract view, it’s titled “Abstract Tag Recs” (off by default, must be toggled on in Nest Settings).
If a recommendation is available for the selected question, it will be displayed (blue). When clicked on, it will auto-scroll to the excerpt within the full text (or highlight it in Abstract view) (orange) and auto-populate in the tag text box to be applied or removed (pink). This tool can be utilized to assist in your systematic evidence collection, and guide targeted reviews as any evidence the AI finds can be applied as a bulk action.
Learn more about how Smart Tag Recommendations work and view our Technical Disclosure on Nested Knowledge AI systems.
Note: Smart Tag Recommendations is an enterprise-level feature only. If you wish to conduct a free 2-month pilot trial at the enterprise level in order to make use of this feature in your workflows, please contact us and we’ll set that up.
Standard Tags #
In addition the format of answering questions in a form, you may find it useful to quickly add one off tags from a dropdown. In the “Tagging” tab on the right-hand side, you can view all applied tags in the study and select those to apply from a dropdown. This is the basis of the Standard Tagging mode, but is also available here in Form-based Tagging mode.
Add New Tags on the Fly #
If you encounter a single or multiple select question where the answer is not one of your pre-configured tags, you can either add it to your hierarchy on the Configure Tagging page, or add it ‘on the fly’ without leaving the page.
To add a tag on the fly, type the title of your new tag answer into the “Select Tag” box, and click “Add Option” that appears at the top of the drop-down list of tags.
In the modal that appears, confirm the tag name, add a description (optional), and the parent tag will already be assigned. Once created, you will now be able to find the new Tag on the drop-down list.
Note: Only tags with text contents can be created on the fly. To toggle on table contents, edit the tag in Configure Tagging.
Tags with Non-Text Contents #
Table Contents #
Tags configured with Table Contents will be displayed exactly as configured:
Highlighting pdfs does not automatically input the text into the box unlike tags with text contents only. However, it will remember any text highlighted or selected in the pdf and auto-scroll to it when the tag is selected again.
Note: If you are entering numerical data into tables, no automated statistics are generated. This is only done in the Meta-Analytical Extraction module.
To alter the columns in the table for this tag, either click on the column header in the Tagging module itself, or head back to Configure Tagging. Learn more about tag tables here.
Options #
Tags configured with Options will display the options only, with no accompanying text box:
Numeric #
Tags configured with Numeric Contents will only allow for numerical input, displaying an error if text content is inputted:
Extracting Data from Images #
While Nested Knowledge doesn’t currently support extraction from images in publications in the software, if you wish to learn, feel free to view our guidance on this.
Related Report Tags #
When you apply a tag to a record that has an associated related report (RR), the tag is also displayed on the related report but with a RR icon to differentiate the tag origin. Learn more about related report tags.
Next Steps after Tagging #
After extracting data in Tagging, next steps include:
- Quality Checks of the data using Study Inspector
- Viewing and sharing auto-extracted Qualitative Synthesis
- Building a Dashboard with cards such as updateable custom tables that can be downloaded
- Export all data from Tagging as a spreadsheet
- Conduct Critical Appraisal
- Optional: Perform Meta-Analytical Extraction to generate Quantitative Synthesis
Download Annotated Full Texts/PDFs #
After completing Tagging, you may wish to download an annotated version of the full text. To do so, you can download individual annotated pdfs or download them in bulk.
