Adaptive Smart Tags (ASTs) is most effective when the tag hierarchy is kept as simple and concrete as possible. See below for tips on how to optimise your questions for Adaptive Smart Tags:
Configuring the Tag Hierarchy #
Mode
- Form-based Tagging mode (default) works best
- ASTs work in Standard mode too, but does not have explicit Questions to feed to the LLM
Tags
Ensure you provide:
- Tag Name
- Question Type (where relevant),
- Question and Description text to give details beyond the Tag Name
The hierarchical structure of select question responses is fed to the LLM.
Tag Name
- Limit to 4-5 words max, spell out acronyms, be careful with special characters ($, :, -, % etc.) only use if the term is commonly reported
- Ensure terms are spelled correctly but don’t worry about variations (e.g. UK vs US English)
Question Type
- Use Single Apply where possible
- While only one recommendation is generated per single apply question, it is content-type aware. I.e. whether the content type is text, table, options or numerical, the LLM will identify and fill out these tag contents as appropriate.
- Use Select types if total answers/child tags are less than 8, too many tags can overwhelm the LLM
- Multiple recommendations are generated for select type questions; the LLM is not content-type aware for select type questions.
Questions
- Add as much detail as is applicable to your desired data.
- Present or past tense questions starting with “What” or “Where” tend to work best
Contents (Tag Text, Tables, Options, Numerical)
- For Single Apply questions, ASTs can extract all content types: text, table, options, and numerical so ensure tags with these content types are Single Apply if you wish to generate ASTs for them.
- Provide as much specificity as possible in table column headers and options to allow the LLM to fill out the correct cells as accurately as possible and select the most appropriate option, respectively.
Examples of Optimized Questions:
- Tag Name: “Chronic kidney disease”; Question Type: Single Apply, Question: “Is chronic kidney disease reported?”
- Tag Name: “Prevalence”; Question Type: Single Apply, Question: “Was prevalence reported in this study?”
- Tag Name: “Mortality at 90 days”; Question Type: Single Apply, Question: “Was mortality at 90 days reported?”
- Tag Name: “Study Phase”; Question Type: Single Select, Question: “Is this study part of a drug approval phase?” Child Tags: Phase I, Phase II, Phase III, Phase IV
- Tag Name: “Infections”; Question Type: Multiple Select, Question: “What types of infections were reported?” Child Tags: Viral infection, Bacterial infection, Fungal infection, Other infection
Note: only one answer will be generated for Single Apply questions, but multiple answers are generated for Single and Multiple Select type questions.
FAQs #
What data does the LLM use? Are there restrictions in place on the use of my data?
When using Adaptive Smart Tags, your tag hierarchy and all relevant text (abstract and full text) content are sent to OpenAI via their API. OpenAI is blinded to your identity, and processes this information for the express purpose of providing Adaptive Smart Tags, per standard Data Processing Addendum. Your data will not be used to inform or train OpenAI products.
I don’t see any recommendations. What am I doing wrong?
First, make sure recommendations were successfully generated in Background Jobs. Then make sure you are on either the Abstract or Full Text page, you have the right hand Questions and Tag Recs tabs open and a Question is selected. Try going through a few questions in the form, it may be that fewer recommendations were generated than expected. Still nothing? You may need to simplify your hierarchy and regenerate recs. If that doesn’t work, contact support@nested-knowledge.com.
The recommendation is highlighting the same excerpt for multiple questions or isn’t applicable to my question, can I retrain the AI?
The LLM works to find the most succinct answer to the question, which means it often highlights the abstract/similar excerpts. Currently there is no way to retrain the AI but we are working on having multiple recommendations generated for you (including for Single Apply question types) to choose from… coming soon! In the meantime, we recommend fine-tuning your questions and refreshing ASTs.
I added new tags to the hierarchy and new Adaptive Smart Tags were not generated, is it automatic?
No, after configuring new questions, Adaptive Smart Tags must be regenerated again with the Refresh box checked. Note: each nest is limited to 5000 generations for study abstracts and 1000 for study full texts. If you need to increase your limit, contact support@nested-knowledge.com.
