Kalya Research: Complementary and Alternative Medicine (CAM) Virtual Research Assistant from Biomedical Literature

Complementary and alternative medicines (CAM) become an emerging subject of interest both for users and health professionals. Rigorous studies identify efficient and safe methods for human health, frequently called by researchers, non-pharmacological interventions. The challenge is to determine relevant articles in a large and increasing volume of publications and journals. To meet this challenge, we created Kalya Research (KR), a medical assistant tool based on artificial intelligence that selects and characterizes CAM literature and bring support to medical researchers. Based on rule models and ontologies, KR can suggest relevant and recent CAM publications. It presents key indicators through analytical visualizations. KR was evaluated at several points (effectiveness, relevance, usability) in 2 ways, by means of a bibliographic search comparison with MedLine and by questioning more than 40 biomedical researchers who used KR for their research. When compared with Medline, KR highlighted most of the relevant CAM publications. The evaluation by the researchers showed that the majority of them found the tool to be relevant and time saver and feature-rich. Our future objectives are therefore to constantly develop the application to improve our models for detecting CAM publications and named entities (diseases, CAMs, outcomes), and to extend it to new health topics.


Introduction
Complementary and alternative medicines and non-pharmacological interventions increased extensively in biomedical literature toward an integrative approach of medicine and health.Patient demands and practitioner preference motivated researchers to intensify studies on this topic.An Australian survey evaluated the practice of complementary medicine (CM) by general practitioners (GP) and concluded that there is a real need for evidence-based CM [1] .In parallel, a Chinese review evaluated the quality of reporting Randomized Clinical Trials (RCTs) on traditional Chinese medicine treatments [2] .The review also concluded a real need for improving the standards of RCTs reports.A more recent study performed a bibliometric analysis of apitherapy from the scientific literature covering a 36-year period [3] .This study revealed that although there is a growing interest in apitherapy, an imbalance in research on this subject is observed in the literature which could be explained by the lack of resources.These different examples illustrate the point and show the diversity of therapies used in CM.Knowledge in the biomedical field is advancing every day, reaching millions of publications.For example, in cancer, representing one of the leading causes of death in the world after cardiovascular disease [4] , there are approximately 6.6 million publications (GoogleScholar November 2023).Among them, Kalya assessed that about 600,000 (10%) studies focus on complementary and alternative medicines (CAM).Besides, this volume increased to 58% during the last decade.Thus, the problem is how to identify these publications when there is no repository.
Besides, one of the difficulties which the researchers are facing is the exploitation of a vast volume of information resulting from a given request.Convert this mass of information into a structured form is a major challenge that constitutes the starting point for the development and the fine-tuning of a suitable query and automatic processing tool.To meet these needs, we created a virtual research assistant based on text mining techniques and specific ontologies to find, sort, and analyze worldwide CAMs scientific publications.Currently, we have essentially elaborated this tool for the topic of cancer, but extensions to other health topics are planned and still under development.In this article, we present Kalya Research (KR) cancer prevention and care as a real-time digital system dedicated to evidence-based CAMs.It references all CAM that are evaluated from clinical trials, reviews, systematic reviews meta-analysis which also are published in peerreviewed scientific journals.KR identifies all the elements contained in a publication, characterizes these elements by creating mastered metadata and ontologies.One of KR's strengths is its ability to characterize the trio: outcomes, diseases, and CAMs.With KR, researchers can refine their results with original filters and thus better target the desired corpus of articles.Not only that they can also visualize correlations, trends but also themes allowing to nourish their reflection.In addition, it can also visualize the strongholds of its research.Furthermore, KR has been designed to stay as close as possible to the needs of its users.With this in mind, we compared our tool with a Medline search for CAM publications addressing alopecia issues in women with breast cancer.The aim of this paper is to present KR, our new bibliographic research assistant dedicated to CAM.In the following, we provide a state-of-the-art review of existing bibliographic search tools, in section 2. Next, we introduce KR tool, the functionality and use of the tool in section 3.Then, we present the application architecture, and implementation in the section 4. In addition, we explained the experimental comparison between KR and Medline in a concrete example of bibliographic research as well as the impressions of test users.In the section 5, we present the results of this experiment.
The section 6 is dedicated to summarizing the current approach, discuss the learned lessons, and suggest future directions to meet the researchers' expectations.Finally, we conclude the paper in the section 0.

Related Work
With the quickly increasing volume of the biomedical literature, any biomedical researcher finds himself confronted with obstacles, well known to scientists, namely the extraction of relevant information, the sorting of documents, and their exploration.To overcome these obstacles, there are already various tools like text exploration by visual analytics that allow filtering query results [5] .The functionalities differ according to the tools.Some of them allow to query directly using keywords and to obtain a list of publications with the annotated abstract on Named Entity Recognition (NER).It's the case for BioIE (based on a rule model) [6] , LitSense (specialized in sentence retrieval) [7] , and GeneView (targeted biological entities) which also annotates the full text if it is avalaible [8] .Other tools provide additional functionalities, as in Thalia which displays the frequencies of an entity in the corpus of texts resulting from the query [9] , BioTextQuest+ that suggests various methods to cluster the abstracts [10] .Some tools provide other functionalities such as BEST [11] which is a biomedical entity search tool that provides visual analytics on frequent terms and interaction network for each identified entity.There is also FACTA+ [12] which furnishes the biomedical associated concepts with some text analysis pipeline.
There are some tools dedicated to a specific task like Quertle [13] that performs a semantic search in multiple biomedical databases (PubMed included) and runs a query via relationships between concepts.BioReader [14] is a binary text clustering tool enabling filtering relevant articles from query results.Then, in a general context, we notice that text mining techniques are becoming essential tools for finding and sorting relevant articles.
Digital bibliographic tools dedicated to CAMs are also available [15] .The Table 1 listed and compared CAM databases.CAM-QUEST® [16] , MOTRIAL [17] , LIVIVO [18] , AMED [19] , and CAMbase [20] are bibliographic databases covering a wide spectrum of CAM therapies.Only CAM-QUEST has a pre-defined request system by disease, therapy, and study design categories.CAMbase has not been kept up-to-date since 2005 and AMED groups mainly European journals.Moreover, there are many specialized bibliographic databases, for example, PEDro for physiotherapy [21] , HOMEOINDEX for homeopathy [22] , ABIM for phytotherapy and Indian medicine [23] , NAPRALERT for natural products [24] , OTseeker for occupational therapy [25] , Arthedata for art therapy [26] , CAIRSS for music therapy [27] , and CARDS for dietary supplements [28] .Furthermore, the enthusiasm of patients for natural medicines such as Chinese medicine questions "evidence-based medicine" and the number of websites dedicated to it is significant compared to other disciplines.The tendency to evaluate these practices is also growing and the bibliographic databases are flourishing.Many examples including -but not limited to-AcuTrials [29] , MANTIS [30] , CNKI [31] , Societas Medicinae Sinensis [32] , Wanfang [33] and Qigong FACTA+ [12] Text-mining system that helps users explore various types of (possibly hidden) associations in an easy and comprehensible HOMEOINDEX [22] The methods adopted are those used by major international medical databases (MEDLINE, LILACS), allowing data exchange.

Homeopathy Multidisciplinary Coverage
Variable Quality of

Information Outdated
Information MOTRIAL [17] Free and academic meta-search engine from data collection of non-pharmacological intervention trials All CAM Evidence-

Based Information
Search Functionality Specialized Focus Language Barrier NAPRALET [24] Natural Product Alert is a relational database of natural NCCIH [35] Information Diverse Topics Research Potential Bias NORPHCAM [36] The

Cross query not available
OTseeker [25] Referenced trials have been critically appraised for their validity and interpretability.

Qigong and Energy Medicine
Database [34] The

Tool Description
Currently KR reference 350,088 publications, 2,220 Non-Pharmacological Interventions (NPI), and 2,050 outcomes related to cancer.These results continue to grow as the tool integrates new publications daily.In this section, we present the usage of the tool through its search engine version and its visualization capabilities.
After login, users access the KR interface.The user interface is divided into three main parts (Figure 6): a search box, the left-side panel to select criteria, and the right-side panel for displaying the results.
In the search box, the user can enter keywords and by clicking on the "advanced" button below, users can add and/or exclude disease(s), NPI(s), outcome(s), or other terms.In addition, by clicking on the "Save" button, users can save and name their queries and choose to send the results to their mailboxes.In the following, we will describe the tool through a case study research example: what are the NPIs that prevent alopecia following chemotherapy?To answer this question, we enter the term chemotherapy in the search box and click on the advance button to enter the terms alopecia in the outcome research criterion (Figure 1 and Figure 2).We save our query by clicking on the green "save" button (Figure 2) so that we will be notified about new studies on this subject.The number of articles concerned with chemotherapy-induced alopecia (CIA) contained in the KR database is displayed below on the results panel.At the top left of the right-side panel, there are two tabs: "View by analytics" and "View by list".The first offers an analytical visualization of the results while keeping the filters panel available.The second offers a raw view of the list of articles as presented above.Thus, the first display shows the graphical analysis of the results, and we can see the list of NPIs studied and used to remedy this

Graph-based visualization
The users can navigate through the selection of articles via different graphics.KR offers an original visualization of the different key indicators such as the number of publications by year or by methods (Figure 1) and the countries interested in these research issues (Figure 3). Figure 4, represents the most cited NPIs, in our previous mentioned example, the frequent journals.The graphics are interactive.Indeed, users can also apply filters or refine their results by clicking on the bars of the histograms.For example, a user can choose to focus only on one NPI, the scalp cooling.To do so, users should click on the scalp cooling bar of the NPIs histogram at the top left (Figure 4) that will automatically reduce the selection (Figure 5).

Text information
In the "View by List" panel, the list of articles is displayed under the search box in the right-side panel, in chronological order -the most recent to the oldest.For each article, KR displays its title, its journal name, its date of publication, and the name of the first author.By clicking on the black arrow, the first lines of the abstract appear as well as the health goal, the disease, the outcomes, and NPI terms related to this article.In our example (Figure 7), for the first article, the health goal is Care and the disease is Various cancer (it means all cancers) whereas the outcomes are alopecia and fatigue while the NPI terms related to the article are the physical activity.The hand display, allowing users to click and access the article reference page.It is activated by hovering over the article information.
The left side panel allows the user to add filters to the initial search (Figure 6 and Figure 7).The publication period can be defined by filling in the fields under Publication date.Then, the study design of the article sought can be defined by checking the corresponding boxes in the "Methods" field.For example, it is easy to quickly know the number of trials (Figure 3).The choice is made by checking among the following methods: journal article, review, trial, observational study, randomized controlled trial (RCT), controlled trial, a meta-analysis (MA), and systematic review (SR).In the same way, the symptoms sought can be selected in the "Outcomes" field located just below.The list includes all outcomes related to the list of articles resulting from the initial research (made using the search box).It is the same for the "Disease" field.The prototypal study is a design study to evaluate the feasibility of a study.
All filters apply when the user clicks on the "Apply filter" button.These filters can be either deleted or saved for a future search, by clicking on the relevant buttons ("Remove filters" or "Save") at the top of the panel.In addition, at the top left of the filter panel, a blue button displays a contextual menu offering the user access to the saved queries, as well as to the personal lists of the saved articles.Back to our CIA research example, we had selected the NPIs of scalp cooling, to have the best levels of proof, we check RCT, SR and MA in the Method filter (Figure 8) and if we focus on the most recent publication i.e. the most recent, we can see by reading the conclusion of the abstract that the scalp cooling would be the most effective method to prevent CIA.

Methods
In this section, we present the tool's design, architecture, and implementation.Finally, we describe how our tool compares with a Medline search in a concrete example of bibliographic research.

Material
KR focuses on CAM techniques also called NPI. "NPIs are non-invasive methods of care (programs, products or services) whose efficacy in improving the health and quality of life (QOL) of human beings has been proven.Their effects on health and QOL markers are observable (with measured risks and benefits beyond mere user opinions) and can be linked to identified biological and/or psychosocial processes.They can also have a positive impact on health behaviors and socioeconomic indicators." [38][39]According to the French Platform CEPS (Collaborative d'Evaluation des programmes de Prévention et de Soins de support) platform, NPIs can be divided into 5 categories of intervention: nutritional; digital; physical; psychological and elemental interventions.
To identify CAM or NPI publications, we have created a lexicon of NPI terms, classified according to the 5 categories specified above.The classification is based on various sources such as the CEPS platform (https://plateformeceps.www.univ-montp3.fr/fr/english-0),the lexical resources from the National Center for Complementary and Integrative Health (NCCIH) [40] , and many others.This list cannot be exhaustive because of the daily monitoring of the different resources.
PubMed is considered as the most commonly used search engine by the medical researchers as it identifies much of the biomedical literature.Therefore, we are detecting CAM publications from this resource.At the beginning of 2020, PubMed references more than 30 million citations for biomedical literature from MEDLINE, life science journals, and online books.
We supplement this resource with other journals specialized in CAM but not referenced in PubMed such as Journal of Client-Centered Nursing Care.

Kalya Research Implementation
KR architecture is based on 5 components: frontend, backend, data science, medical and scientific expertise, and user evaluations (Figure 10).The operating principle is as follows.The backend manages data recovery via APIs and integration into data warehouses.The data science component is decomposed of many modules that make it possible to extract the information of interest and transform this data into valuable information.The frontend provides the web interface that will be accessible by the researchers.The medical expertise ensures the integrity, the enrichment of the data, and makes it possible to decide in the event of litigation.Finally, user assessment allows KR to provide a service that meets the real needs of the end-user.The security of the entire architecture is ensured by a strong authentication via an Open-Id SSO (Single Sign-On).

Detecting NPI in Natural language
KR uses a rule-based model to identify CAM articles among the thousands of biomedical literature publications.To optimize retrieval effectiveness, we define three rules: i.The first rule focuses on relevant publications containing NPI terms.Specifically, we define a score based on the term frequency and inverse document frequency (TF-IDF) ranking function to find the most appropriate documents [41] .The score is calculated for a predetermined list of terms: the lexicon of NPI terms, mentioned in the Material section, on a preprocessed text.This text is the result of the concatenation of the title, abstract, and keywords.We use pure text with natural language processing frameworks without lemmatization [42] .
1.The second rule focuses on the rejected articles to check false negatives from the first rule.We define a score based on the occurrences of words combining both the interest terms in addition to the exclusion terms.According to the definition of an NPI, we have created a lexicon of exclusion terms linked, most often, to the administration mode.For example, vitamin C is an NPI but if the administration mode is by injection then, this method will be rejected, and the article concerned will not be listed in our search engine.The score is applied for the NPI lexicon combined with the exclusion terms lexicon to the same text as for the previous rule but this time with a lemmatization step.
2. The third rule manages the false positives from the previous two rules.We define a score base on journal relevance and citation occurrences of CAM articles.This score is defined in such a way that it will favor articles published in dedicated journals (with a high CAM score, such as, the Integrative Cancer Therapies journal (https://journals.sagepub.com/home/ict).Besides, articles with many CAM publications in their references.This information is not always available.Indeed, it is possible that a journal has not yet been listed in our model (but listed in our database) or that we do not have access to the reference list of the publication analyzed.In this case, the decision of the last rule takes precedence.

Evaluation of the tool on a practical case
We have evaluated the tool in a practical case, that of alopecia in breast cancer.The aim is to compare the results, their relevance and the time taken to sort the results.
We used the criteria of the PICO (Patient Intervention Comparison and Outcomes) method.The bibliographic search was carried out using 2 different methods: Medline (see Table 2) and the KR tool (see Table 2).Given the search theme, the criteria are as follows: Patient: patients with breast cancer, whatever age and stage of cancer.
Intervention: any NPI Comparison: standard care or any NPI Outcome: alopecia.The literature search identified several publications whose quality had yet to be determined.It was therefore necessary to carry out a second selection of the publications thus obtained, so as to optimize the quality of the results and identify the studies answering the research question.
This selection was made based on the title and the abstract.The criteria used to include the article were the same as those used to define the search strategy.
Rejected studies met one of the following criteria: Population: studies not assessing the population concerned, lack of population data Intervention: not involving an NMI Type of study: pharmacological studies, in vitro or in vivo animal studies, study protocols.

Tool usability assessment
We assessed the usability of the tool with a group of 40 participants.They were mostly medical researchers from France and the USA.The evaluation session was divided into 3 steps: a) presentation of KR; b) training in the basic functionalities of KR; c) collection of user impressions and suggestions to improve KR through open discussions.literature monitoring.This leads to the study of how to further facilitate research to save working time and efforts.To remedy this, we plan to integrate text annotation models to facilitate the reading of abstracts by highlighting diseases, outcomes, NPI, etc.We are also studying the possibility of producing a graphical summary of all the abstracts as a complementary result for each query.

Results
In the light of these application examples, further improvements are possible.To manage the high volume of data, we planned to strengthen our architecture with Elastic Search for indexing data [46] , Redis for the in-memory data structured storage [47] , RabbitMQ to streamline communication between micro-services [48] .Likewise, the NPI detection model is based on methods that have been tested in many natural language processing models [49] .Since we have built a substantial knowledge database, we planned to assess the performance of models via text similarity analysis to detect CAM publications [50] .The challenge is to determine the most appropriate method, in terms of performance in our context, among the different existing approaches.We also plan to refine the annotation of the texts to enrich our metadata system.
In the literature, there are many biomedical NER systems linked to the surge of biomedical data [51][52] [53] .However, the task of identifying named entities in biomedicine still a complex task [54] .Another future challenge will be to assess the annotation system that best meets our needs or even combines several ones.
The development of computational linguistics and identification methods in data mining shows promising opportunities to guide the extraction and optimization of the sequestered CAM knowledge.These may be contained within the continuously growing biomedical literature.However, automatic approaches cannot completely replace the in-depth analysis required by a conservative expert.In addition, the diversity of both studies and evaluations challenges the implementation of a robust automatic system.It should also be noted that the construction of the KR database is based on the validation of our experts to reinforce the automatic detection system.Nonetheless, automated systems taken in combination with the expert knowledge is much opportunity to leverage new CAM knowledge.
Such a tool as KR is useful for any biomedical researcher in his process of scientific discovery in order to identify work plan before the start of a scientific study, discuss the results at the end of a study, define the scientific validity of an NPI in prevention or treatment of several pathologies and health issues (e.g., Nordic walking to combat cancer, meditation to limit stress...), and/or conduct systematic review higher covered.Besides, KR is useful for clinicians looking for new and proven results to support clinical decisions, for instance, to identify which NPIs are the most effective in preventing certain pathologies (Cancer, Diabetes...).The usefulness of KR is even more important as it allows to link the diseases with the NPIs and their outcomes.Moreover, we have created and structured our metadata, such a way for KR to make it possible to cross queries like "the set of studies on breast cancer and nutritional therapies with an economic evaluation", by using words "breast" and "nutritional therapies" and check "eco" in the Outcome field.The user can select the publications based on other outcomes evaluated by the study.We identify several outcomes such as clinical, psychological, social, behavioral, and economical.CAM-QUEST and LIVIVO allow crossing information.However, in CAM-QUEST the user can filter by study design no more else whereas in LIVIVO there are more criteria, but the proposed metadata does not contain the outcomes.So, to the best of our knowledge, this type of search does not exist on any other search engine and accords the uniqueness and the novelty of KR.

Limitations
We have identified several limitations to our work, which we describe below.
KR is mainly based on a single data source PubMed, which is the most common in biomedical research.However, the CAM literature finds its source in lesser-known journals such as Acta Chimica and Pharmaceutica Indica or Journal of Alternative and Complementary Medicine that are not referenced in PubMed.To remedy this, we could enrich our model with other bibliographic sources (Hindawi, ScienceDirect...).
CAM publications are detected using a single rule model, the operating principle of which is described in the article.
However, this strategy has its limitations, as it may miss publications of interest that will not be detected by our model or select publications that will not be of interest.One solution would be to implement several models with different operating modes, such as deep learning models based on BERT classifiers [55][56] , and add an arbitration model to select CAM publications.In this way, if several models are matched, we could increase our correct detection rate and decrease our error rate.
While specific approaches to extracting information from text have proved highly effective in the general domain, they have shown their limitations in specific sectors such as healthcare, for several reasons [38] .Firstly, pre-processing chains are difficult to set up to deal with complex medical vocabularies, acronyms, abbreviations and so on.There is also a variety of expression inherent to each author, which makes it difficult to recognize domain terms in texts and consequently the concepts linked to them, as well as to identify the relationships linking them.Furthermore, the machine learning approaches currently in use generally require a lot of data annotated by human experts, which is difficult to obtain in the medical environment, as annotations require a lot of time -which experts don't have -and sometimes, a significant level of expertise.Labeled examples may contain errors and inconsistencies due to variations in the attention paid by annotators.
What's more, these automatic approaches provide results with little explanation, making them difficult to interpret.In addition, in our case, simple recognition is not enough, as it can lead to confusion.Indeed, if we take the example of vitamin D, it's an NPI when it comes to dietary supplementation, but it's also a blood indicator.We need to find models that take context into account to face this difficulty.Several authors have taken an interest in the subject, one example being the BioALBERT model, which targets the modeling of inter-phrase coherence to better learn context-dependent representations [57] .
Finally, there are different terms for the same thing, for example, vitamin B5 and pantothenic acid.The NPI lexicon we've created manages some synonyms, but not all.To remedy this, NPIs' ontologies are under building [38] .This aims to facilitate the detection of articles, the management of synonyms and make it possible to propose categories for classifying articles.This can be combined with other sources, for example, iDISK (for dietary supplements) [58] , ethnopharmacological knowledge [59] , Customary Medicinal Knowledgebase (aboriginal plants) [60] , TCMGeneDIT (a database for associated traditional Chinese medicine, gene and disease information using text mining) [61] , TM-MC (constituent compounds of medicinal materials in Northeast Asia traditional medicine) [62] , and many others.In our perspectives, this ontology will be accessible via KR in which the user can navigate.
KR is constantly being updated and enriched with the extension to other medical themes, such as, healthy aging and physical activity which are scheduled for the next version.In addition, technologies are constantly developing to offer an even more efficient and relevant application to meet the needs of our end-users.

Conclusion
In this article, we present KR as a novel virtual research assistant tool for biomedical literature.KR is mainly dedicated to CAM.The originality of KR is to offer a double interface allowing the end-users to display the list of the results of their queries (as in a conventional search engine), perform an analytic visualization of these results, and notify the users with their saved queries' results.The biomedical researchers could easily navigate through these different displays and extract both the publications of interest meanwhile scrutinize key indicators from which they could establish initial findings or orient their research.In the context of enthusiasm for CAM prompted by an awakening of interest in mainstream medicine, an anthology of small tools has emerged [15] .Thus, documentary research on CAM has hitherto been possible using these different search engines but was a real challenge because most of these tools are very specialized for a specific set of therapies, some of which are no longer kept up-to-date [63][64] .KR is therefore prompt and time saver.
We compared literature searches via KR and Medline to find NPI solutions for breast cancer patients with alopecia.This simple example highlighted KR's advantage over Medline in terms of ease of use, speed and relevance of results.These initial results are encouraging and open up the prospect of modifications to optimize our detection models.
In addition, KR was evaluated and tested by more than 40 experts.Although our tool has a number of limitations, these evaluations provoke the enthusiasm of the experts on ergonomics and filters in the sense that KR saves months of work.
However, the experts believe KR needs to enrich its metadata and to dive into the data by proposing, for a given query, a visual exploration of a texts' corpus and/or highlighting the key results of this corpus.All these proposed modifications and more come in-line with our perspectives for future improvements.
In the future, we plan to improve KR on technical points to streamline data management with the implementation of Elastic Search [46] and RabbitMQ [48] or the implementation of text similarity to detect CAM publications more easily [50] .
Besides, we propose to increase KR coverage by adding other biomedical literature sources like Google Scholar and Scopus and by expanding to other health topics.Furthermore, we are working on a new evaluation indicator for scientific publications to measure their scientific relevance and consequently the benefit of a CAM for a given context.In addition, we work on new functionalities such as NER to highlight diseases, NPIs, and outcomes at a glance.

Figure 1 .
Figure 1.Screen shot of a research example in Kalya Research tool -enter query

Figure 2 .
Figure 2. Screen shot of a research example in Kalya Research tool -save query.

Figure 3 .
Figure 3. World map of articles related to chemotherapy-induced alopecia.

Figure 4 . 26 Figure 5 .
Figure 4. Visualizations of key indicators summarizing results of the query related to chemotherapy-induced alopecia.
health goal criterion includes the following choices: Care, Cure and Prevent.Indeed, Care is related to supportive or treatment care, Cure is related to healing or recovery, and Prevent is related to health troubles that require to be prevented.The different types of the population studied are Adult, Child and Elderly.The minimum number of participants is chosen by filling the ad hoc input field.Next, a selection can be made on the country using the checkbox.Finally, the last criterion concerns the Research design of the study with the following choices: Mechanistic, Interventional, Observational, Implementation, and Prototypal.In details, a mechanistic study uses various experiments to find causal Qeios, CC-BY 4.0 • Article, December 11, 2023 Qeios ID: IW54X7.2 • https://doi.org/10.32388/IW54X7.2 11/26observational study describes the natural use and response of NPIs, an implementation study is a monitoring study, and a

Figure 6 . 26 Figure 7 .
Figure 6.Screen shot of a research example in Kalya Research tool -View by List -view 1.

Figure 8 .
Figure 8. Kalya research tool: refine a query with Method filter to have the best level of proof.

Figure 9 .
Figure 9. Kalya research tool display new results for saved queries.By clicking in the contextual menu in the left banner, New Results is displayed

(
Integrative medicine[MeSH Terms] OR Complementary Therapies[MeSH Terms] OR Alternative Medicine[MeSH Terms] OR Traditional Medicine Mind-Body Therapies[MeSH Terms] OR Dietary Supplements[MeSH Terms] OR Therapeutics[MeSH Terms] OR Physical Therapy Modalities[MeSH Terms] OR Psychotherapy[MeSH Terms] OR Rehabilitation[MeSH Terms]) NOT Tamoxifen NOT "Aromatase inhibitors " NOT Anastrozole NOT Letrozole NOT Exemestane NOT Fulvestrant NOT Palbociclib NOT Ribociclib NOT Adriamycin NOT Doxorubicin NOT Cyclophosphamide NOT Paclitaxel NOT Docetaxel NOT Epirubicin NOT Gemcitabine NOT "5-Fluorouracil 5-FU" NOT Methotrexate NOT Vinorelbine NOT Trastuzumab NOT Herceptin NOT Pertuzumab NOT Perjeta NOT Lapatinib NOT Tykerb NOT T-DM1 NOT "Ado-Trastuzumab Emtansine" NOT Everolimus NOT Afinitor NOT Palbociclib NOT Ibrance NOT Ribociclib NOT Kisqali NOT Abemaciclib NOT Verzenio NOT Atezolizumab NOT Tecentriq NOT Pembrolizumab NOT Keytruda NOT Nivolumab NOT Opdivo NOT Bisphosphonates NOT Denosumab NOT "Zoledronic acid" NOT Zometa NOT Pamidronate NOT Aredia NOT Denosumab NOT Xgeva NOT "PARP Inhibitors" NOT Olaparib NOT Lynparza NOT Talazoparib NOT Talzenna NOT "Cytotoxic Antibiotics" NOT Eribulin NOT Halaven NOT Bevacizumab NOT Avastin (alopecia[Title/Abstract] or "hair loss" [Title/Abstract])

Table 1 .
Comparing CAM databases strenghs and weaknesses Qeios, CC-BY 4.0 • Article, December 11, 2023 Network of Researchers in the Public Health of

Table 2 .
Research algorithms for Kalya Research and Medline for alopecia in breast cancer.