Systematic review and meta-analysis: processes towards selection automation

Abstract

Public health evidence and other disciplines are being produced and published at an unprecedented rate and scale. This evidence can take many forms, the more common and conventional types and sources of published evidence include journal articles, conference abstracts/proceedings, technical reports, and clinical trial records/registries. To keep pace with the rapidly evolving public health landscape, and to respond to the critical needs, issues, and public health crises of today in a timely manner, there is a growing need to explore, leverage and integrate insights from more novel sources of evidence. A common thread across all this evidence is that such data is, at large, stored in a noisy, unstructured format, which makes secondary research-led activities in data extraction, synthesis, and reporting incredibly challenging. Secondary public health research methods, such as evidence synthesis and systematic reviewing, are spreading across all research fields. The aim of this research project was to establish an evidence-based framework for an optimal Natural Language Processing (NLP) solution (including a working prototype) to support public health evidence extraction and synthesis research activity. The latest innovations in artificial intelligence (AI), machine learning, and NLP tools and techniques offer the ability to rapidly extract, analyze, synthesize, and understand unstructured textual data, at scale. Recent breakthroughs in these technologies have led to vastly improved NLP models, which are able to capture and model more complex linguistic relationships than ever before. By providing the ability to assess and analyze large quantities of this data, NLP has opened up vast opportunities The aim of this research project was to establish an evidence-based framework for an optimal NLP solution (including a working prototype) to support public health evidence extraction and synthesis research activity. The traditional systematic review framework is a feasible starting point, where all steps are predicted and standardized. In order to reduce systematic reviewer burden while maintaining the high standards of systematic review validity and comprehensiveness we, developed a semi-automation screening approach using the reviewer’s criteria written in natural language. We offer a simplified topic’s extraction too and compare it to the traditional Latent Dirichlet Allocation (LDA). For clustering of studies, we transformed the title, abstract and keywords, into a wordcloud for each study, and grouped using a NLP technique called Sentence Boundary Detection for finding and segmenting meaningful individual sentences, studies with same sentences are put together, organized, and clustered by sentences frequency. We achieve the generation of summary for clustered studies using natural language generation. We perform a comparison of Markov Chain Generation with Recurrent Neural Network Generation for quality assessment of the generated text. We obtain data graphics by exploring BIBTEXdata already available, and mining relations of semantic changes or author’s groups of collaboration. The results methodology follows the best practices for conducting and reporting reviews, thus solving a practical problem effectively with reproducible and repeatable results. These results show that the desired tool is feasible with the current state of the art technology. This work resulted in a startup that delivers products to explore and analyze scientific documents in large scale, and it has been validated by the end user.