Systematic Review With Meta-analysis

Artificial Intelligence in the Diagnosis of Oesophageal Diseases

Pierfrancesco Visaggi; Brigida Barberio; Dario Gregori; Danila Azzolina; Matteo Martinato; Cesare Hassan; Prateek Sharma; Edoardo Savarino; Nicola de Bortoli

Disclosures

Aliment Pharmacol Ther. 2022;55(5):528-540.

Abstract and Introduction

Abstract

Background: Artificial intelligence (AI) has recently been applied to endoscopy and questionnaires for the evaluation of oesophageal diseases (ODs).

Aim: We performed a systematic review with meta-analysis to evaluate the performance of AI in the diagnosis of malignant and benign OD.

Methods: We searched MEDLINE, EMBASE, EMBASE Classic and the Cochrane Library. A bivariate random-effect model was used to calculate pooled diagnostic efficacy of AI models and endoscopists. The reference tests were histology for neoplasms and the clinical and instrumental diagnosis for gastro-oesophageal reflux disease (GERD). The pooled area under the summary receiver operating characteristic (AUROC), sensitivity, specificity, positive and negative likelihood ratio (PLR and NLR) and diagnostic odds ratio (DOR) were estimated.

Results: For the diagnosis of Barrett's neoplasia, AI had AUROC of 0.90, sensitivity 0.89, specificity 0.86, PLR 6.50, NLR 0.13 and DOR 50.53. AI models' performance was comparable with that of endoscopists (P = 0.35). For the diagnosis of oesophageal squamous cell carcinoma, the AUROC, sensitivity, specificity, PLR, NLR and DOR were 0.97, 0.95, 0.92, 12.65, 0.05 and DOR 258.36, respectively. In this task, AI performed better than endoscopists although without statistically significant differences. In the detection of abnormal intrapapillary capillary loops, the performance of AI was: AUROC 0.98, sensitivity 0.94, specificity 0.94, PLR 14.75, NLR 0.07 and DOR 225.83. For the diagnosis of GERD based on questionnaires, the AUROC, sensitivity, specificity, PLR, NLR and DOR were 0.99, 0.97, 0.97, 38.26, 0.03 and 1159.6, respectively.

Conclusions: AI demonstrated high performance in the clinical and endoscopic diagnosis of OD.

Introduction

Artificial intelligence (AI) is being extensively applied to different medical settings aiming to improve the performance in the diagnosis of various diseases, including gastrointestinal (GI) diseases. The term AI generically refers to complex computer algorithms that mimic human cognitive functions, including learning and problem-solving.^[1] Machine learning (ML) is a field of AI that can be taught to discriminate characteristics of data samples and then apply experience to interpret previously unknown information.^[2] Supervised ML with support vector machine (SVM) is based on hand-crafted algorithms in which researchers, based on clinical knowledge, manually indicate features of interest of an input data set (labelled data set) to train the system to recognise discriminative features and provide appropriate outputs.^[1] Deep learning (DL) is a subset of ML which can autonomously extract discriminative attributes of input data through artificial neural networks, often organised as convolutional neural networks (CNNs), which are constituted of multiple layers of non-linear functions.^[1,3]

AI is increasingly being integrated into computer-aided diagnosis (CAD) systems for GI diseases to improve detection (CADe) and characterisation (CADx) of pathology. Consistently, recent meta-analyses concluded that the use of AI during lower endoscopic procedures significantly increased the detection of colorectal neoplasia.^[4,5]

More recently, various studies evaluating the performance of AI in the diagnosis of oesophageal diseases (ODs) have been published. The main application of AI in the upper GI tract is endoscopy and neoplasia detection. Ideally, upper GI endoscopies and biopsies should not miss lesions, but the ability to recognise endoscopic images depends on individual expertise. This is particularly relevant for subtle upper GI lesions, where experienced endoscopists can make a difference in the diagnosis. In this setting, CAD tools have the potential to successfully assist both trainee and expert physicians to reduce variability in the detection of upper GI pathology, increasing the diagnostic accuracy regardless of individual expertise and virtually overcoming inter- and intra-observer variability.^[6]

In addition, deep learning using multi-layered neural networks powered by high-performance computing clusters are capable of recognizing complex non-linear patterns in datatypes that previously were intractable to process, such as endoscopic images and videos. In this setting, AI has been applied to clinical questionnaires for gastro-oesophageal reflux disease (GERD), pH-impedance and oesophageal manometry tracings, and for the evaluation of mRNA transcripts in the diagnosis of eosinophilic oesophagitis (EoE).

DL models are black boxes in which the input data and the output (diagnosis) are known, but the processes by which the diagnosis is achieved are not, and this may be counterproductive.^[6] Accordingly, research is already heading to understand how DL models make decisions to solve interpretability gaps, and methods to understand the process of CNN-based choices are being developed.^[7,8]

AI support in decision-making is a fascinating and rapidly evolving topic. Accordingly, we performed a systematic review with meta-analysis of currently available evidence on the performance of AI in the diagnosis of oesophageal diseases (ODs), updating previous evidence on oesophageal cancer^[9,10] and assessing evidence on the performance of AI in the detection of intrapapillary capillary loops (IPCLs) and in the diagnosis of benign ODs.

1 2 3 4

Next Section

Abstract and Introduction
Methods
Results
Discussion
References

Table 1. Sub-group analyses of the performance of artificial intelligence in the diagnosis of oesophageal diseases
Oesophageal disease	Subgroups	Number of studies	Sensitivity (95% CI)	Specificity (95% CI)	PLR (95% CI)	NLR (95% CI)	DOR (95% CI)	AUROC (95% CI)	P value*
Barrett's neoplasia	All studies	9	0.89 (0.84–0.93)	0.86 (0.83–0.93)	6.50 (1.59–2.15)	0.13 (0.20–0.08)	50.53 (24.74–103.22)	0.90 (0.85–0.94)	—
	Country								0.04
	Europe	8	0.85 (0.81–0.89)	0.84 (0.80–0.88)	5.50 (1.43–1.98)	0.17 (0.23 −0.13)	32.07 (18.04–57.00)	0.85 (0.84–0.93)
	America	1	0.97 (0.82–0.99)	0.97 (0.66–1.00)	28.62 (0.87–6.06)	0.04 (0.27–0.01)	816.6 (8.73–76 349.10)	0.98 (0.90–0.99)
	Study type								0.45
	Retrospective	6	0.90 (0.85–0.94)	0.87 (0.82–0.90)	6.69 (1.54–2.25)	0.11 (0.18–0.07)	59.54 (25.57–138.60)	0.93 (0.87–0.97)
	Prospective	3	0.84 (0.70–0.92)	0.86 (0.79–0.91)	5.87 (1.19–2.28)	0.19 (0.39–0.08)	31.66 (8.51–117.78)	0.87 (0.80–0.94)
	Algorithm type								0.94
	DL	7	0.89 (0.83–0.93)	0.87 (0.83–0.90)	6.80 (1.60–2.22)	0.12 (0.21–0.07)	54.65 (24.01–124.4)	0.91 (0.86–0.95)
	SVM	2	0.89 (0.70–0.97)	0.84 (0.72–0.91)	5.45 (0.91–2.39)	0.13 (0.42–0.04)	42.86 (5.95–308.51)	0.87 (0.78–0.97)
	Endoscopy type^d,e								0.64
	WLE	7	0.89 (0.82–0.94)	0.86 (0.82–0.89)	6.43 (1.53–2.17)	0.12 (0.21–0.01)	52.03 (21.56–125.58)	0.89 (0.84–0.94)
	NBI	2	0.89 (0.77–0.95)	0.96 (0.47–1.00)	20.19 (0.37–6.23)	0.11 (0.5–0.05)	177.11 (2.9–10 821.79)	0.93 (0.75–0.99)
	Real-time								0.2
	Yes	3	0.81 (0.73–0.87)	0.84 (0.79–0.89)	5.20 (1.25–2.03)	0.22 (0.94–0.15)	23.16 (10.35–51.81)	0.82 (0.80–0.92)
	No	6	0.92 (0.87–0.95)	0.87 (0.82–0.91)	7.11 (1.60–2.32)	0.10 (0.16–0.06)	73.32 (30.61–175.63)	0.93 (0.86–0.96)
OSCC^a	All studies	5	0.95 (0.91–0.98)	0.92 (0.82–0.97)	12.65 (1.61–3.51)	0.05 (0.11–0.02)	258.36 (44.18–1510.7)	0.97 (0.92–0.98)	—
	Endoscopy type^f								0.74
	WLE	4	0.95 (0.86–0.98)	0.93 (0.77–0.98)	14.42 (1.31–4.11)	0.05 (0.18–0.02)	277.2 (19.94–3852.9)	0.98 (0.95–0.99)
	NBI	2	0.96 (0.83–0.99)	0.96 (0.94–0.97)	23.49 (2.59–3.62)	0.04 (0.19–0.01)	537.21 (71.81–4018.64)	0.98 (0.94–0.99)
	Real-time								0.1
	Yes	2	0.94 (0.79–0.99)	0.98 (0.94–0.99)	39.4 (2.51–4.73)	0.06 (0.23–0.01)	651.92 (53.83–7895.1)	0.99 (0.94–0.99)
	No	3	0.96 (0.92–0.98)	0.87 (0.71–0.95)	7.29 (1.14–2.93)	0.05 (0.11–0.03)	143.03 (27.61–741.01)	0.96 (0.89–0.97)
GERD^b	All studies	3	0.97 (0.67–1.00)	0.97 (0.75–1.00)	38.26 (0.98–6.22)	0.03 (0.44–0.00)	1159.6 (6.12–219 711.69)	0.99 (0.80–0.99)	—
	Country								—
	Europe	2	0.99 (0.98–1.00)	0.99 (0.95–1.00)	145.88 (3.05 −6.95)	0.01 (0.02–0.00)	16 120.13 (1009.41–257436.50)	0.98 (0.97–0.99)
	Asia	1	0.70 (0.59–0.80)	0.78 (0.66–0.87)	3.25 (0.55–1.81)	0.38 (0.62–0.23)	8.61 (2.8–26.48)	—
	Algorithm type								—
	SVM	2	0.99 (0.98–1.00)	0.99 (0.95–1.00)	145.88 (3.05 −6.95)	0.01 (0.02–0.00)	16 120.13 (1009.41–257436.50)	0.98 (0.98–0.99)
	DL	1	0.70 (0.59–0.80)	0.78 (0.66–0.87)	3.25 (0.55–1.81)	0.38 (0.62–0.23)	8.61 (2.8–26.48)	—
IPCL^c	All studies	2	0.94 (0.67–0.99)	0.94 (0.84–0.98)	14.75 (1.46–3.70)	0.07 (0.39–0.01)	225.83 (11.05–4613.93)	0.98 (0.86–0.99)	—
	Algorithms								0.001
	Best	2	0.99 (0.97–1.00)	0.97 (0.96–0.98)	32.18 (3.18–3.76)	0.01 (0.03–0.00)	2779.61 (804.87–9599.39)	0.98 (0.97–0.99)
	Worst	2	0.73 (0.55–0.86)	0.87 (0.71–0.95)	5.74 (0.64–2.85)	0.31 (0.64–0.15)	18.56 (2.97–115.85)	0.87 (0.66–0.96)

Abbreviations: AI, artificial intelligence; AUROC, area under the summary receiver operating characteristic curve; DL, deep learning; DOR, diagnostic odds ratio; IPCL, intrapapillary capillary loop; NLR, negative likelihood ratio; OSCC, oesophageal squamous cell carcinoma; PLR, positive likelihood ratio; SVM, support vector machine.
^aAll studies have been conducted in Asia, with deep learning, and are retrospective.
^bAll the studies have been conducted prospectively and are real time.
^cAll studies have been conducted in Asia, with deep learning and narrow-band imaging, and are retrospective and real time.
^dHashimoto et al assessed both WLE and NBI in the same study.
^eOne study was not included in the sub-analysis as data for white-light endoscopy and narrow band imaging had not been analysed separately in the study.
^fLi et al assessed both WLE and NBI in the same study.
*P value of comparison between groups. The P value has been computed via bootstrap with 2000 resampling.

Table 2. Performance of endoscopists versus performance of artificial intelligence in the diagnosis of oesophageal diseases
Oesophageal disease	Endoscopist or AI	Number of studies	Sensitivity (95% CI)	Specificity (95% CI)	PLR (95% CI)	NLR (95% CI)	DOR (95% CI)	AUROC (95% CI)	P value
Barrett's neoplasia	AI	10	0.89 (0.84–0.93)	0.86 (0.83–0.93)	6.50 (1.59–2.15)	0.13 (0.20–0.08)	50.53 (24.74–103.22)	0.91 (0.82–0.95)	0.98
Barrett's neoplasia	Endoscopist	3	0.93 (0.66–0.99)	0.85 (0.71–0.93)	6.17 (0.82–2.63)	0.09 (0.48–0.01)	70.12 (4.70–1045.93)	0.90 (0.85–0.95)	0.98
OSCC	AI	8	0.95 (0.91–0.98)	0.92 (0.82–0.97)	12.65 (1.61–3.51)	0.05 (0.11–0.02)	258.36 (44.18–1510.7)	0.97 (0.92–0.98)	0.11
OSCC	Endoscopist	2	0.75 (0.68–0.80)	0.88 (0.84–0.92)	6.46 (1.46–2.27)	0.29 (0.38–0.22)	22.45 (11.5–43.84)	0.88 (0.83–0.98)	0.11

Abbreviations: AI, artificial Intelligence; AUROC, area under the summary receiver operating characteristic curve; DOR, diagnostic odds ratio; IPCL, intrapapillary capillary loop; NLR, negative likelihood ratio; OSCC, oesophageal squamous cell carcinoma; PLR, positive likelihood ratio.