Pulmonary nodules are small growths in the lung and a common incidental finding on computed tomography (CT) scans. They are mostly benign but in some cases are early lung cancers. The earlier that a lung cancer is identified and treated, the more likely the outcome will be successful. However, currently, lung nodules may need to be observed with repeat CT scans for up to two years to be able distinguish between benign and malignant nodules. A key clinical challenge is to improve the ability to predict malignancy using the initial CT scan, which would remove the diagnostic delay introduced by the need for repeat scans.
There are two types of lung cancer risk prediction models, which use either statistical approaches, such as the Brock model, or machine learning approaches such as the Lung Cancer Prediction Convolutional Neural Network, LCP-CNN developed by Oxford spin-out Optellum Ltd. The Brock model is currently recommended in clinical practice by the British Thoracic Society and incorporates patient clinical information and nodule characteristics to predict which nodules are likely to be malignant.
LCP-CNN is an externally validated artificial intelligence model that performs better than the Brock model at predicting malignancy. However, because of its AI-based nature, LCP-CNN is not fully interpretable and therefore the importance of individual parameters on the model performance are not known.
In a report published in the journal European Radiology, Dr Madhurima Chetan (Oxford University Hospitals NHS Trust) and colleagues investigated the importance of CT imaging features that contributed to the enhanced performance of LCP-CNN. The team studied data from 4,660 participants with 10,485 lung nodules, of which 556 were malignant.
While automating the measurement of nodules improves the accuracy of the Brock model compared to manual measurement, the LCP-CNN still outperforms both, indicating that the improvements seen with the LCP-CNN model are not simply due to measuring nodule size optimally. To study the effects of individual predictors on the LCP-CNN accuracy, a series of features were ablated from the CT images. Similar to the Brock model, nodule size and shape play the largest role in AI prediction, with nodule internal texture and background lung tissue having a limited role.
This work demonstrates the feasibility of interrogating AI model performance using CT image feature ablation and provides a basis for further research on understanding AI prediction.