Parameter-efficient fine-tuning and few-shot learning of multiscale vision transformers for liver tumour segmentation in CT Conference Paper
| Authors: | Mojtahedi, R.; Hamghalam, M.; Peoples, J. J.; Jarnagin, W. R.; Do, R. K. G.; Simpson, A. L. |
| Title: | Parameter-efficient fine-tuning and few-shot learning of multiscale vision transformers for liver tumour segmentation in CT |
| Conference Title: | Medical Imaging 2025: Computer-Aided Diagnosis |
| Abstract: | Parameter-efficient fine-tuning (PEFT) of models is a rapidly growing field of research in medical image segmentation. In this study, we integrated Low-Rank Adaptation (LoRA) blocks into the state-of-the-art (SOTA) abdominal organ segmentation model—Swin UNETR—to improve fine-tuning efficiency with limited imaging data for liver and tumour segmentation. LoRA reduces the number of trainable parameters by using low-rank matrix decomposition to adapt the model efficiently. Since the LoRA rank choice affects the balance between parameter efficiency and performance, we compared segmentation performance across different LoRA ranks (2, 4, 8, and 16) with varying sample sizes (5, 10, and 15) against the entire training set. We used computed tomography (CT) scans from patients with intrahepatic cholangiocarcinoma (ICC), colorectal liver metastases (CRLM), and hepatocellular carcinoma (HCC). With an average liver and tumour Dice score of 0.666 and a 95th percentile Hausdorff distance (HD95) of 49.4 mm, the LoRA-enhanced model with rank 8 showed the best segmentation performance. Compared to traditionally fine-tuning all model parameters, this resulted in a 66% decrease in HD95 and a 15% improvement in Dice score. Furthermore, the model trained on only 10 data samples achieved a Dice score of 0.666, just 0.3% lower than the score achieved using the entire training set. This study demonstrates how the proposed pipeline can potentially improve computational efficiency and clinical workflows in cancer treatment for scenarios with limited access to annotated data. The code can be accessed at: https://github.com/Ramtin-Mojtahedi/PEFT-FSL-MViT-LTS. © 2025 SPIE. |
| Keywords: | computed tomography; fine tuning; computed tomography (ct); image segmentation; segmentation performance; vision transformer; swin transformer; adaptive learning; few-shot learning (fsl); liver tumour segmentation; low-rank adaptation (lora); parameter-efficient fine-tuning (peft); swin transformers; vision transformers (vits); few-shot learning; liver tumor segmentations; low-rank adaptation; parameter-efficient fine-tuning |
| Journal Title | Progress in Biomedical Optics and Imaging - Proceedings of SPIE |
| Volume: | 13407 |
| Conference Dates: | 2025 Feb 17-20 |
| Conference Location: | San Diego, CA |
| ISBN: | 1605-7422 |
| Publisher: | SPIE |
| Date Published: | 2025-01-01 |
| Start Page: | 1340738 |
| Language: | English |
| DOI: | 10.1117/12.3046253 |
| PROVIDER: | scopus |
| DOI/URL: | |
| Notes: | Conference paper (ISBN: 9781510685925) -- Source: Scopus |
Altmetric
Citation Impact
BMJ Impact Analytics
Related MSK Work