Manuscript received February 12, 2025; revised April 21, 2025; accepted May 8, 2025; published August 7, 2025.

Abstract—Breast Cancer (BC) is the most frequent form of cancer, accounting for 24.5% of all cancer cases worldwide, with projections estimating 364,000 cases by 2040. Accurate diagnosis and effective categorization of BC are essential for proper treatment planning, patient management, and improved survival. Traditionally, pathologists examine histopathology specimens manually using a microscope to categorize the BC, which is labor-intensive, time-consuming, prone to subjectivity and constrained by experts’ availability. An automated approach can address these limitations; however, previous methods, particularly those based on Convolutional Neural Networks (CNNs), often struggle with data imbalance, poor accuracy and poor generalizability across datasets, especially in multiclass BC categorization. This study presents an automated BC categorization method leveraging whole slide histopathology images and a transformer-based deep learning model. The proposed method uses a cascade of transformers to classify BC using 40× histopathology images, following the taxonomy defined by the BRACS dataset, distinguishing between benign, atypical, and malignant cases. First, it classifies BC into three primary categories—benign, atypical and malignant—and subsequently determines the specific sub-types within each category. The proposed method was validated using two widely recognized datasets: BRACS and BreakHis. On BRACS, it achieved 95.6% accuracy in classifying BC into benign, atypical, and malignant categories, with sub-type accuracies of 94.7% for benign, 98.6% for atypical, and 99.1% for malignant cases. On the BreakHis dataset, the model achieved 93% accuracy for binary benign-malignant classification, with sub-type accuracies of 94% and 91% for benign and malignant cases, respectively. The proposed method outperformed existing methods in accuracy and robustness, making it a promising tool for automated BC diagnosis and classification.

Keywords—breast cancer, whole slide image, vision transformer, histopathology image, cancer classification

Cite: Md Shakhawat Hossain, Ashifur Rahman, Munim Ahmed, Kaniz Fatema, MM Mahbubul Syeed, Mohammad Anowar Hussen, and Mohammad Faisal Uddin, "Vision Transformer for the Categorization of Breast Cancer from H&E Histopathology Images," Journal of Image and Graphics, Vol. 13, No. 4, pp. 380-393, 2025.

Copyright © 2025 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC-BY-4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.