Explainable Histology Images Classification Using an Ensemble of Fractal Geometry, CNNs, and ViT
DOI:
https://doi.org/10.47839/ijc.25.3.4843Keywords:
Histology Image Classification, Fractal Geometry, Convolutional Neural Networks, Ensemble Learning, Deep Learning, Vision Transformers, Grad-CAM VisualizationAbstract
Accurately classifying histopathological images is still a fundamental but difficult task in medical diagnostics, because biological tissues are irregular, self-similar, and multi-scale. While effective in tasks involving natural images, convolutional neural networks (CNNs) show limited generalization when confronted with highly heterogeneous tissue structures, small datasets, and variable resolutions. In this paper, we propose a tri-branch ensemble model, called Fractal-CNN-ViT, for histology images classification, which combines the advantages of Vision Transformers (ViTs), CNNs, and fractal geometry. In this model, fractal descriptors—specifically fractal dimension, lacunarity, and percolation—are extracted from histology images, transformed into artificial RGB images, and then processed by a specialized CNN (Fractal-CNN). In order to capture global contextual relationships, a second CNN (ResNet-50) and a ViT simultaneously analyze the original images. The outputs of the three branches are fused through an adaptive weighted ensemble module, which maps the optimal contribution of each stream to the final decision. Experimental results on the BreakHis dataset demonstrated that Fractal-CNN-ViT outperforms standalone deep learning architectures: Fractal-CNN, CNN (ResNet-50) and ViT. It achieved an accuracy of 92.3%, surpassing Fractal-CNN (85.9%), ResNet-50 (87.2%) and ViT (88.5%), confirming the benefits of integrating fractal-based features with deep learning methods. These findings demonstrate how well the proposed ensemble model combines local and global tissue representations, reduces overfitting on small datasets, and generalizes across magnification levels. Finally, Grad-CAM visualization was employed to provide visual interpretability, increasing clinical trust by highlighting image regions most influential to the studied models' classification decisions.
References
G. Litjens, T. Kooi, B. E. Bejnordi, et al., “A survey on deep learning in medical image analysis,” Medical Image Analysis, vol. 42, pp. 60-88, 2017. https://doi.org/10.1016/j.media.2017.07.005.
A. Janowczyk and A. Madabhushi, “Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases,” Journal of Pathology Informatics, vol. 7, issue 1, 2016. https://doi.org/10.4103/2153-3539.186902.
G. Aresta, T. Araújo, S. Kwok, et al., “BACH: Grand challenge on breast cancer histology images,” Medical Image Analysis, vol. 56, pp. 122–139, 2019. https://doi.org/10.1016/j.media.2019.05.010.
R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, “Convolutional neural networks: an overview and application in radiology,” Insights into Imaging, vol. 9, issue 4, pp. 611–629, 2018. https://doi.org/10.1007/s13244-018-0639-9.
F. A. Spanhol, L. S. Oliveira, C. Petitjean and L. Heutte, “A dataset for breast cancer histopathological image classification,” IEEE Transactions on Biomedical Engineering, vol. 63, issue 7, pp. 1455-1462, 2016. https://doi.org/10.1109/TBME.2015.2496264.
T. Araújo, G. Aresta, E. Castro, J. Rouco, P. Aguiar, C. Eloy, A. Polónia and A. Campilho, “Classification of breast cancer histology images using convolutional neural networks,” PLoS One, vol. 12, issue 6, 2017. https://doi.org/10.1371/journal.pone.0177544.
A. Madabhushi and G. Lee, “Image analysis and machine learning in digital pathology: Challenges and opportunities,” Medical Image Analysis, vol. 33, pp. 170-175, 2016. https://doi.org/10.1016/j.media.2016.06.037.
C. Wemmert, J. Weber, F. Feuerhake, and G. Forestier, “Deep learning for histopathological image analysis,” in Mourad Elloumi (Ed.), Deep Learning for Biomedical Data Analysis. Techniques, Approaches, and Applications, Springer, pp.153-169, 2021. https://doi.org/10.1007/978-3-030-71676-9_7.
B. B. Mandelbrot, The Fractal Geometry of Nature, third ed., New York, NY, USA: W. H. Freeman, 1983. https://doi.org/10.1119/1.13295.
G. F. Roberto, F. A. Faria, H. Bragança, A. A. Santos, J. P. Papa, and D. Menotti, “Fractal neural network: A new ensemble of fractal geometry and convolutional neural networks for the classification of histology images,” Expert Systems with Applications, vol. 166, p. 114103, 2021. https://doi.org/10.1016/j.eswa.2020.114103.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” International Conference on Learning Representations (ICLR), Vienna, Austria, May 04 2021.
S. Khan, M. Naseer, M. Hayat, et al., “Transformers in vision: A survey,” arXiv:2101.01169v1 [cs.CV] 4 Jan 2021. https://doi.org/10.1145/3505244.
R. R. Selvaraju, M. Cogswell, A. Das, et al., “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision (IJCV), vol. 128, pp. 336–359, 2020. https://doi.org/10.1007/s11263-019-01228-7.
G. F. Roberto, L. A. Neves, M. Z. Nascimento, T. A. Tosta, L. C. Longo, A. S. Martins, and P. R. Faria, “Features based on the percolation theory for quantification of non-Hodgkin lymphomas,” Computers in Biology and Medicine, vol. 91, pp. 135–147, 2017. https://doi.org/10.1016/j.compbiomed.2017.10.012.
M. Ivanovici, and N. Richard, “Fractal dimension of color fractal images,” IEEE Transactions on Image Processing, vol. 20, issue 1, pp. 227–235, 2011. https://doi.org/10.1109/TIP.2010.2059032.
N. Bayramoglu, J. Kannala, and J. Heikkilä, “Deep learning for magnification independent breast cancer histopathology image classification,” Proceedings of the 23rd International Conference on Pattern Recognition (ICPR), 04-08 December 2016, Cancun, Mexico, pp. 2440–2445. https://doi.org/10.1109/ICPR.2016.7900002.
M. Talo, “Automated classification of histopathology images using transfer learning,” Artificial Intelligence in Medicine, vol. 101, 101743, 2019. https://doi.org/10.1016/j.artmed.2019.101743.
R. E. Plotnick, R. H. Gardner, W. W. Hargrove, K. Prestegaard, and M. Perlmutter, “Lacunarity analysis: A general technique for the analysis of spatial patterns,” Physical Review E, vol. 53, issue 5, pp. 5461–5468, 1996. https://doi.org/10.1103/PhysRevE.53.5461.
C. Allain and M. Cloitre, “Characterizing the lacunarity of random and deterministic fractal sets,” Physical Review A, vol. 44, issue 6, pp. 3552–3558, 1991. https://doi.org/10.1103/PhysRevA.44.3552.
A. Di Ieva, F. Grizzi, H. Jelinek, A. J. Pellionisz, and G. A. Losa, “Fractals in the neurosciences, part I: General principles and basic neurosciences,” The Neuroscientist, vol. 20, issue 4, pp. 403–417, 2014. https://doi.org/10.1177/1073858413513927.
N. Bird and E. Perrier, “Multiscale percolation properties of a fractal pore network,” Geoderma, vol. 160, issue 1, pp. 105–110, 2010. https://doi.org/10.1016/j.geoderma.2009.10.009.
J. W. Baish and R. K. Jain, “Fractals and cancer,” Cancer Res, vol. 60, issue 14, pp. 3683–3688, 2000.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, pp. 1097–1105, 2012.
L. Nanni, S. Ghidoni, and S. Brahnam, “Handcrafted vs. Non handcrafted features for computer vision classification,” Pattern Recognition, vol. 71, pp. 158–172, 2017. https://doi.org/10.1016/j.patcog.2017.05.025.
A. Zadeh, M. Chen, S. Poria, E. Cambria, and L. Morency, “Tensor fusion network for multimodal sentiment analysis,” Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, (EMNLP’2017), pp. 1103–1114, Copenhagen, Denmark, September 7–11, 2017, Association for Computational Linguistics. https://doi.org/10.18653/v1/D17-1115.
S. Narkhede, “Understanding AUC-ROC curve,” Towards Data Science, vol. 26, issue 1, pp. 220-227, 2018.
G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, “Densely connected convolutional networks,” Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21-26 July 2017, pp. 2261-2269. https://doi.org/10.1109/CVPR.2017.243.
R. Kosarevych, O. Lutsyk, B. Rusyn, et al., “Adaptive patch reshaping for edge-based semantic segmentation in remote sensing,” IEEE Access, vol. 14, pp. 38951–38964, 2026, https://doi.org/10.1109/ACCESS.2026.3666289.
Downloads
Published
How to Cite
Issue
Section
License
International Journal of Computing is an open access journal. Authors who publish with this journal agree to the following terms:• Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
• Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
• Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.