Explainable Histology Images Classification Using an Ensemble of Fractal Geometry, CNNs, and ViT

Authors

  • Moheb R. Girgis
  • Al Hussien Seddik Saad
  • Mohammed M. Talaat

DOI:

https://doi.org/10.47839/ijc.25.3.4843

Keywords:

Histology Image Classification, Fractal Geometry, Convolutional Neural Networks, Ensemble Learning, Deep Learning, Vision Transformers, Grad-CAM Visualization

Abstract

Accurately classifying histopathological images is still a fundamental but difficult task in medical diagnostics, because biological tissues are irregular, self-similar, and multi-scale. While effective in tasks involving natural images, convolutional neural networks (CNNs) show limited generalization when confronted with highly heterogeneous tissue structures, small datasets, and variable resolutions. In this paper, we propose a tri-branch ensemble model, called Fractal-CNN-ViT, for histology images classification, which combines the advantages of Vision Transformers (ViTs), CNNs, and fractal geometry. In this model, fractal descriptors—specifically fractal dimension, lacunarity, and percolation—are extracted from histology images, transformed into artificial RGB images, and then processed by a specialized CNN (Fractal-CNN). In order to capture global contextual relationships, a second CNN (ResNet-50) and a ViT simultaneously analyze the original images. The outputs of the three branches are fused through an adaptive weighted ensemble module, which maps the optimal contribution of each stream to the final decision. Experimental results on the BreakHis dataset demonstrated that Fractal-CNN-ViT outperforms standalone deep learning architectures: Fractal-CNN, CNN (ResNet-50) and ViT. It achieved an accuracy of 92.3%, surpassing Fractal-CNN (85.9%), ResNet-50 (87.2%) and ViT (88.5%), confirming the benefits of integrating fractal-based features with deep learning methods. These findings demonstrate how well the proposed ensemble model combines local and global tissue representations, reduces overfitting on small datasets, and generalizes across magnification levels. Finally, Grad-CAM visualization was employed to provide visual interpretability, increasing clinical trust by highlighting image regions most influential to the studied models' classification decisions.

References

G. Litjens, T. Kooi, B. E. Bejnordi, et al., “A survey on deep learning in medical image analysis,” Medical Image Analysis, vol. 42, pp. 60-88, 2017. https://doi.org/10.1016/j.media.2017.07.005.

A. Janowczyk and A. Madabhushi, “Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases,” Journal of Pathology Informatics, vol. 7, issue 1, 2016. https://doi.org/10.4103/2153-3539.186902.

G. Aresta, T. Araújo, S. Kwok, et al., “BACH: Grand challenge on breast cancer histology images,” Medical Image Analysis, vol. 56, pp. 122–139, 2019. https://doi.org/10.1016/j.media.2019.05.010.

R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, “Convolutional neural networks: an overview and application in radiology,” Insights into Imaging, vol. 9, issue 4, pp. 611–629, 2018. https://doi.org/10.1007/s13244-018-0639-9.

F. A. Spanhol, L. S. Oliveira, C. Petitjean and L. Heutte, “A dataset for breast cancer histopathological image classification,” IEEE Transactions on Biomedical Engineering, vol. 63, issue 7, pp. 1455-1462, 2016. https://doi.org/10.1109/TBME.2015.2496264.

T. Araújo, G. Aresta, E. Castro, J. Rouco, P. Aguiar, C. Eloy, A. Polónia and A. Campilho, “Classification of breast cancer histology images using convolutional neural networks,” PLoS One, vol. 12, issue 6, 2017. https://doi.org/10.1371/journal.pone.0177544.

A. Madabhushi and G. Lee, “Image analysis and machine learning in digital pathology: Challenges and opportunities,” Medical Image Analysis, vol. 33, pp. 170-175, 2016. https://doi.org/10.1016/j.media.2016.06.037.

C. Wemmert, J. Weber, F. Feuerhake, and G. Forestier, “Deep learning for histopathological image analysis,” in Mourad Elloumi (Ed.), Deep Learning for Biomedical Data Analysis. Techniques, Approaches, and Applications, Springer, pp.153-169, 2021. https://doi.org/10.1007/978-3-030-71676-9_7.

B. B. Mandelbrot, The Fractal Geometry of Nature, third ed., New York, NY, USA: W. H. Freeman, 1983. https://doi.org/10.1119/1.13295.

G. F. Roberto, F. A. Faria, H. Bragança, A. A. Santos, J. P. Papa, and D. Menotti, “Fractal neural network: A new ensemble of fractal geometry and convolutional neural networks for the classification of histology images,” Expert Systems with Applications, vol. 166, p. 114103, 2021. https://doi.org/10.1016/j.eswa.2020.114103.

A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” International Conference on Learning Representations (ICLR), Vienna, Austria, May 04 2021.

S. Khan, M. Naseer, M. Hayat, et al., “Transformers in vision: A survey,” arXiv:2101.01169v1 [cs.CV] 4 Jan 2021. https://doi.org/10.1145/3505244.

R. R. Selvaraju, M. Cogswell, A. Das, et al., “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision (IJCV), vol. 128, pp. 336–359, 2020. https://doi.org/10.1007/s11263-019-01228-7.

G. F. Roberto, L. A. Neves, M. Z. Nascimento, T. A. Tosta, L. C. Longo, A. S. Martins, and P. R. Faria, “Features based on the percolation theory for quantification of non-Hodgkin lymphomas,” Computers in Biology and Medicine, vol. 91, pp. 135–147, 2017. https://doi.org/10.1016/j.compbiomed.2017.10.012.

M. Ivanovici, and N. Richard, “Fractal dimension of color fractal images,” IEEE Transactions on Image Processing, vol. 20, issue 1, pp. 227–235, 2011. https://doi.org/10.1109/TIP.2010.2059032.

N. Bayramoglu, J. Kannala, and J. Heikkilä, “Deep learning for magnification independent breast cancer histopathology image classification,” Proceedings of the 23rd International Conference on Pattern Recognition (ICPR), 04-08 December 2016, Cancun, Mexico, pp. 2440–2445. https://doi.org/10.1109/ICPR.2016.7900002.

M. Talo, “Automated classification of histopathology images using transfer learning,” Artificial Intelligence in Medicine, vol. 101, 101743, 2019. https://doi.org/10.1016/j.artmed.2019.101743.

R. E. Plotnick, R. H. Gardner, W. W. Hargrove, K. Prestegaard, and M. Perlmutter, “Lacunarity analysis: A general technique for the analysis of spatial patterns,” Physical Review E, vol. 53, issue 5, pp. 5461–5468, 1996. https://doi.org/10.1103/PhysRevE.53.5461.

C. Allain and M. Cloitre, “Characterizing the lacunarity of random and deterministic fractal sets,” Physical Review A, vol. 44, issue 6, pp. 3552–3558, 1991. https://doi.org/10.1103/PhysRevA.44.3552.

A. Di Ieva, F. Grizzi, H. Jelinek, A. J. Pellionisz, and G. A. Losa, “Fractals in the neurosciences, part I: General principles and basic neurosciences,” The Neuroscientist, vol. 20, issue 4, pp. 403–417, 2014. https://doi.org/10.1177/1073858413513927.

N. Bird and E. Perrier, “Multiscale percolation properties of a fractal pore network,” Geoderma, vol. 160, issue 1, pp. 105–110, 2010. https://doi.org/10.1016/j.geoderma.2009.10.009.

J. W. Baish and R. K. Jain, “Fractals and cancer,” Cancer Res, vol. 60, issue 14, pp. 3683–3688, 2000.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, pp. 1097–1105, 2012.

L. Nanni, S. Ghidoni, and S. Brahnam, “Handcrafted vs. Non handcrafted features for computer vision classification,” Pattern Recognition, vol. 71, pp. 158–172, 2017. https://doi.org/10.1016/j.patcog.2017.05.025.

A. Zadeh, M. Chen, S. Poria, E. Cambria, and L. Morency, “Tensor fusion network for multimodal sentiment analysis,” Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, (EMNLP’2017), pp. 1103–1114, Copenhagen, Denmark, September 7–11, 2017, Association for Computational Linguistics. https://doi.org/10.18653/v1/D17-1115.

S. Narkhede, “Understanding AUC-ROC curve,” Towards Data Science, vol. 26, issue 1, pp. 220-227, 2018.

G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, “Densely connected convolutional networks,” Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21-26 July 2017, pp. 2261-2269. https://doi.org/10.1109/CVPR.2017.243.

R. Kosarevych, O. Lutsyk, B. Rusyn, et al., “Adaptive patch reshaping for edge-based semantic segmentation in remote sensing,” IEEE Access, vol. 14, pp. 38951–38964, 2026, https://doi.org/10.1109/ACCESS.2026.3666289.

Downloads

Published

2026-09-30

How to Cite

Girgis, M. R., Saad, A. H. S., & Talaat, M. M. (2026). Explainable Histology Images Classification Using an Ensemble of Fractal Geometry, CNNs, and ViT. International Journal of Computing, 25(3), 475-487. https://doi.org/10.47839/ijc.25.3.4843

Issue

Section

Articles