Hybrid Deep Learning Models for Text Classification: Performance Evaluation of TriDistilBERT and BiGRU Architectures

Text can be a valuable source of information, but its unstructured nature makes analysis challenging and time-consuming. Machine Learning (ML) algorithms can efficiently analyze and structure text, enabling organizations to automate processes and uncover insights that support better decision-making. This study focuses on applying ML to a classification problem using two datasets. Four deep learning models are introduced, combining Bi and Tri-layer hybrids of BERT and DistilBERT with a Bidirectional Gated Recurrent Unit (BiGRU) algorithm. These methods aim to enhance accuracy while examining the impact of hybridizing BERT and DistilBERT layers with BiGRU. The proposed models were evaluated against standalone BERT and DistilBERT approaches. Among them, the TriDistilBERT with BiGRU architecture achieved the highest accuracy, delivering 91.6% for the WASSA-17 dataset and 99.6% for the BBC dataset.