Manara - Qatar Research Repository
Browse

Defense against adversarial attacks: robust and efficient compressed optimized neural networks

Download (4.53 MB)
journal contribution
submitted on 2024-07-24, 06:49 and posted on 2024-07-24, 08:25 authored by Insaf Kraidia, Afifa Ghenai, Samir Brahim Belhaouari

In the ongoing battle against adversarial attacks, adopting a suitable strategy to enhance model efficiency, bolster resistance to adversarial threats, and ensure practical deployment is crucial. To achieve this goal, a novel four-component methodology is introduced. First, introducing a pioneering batch-cumulative approach, the exponential particle swarm optimization (ExPSO) algorithm was developed for meticulous parameter fine-tuning within each batch. A cumulative updating loss function was employed for overall optimization, demonstrating remarkable superiority over traditional optimization techniques. Second, weight compression is applied to streamline the deep neural network (DNN) parameters, boosting the storage efficiency and accelerating inference. It also introduces complexity to deter potential attackers, enhancing model accuracy in adversarial settings. This study compresses the generative pre-trained transformer (GPT) by 65%, saving time and memory without causing performance loss. Compared to state-of-the-art methods, the proposed method achieves the lowest perplexity (14.28), the highest accuracy (93.72%), and an 8 × speedup in the central processing unit. The integration of the preceding two components involves the simultaneous training of multiple versions of the compressed GPT. This training occurs across various compression rates and different segments of a dataset and is ultimately associated with a novel multi-expert architecture. This enhancement significantly fortifies the model's resistance to adversarial attacks by introducing complexity into attackers' attempts to anticipate the model's prediction integration process. Consequently, this leads to a remarkable average performance improvement of 25% across 14 different attack scenarios and various datasets, surpassing the capabilities of current state-of-the-art methods.

Other Information

Published in: Scientific Reports
License: https://creativecommons.org/licenses/by/4.0
See article on publisher's website: https://dx.doi.org/10.1038/s41598-024-56259-z

Funding

Open Access funding provided by the Qatar National Library.

History

Language

  • English

Publisher

Springer Nature

Publication Year

  • 2024

License statement

This Item is licensed under the Creative Commons Attribution 4.0 International License.

Institution affiliated with

  • Hamad Bin Khalifa University
  • College of Science and Engineering - HBKU

Related Publications

Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017). Automated Hate Speech Detection and the Problem of Offensive Language. Proceedings of the International AAAI Conference on Web and Social Media, 11(1), 512–515. https://doi.org/10.1609/icwsm.v11i1.14955 Ankesh Anand, Tanmoy Chakraborty, Noseong Park (2019). We used Neural Networks to Detect Clickbaits: You won't believe what happened Next!. arXiv Repository. https://arxiv.org/abs/1612.01340

Related Datasets

Ankesh Anand. (2024). deep-clickbait-detection. Last modified 2018. GitHub Repository. https://github.com/ankeshanand/deep-clickbait-detection Tom Davidson. (2024). hate-speech-and-offensive-language. Last modified 2023. GitHub Repository. https://github.com/t-davidson/hate-speech-and-offensive-language Shekhar Gulati. (2024). sentiment-analysis-python. Last modified 2014. GitHub Repository. https://https://github.com/shekhargulati/sentiment-analysis-python/tree/master/polarity-data/rt-polaritydata

Usage metrics

    College of Science and Engineering - HBKU

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC