GNNs Revolutionize Malware Detection with Explainability

In the ever-evolving landscape of cybersecurity, malware continues to pose a critical threat, with attackers deploying increasingly sophisticated techniques that outpace traditional signature-based defenses. Graph Neural Networks (GNNs) have emerged as a powerful tool for detecting malware by modeling program behaviors through graph structures like control flow graphs (CFGs), which capture intricate execution paths and dependencies. However, the adoption of GNNs in security settings has been hampered by significant s, including the computational burden of large graphs, the black-box nature of models that limits trust, and a scarcity of reliable datasets for reproducible research. This portfolio of six interconnected studies, led by researchers at the University of New Brunswick, addresses these issues head-on, proposing innovative solutions that enhance efficiency, interpretability, and data availability in GNN-based malware detection. By integrating graph reduction, explainability frameworks, and curated datasets, the work paves the way for more scalable and transparent cybersecurity systems that can keep up with modern threats.

To build a solid foundation, the portfolio begins with a comprehensive survey that maps the current state of graph-based malware detection and explainability, highlighting the shift from feature-based s to graph learning approaches. This survey systematically reviews malware datasets, analysis techniques such as static, dynamic, and hybrid s, and feature engineering strategies, emphasizing the importance of graph-aware data for reproducibility. It delves into graph reduction techniques like sparsification, condensation, and coarsening, which simplify complex program graphs while retaining critical structural information, and explores embedding s that transform graph data into lower-dimensional representations suitable for GNN training. Additionally, the survey categorizes explainability into intrinsic and post-hoc s, underscoring the need for transparency in security applications where analyst trust is paramount. This foundational work not only identifies open s but also sets the stage for the portfolio's subsequent contributions, illustrating how datasets, reduction, embeddings, and explainability interlink to form a cohesive research roadmap.

Addressing the scalability issue, the portfolio introduces novel graph reduction s designed to handle the size and complexity of program graphs, which often contain thousands of nodes and edges with redundant or noisy components. A key innovation is Node-Centric Pruning (NCP), a sparsification technique that categorizes nodes into Nexus, Connector, and Sparse types based on connectivity analyses, pruning peripheral nodes to reduce graph size without sacrificing classification accuracy. Experiments showed that NCP outperforms state-of-the-art s like Walk Index Sparsification, delivering significant computational savings and stable performance across malware datasets. Complementing this, an integrated framework combines multiple pruning strategies—such as Leaf Prune and k-core decomposition—with learning and explainability modules, applying tools like GNNExplainer to extract influential subgraphs post-reduction. This approach not only cuts training and inference costs but also enhances interpretability by focusing analysts on concise, meaningful program structures, demonstrating that efficiency and transparency can be achieved simultaneously in malware detection pipelines.

On the explainability front, the portfolio makes strides in ensuring that GNN outputs are reliable and actionable for cybersecurity practitioners. One study evaluates the consistency of explanations using a dynamic malware detection framework that embeds node features from assembly instructions into compact 64-dimensional vectors via autoencoders, providing rich input for GNNs. It tests explainers like GNNExplainer, PGExplainer, and CaptumExplainer, introducing the RankFusion to aggregate edge rankings and the Greedy Edge-wise Composition algorithm to build coherent explanatory subgraphs, resulting in improved stability and fidelity under perturbations. Another contribution is a dual explanation framework that pairs standard GNN explainers with a prototype-driven layer, where verified malicious or benign subgraphs are stored and matched against target CFGs using algorithms like VF2 for subgraph alignment. This dual approach not only highlights critical regions in program graphs but also grounds explanations in known behavioral patterns, making model decisions more interpretable and trustworthy in high-stakes security environments.

Of this research are profound for both academia and industry, as it bridges gaps in AI-driven cybersecurity by making GNN-based detection more practical and dependable. The ensemble learning framework, which combines diverse GNNs like GCN, GIN, and GAT with attention-guided meta-learners, boosts detection accuracy and offers model-level interpretability by showing which architectures contribute most to decisions. Moreover, the release of curated datasets—CIC-SGG-2024 and CIC-DGG-2025—provides essential resources for reproducibility, including static and dynamic CFGs and FCGs with embedded features and explanations, supporting future research in graph-based malware analysis. However, limitations persist, such as the reliance on specific binary samples that may not cover all malware variants and the computational demands of dynamic graph generation, which the authors acknowledge could hinder broader adoption. Future work should explore adversarial robustness and expand dataset diversity to ensure these s remain effective against evolving threats, solidifying the portfolio's role in advancing trustworthy AI for cybersecurity.