Vibrational Inequality Based Framework for Stability and Generalization in Graph Neural Networks
Abstract
Graph Neural Networks (GNNs) have transformed graph learning, and Graph Convolutional Networks (GCNs) set new state of the art performance in node classification. GNNs can however be very unstable to train, particularly in the case of noisy graph topologies or scarce labelled data, which may delay convergence and generalisation. This paper introduces a new regularization theory in the framework of the theories of Variational Inequality (VI): the theory is based on introducing a regularization term in GNN training that may be formulated directly in response to the stability issues of GNNs. In contrast to simply training GCNs, our method complements the conventional training process, and adds a regularizer term based on the VI, which constrains the direction in which parameters are updated whereas it does not modify the model architecture, thus optimizing smoother optimization paths. On Cora citation network, we empirically analyze the suggested VI GCN model. The experimental evidence shows that VI GCN performs better than the default GCN baseline at being more stable and more accurate. Key metrics of the performance, such as the loss routes in training, the test accuracy, the dynamics of the norms on the weights, and histograms, show that VI regularization contributes to a smoother convergence as well as the better generalization. This epoch versus epoch power loss comparison provides additional support to such increase in stability being brought by our scheme. This paper not only emphasizes the effectiveness of adding optimization level regularization to GNN training but also opens up the research path in handling the issue of graph based learning instability. The suggested VI based approach is model independent, simple to deploy, and computationally efficient, which is why it can be applied to diverse graph neural network architectures and real life graph learning problems.
https://doi.org/10.5281/zenodo.17120321
