Journal Home Online First Current Issue Archive For Authors Journal Information 中文版

Frontiers of Information Technology & Electronic Engineering >> 2022, Volume 23, Issue 3 doi: 10.1631/FITEE.2000657

NGAT: attention in breadth and depth exploration for semi-supervised graph representation learning

浙江大学计算机科学与技术学院,中国杭州市,310027

Received: 2020-11-22 Accepted: 2022-03-22 Available online: 2022-03-22

Next Previous

Abstract

Recently, graph neural networks (GNNs) have achieved remarkable performance in representation learning on graph-structured data. However, as the number of network layers increases, GNNs based on the neighborhood aggregation strategy deteriorate due to the problem of oversmoothing, which is the major bottleneck for applying GNNs to real-world graphs. Many efforts have been made to improve the process of feature information aggregation from directly connected nodes, i.e., breadth exploration. However, these models perform the best only in the case of three or fewer layers, and the performance drops rapidly for deep layers. To alleviate oversmoothing, we propose a nested graph network (NGAT), which can work in a semi-supervised manner. In addition to breadth exploration, a -layer NGAT uses a layer-wise aggregation strategy guided by the mechanism to selectively leverage feature information from the -order neighborhood, i.e., depth exploration. Even with a 10-layer or deeper architecture, NGAT can balance the need for preserving the locality (including root node features and the local structure) and aggregating the information from a large neighborhood. In a number of experiments on standard tasks, NGAT outperforms other novel models and achieves state-of-the-art performance.

Related Research