Topology and Position Aware Graph Neural Networks

Abstract

In machine learning, real-world objects such as molecules and proteins are naturally formulated as graphs. Two types of information are vital for graph representation learning; those are, topology information and position information. Basically, topology information shows connections between nodes, and position information further determines the final conformation of real-world objects. With the advances of deep learning, graph neural networks (GNNs) are intensively studied for learning from graph data. However, the number of neighboring units is not fixed, thus the spatial locality is not well-defined in graphs. Hence, it is challenging to apply deep learning operations such as pooling to graphs with preserving topology information. In addition, regular GNNs fail to incorporate the important 3D position information in real-world objects, leading to incomplete data representations. To tackle these challenges, we propose topology and position aware graph neural networks to explicitly integrate the important topology and position information.

In the category of topology aware graph neural networks, we aim to investigate regular deep learning operations such as convolution and pooling on real-world graphs with preserving topology information. Generally, convolution operation is well-studied while pooling is a challenging yet less explored area. Due to its nature of lacking spatial locality, pooling on the graph usually fails to encode topology information. We propose a novel graph pooling method to explicitly consider graph topology. Our method is essentially a two-stage voting process, which is composed of local voting and global voting. By performing local voting, our method selects important nodes from neighborhood and works similarly as a locality-based max pooling operation on images and texts. In addition, we further incorporate graph topology in global voting to weigh each node globally in the entire graph. Thus, the final pooling is realized by selecting important nodes based on their local and global voting scores. Apparently, topology information is explicitly incorporated in each stage of the two-stage voting process. In addition to realizing pooling, we apply convolution on real-world data like proteins to extract topology information. Previous studies formulate protein as 4D grid-like data to make use of sequential order information, but the inherent topology information is neglected. We represent proteins as graphs and apply graph convolution operations to aggregate topology information. After that, sequence information is restored by reordering the feature matrix based on original amino acid sequences. Basically, our framework considers both the topology and sequence information, leading to more accurate data representations.

In the category of position aware graph neural networks, we aim at incorporating 3D position information in the network design. Generally, we formulate a real-word object as a 3D graph, which contains 3D positional coordinates for each node given in the Cartesian system along with the graph topology. Different types of relative 3D information can be derived from 3D graphs, and they can be vital in some applications, such as bond lengths and angles in molecular modeling. We first propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning. SMP represents an approximately complete learning architecture and is capable of distinguishing almost all molecular structures in practice. We derive physically-based representations of 3D information and propose the SphereNet for learning representations of 3D graphs. We show that existing 3D deep models can be viewed as special cases of the SphereNet. However, we further find that existing works as well as SphereNet only focus on local completeness and SphereNet is only complete in local neighborhood, failing to achieve global completeness for a whole 3D graph. In addition, existing methods exhibit excessive time complexity, severely preventing their scalability in real-world applications. To incorporate 3D information completely and efficiently, we further propose a novel message passing scheme that operates within 1-hop neighborhood. Our method guarantees full completeness of 3D information on 3D graphs by achieving global and local completeness. Notably, we propose the important rotation angles to fulfill global completeness. Additionally, we show that our method is orders of magnitude faster than prior methods. We provide rigorous proof of completeness and analysis of time complexity for our methods. As molecules are in essence quantum systems, we build the complete and efficient graph neural network (ComENet) by combing quantum inspired basis functions and the proposed message passing scheme.

Experimental results on graph classification, protein interface prediction, and molecular property prediction all demonstrate the power of our proposed methods.