regarding /examples/ppi.py

[Udit-records]

Hello,
I'm new to GNN and currently interested in learning GAT for multilabel classification. Out of many source codes, I found an example of GAT on the PPI dataset, /examples/ppi.py, which has a different architecture than I found mostly. E.g. while defining the forward propagation, why torch.nn.linear is used and later added to GAT inside elu()?

       ....
       self.conv1 = GATConv(train_dataset.num_features, 256, heads=4)
       self.lin1 = torch.nn.Linear(train_dataset.num_features, 4 * 256)
       ....
       ....
       def forward(self, x, edge_index):
         x = F.elu(self.conv1(x, edge_index) + self.lin1(x))
     ....

(The link is: https://github.com/pyg-team/pytorch_geometric/blob/master/examples/ppi.py)

Please elaborate on how this GAT structure and functioning differ from typical GAT applications.
Could you share the relevant reference/documentation, if available?

[rusty1s]

Since GATConv does not utilize a skip-connection by default, we add it externally here via conv(x, edge_index) + lin(x). This helps the model to preserve central node information. This is further discussed in the paper (Section 3.3):

We have, however, successfully employed skip connections (He et al., 2016) across the intermediate attentional layer.

[Linnore]

Will PyG team consider adding the option of skip-connection for GATConv and other convolutional layers? If enable, a linear projection will be applied from dimension in_channels to out_channels (if not equals) on the input features x, then proceed to codes of adding bias.

[rusty1s]

We definitely can. Let me know if you have interest in contributing this.

[Linnore]

Sure! I can work on this, at least for GATConv and GATv2Conv. DGL already implemented the residual connection for these GAT layers.

I am not familiar with all GNN layers in PyG, so I am not sure which other layers need the usage of residual connections. Other layers may have different ways to strengthen the central information, e.g. the improved GCNConv.

[rusty1s]

We can just add it to GAT and GATv2 since this is explicitly mentioned in the paper.

[Linnore]

Sorry for the delay! I am new to PyG contribution and setting up the (development) installation on my new Mac is unexpectedly messy, especially for the plug-ins like pyg-lib T_T...

I created a PR for this discussion, please check #9515 . Thanks!