OGB-LSC Leaderboards
Steps to submit to leaderboards
- Carefully read the rules.
- Develop models and save test-dev prediction using the OGB Evaluator.
- Submit your result via this page.
- Wait at least one week until your test result shows up on our leaderboard. Please contact us if this is not the case.
Leaderboards policies
- For each email address on each dataset, another leaderboard submission cannot be made within one week after the last submission. Our system will automatically reject such submissions. Please do not use multiple email addresses within the same team.
- Extra information (hardware information, training/inference time, validation performance, technical report etc) is required for the OGB-LSC leaderboard submissions. Please check the submission page early to understand what is required.
- Please do not submit a placeholder Github repository / technical report. You will be permanently disqualified if we find your submission is based on the placeholder.
Check out leaderboards
Package denotes the required package version for each dataset to be eligible for the leaderboards.
Leaderboard for MAG240M
Classification accuracy on the test-dev and validation sets. The higher, the better.
Package: >=1.3.2
| Rank | Method | Ensemble | Test-dev Accuracy | Validation Accuracy | Team | Contact | References | #Params | Hardware | Date |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | R-UNIMP (KDDCup’21 1st) | Yes | 0.7539 | 0.7773 | BD-PGL | Yunsheng Shi (Baidu) | Paper, Code | N/A | V100 (32GB) | Mar 9, 2022 |
| 2 | MDGNN (OGB-LSC’22 3rd) | Yes | 0.7535 | 0.7756 | CogDL | Yukuo Cen (Tsinghua / Zhipu AI) | Paper, Code | 1,297,305 | 1 NVIDIA A100 GPU (80GB memory) | Nov 21, 2022 |
| 3 | MPNN+BGRL (KDDCup’21 2nd) | Yes | 0.7507 | 0.7710 | Academic | Petar Velickovic (DeepMind) | Paper, Code | N/A | 4 Google Cloud TPUv4 | Mar 9, 2022 |
| 4 | Cleora+EMDE (KDDCup’21 3rd) | Yes | 0.7457 | N/A | Synerise AI | Jacek Dabrowski (Synerise) | Paper, Code | N/A | Tesla V100 (16GB) | Mar 9, 2022 |
| 5 | MPLP+finetune (KDDCup’21 4th) | Yes | 0.7440 | N/A | Topology_mag | Wencai Cao (OPPO Research) | Paper, Code | N/A | V100 GPU (32GB memory) | Mar 9, 2022 |
| 6 | SGC+R-GAT+Finetune (KDDCup’21 5th) | Yes | 0.7377 | 0.7420 | passages | Kaiyuan Li (Nanjing University / BUPT) | Paper, Code | N/A | GeForce RTX 3090 GPU (24GB) | Mar 9, 2022 |
| 7 | GNN180M (KDDCup’21 6th) | Yes | 0.7340 | N/A | DeeperBiggerBetter | Guohao Li (KAUST / Intel) | Paper, Code | N/A | 4 NVIDIA GeForce RTX 6000 (48G) | Mar 9, 2022 |
| 8 | R-GAT (NS) | No | 0.6931 | 0.7002 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 12,255,385 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 9 | R-GraphSAGE (NS) | No | 0.6878 | 0.6986 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 12,234,905 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 10 | GAT (NS) | No | 0.6671 | 0.6715 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 4,890,777 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 11 | GraphSAGE (NS) | No | 0.6621 | 0.6679 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 4,884,633 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 12 | MLP+C&S | No | 0.6605 | 0.6698 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 473,241 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 13 | SIGN | No | 0.6603 | 0.6664 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 3,758,233 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 14 | SGC | No | 0.6530 | 0.6582 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 736,921 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 15 | LabelProp | No | 0.5638 | 0.5844 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 0 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 16 | MLP | No | 0.5276 | 0.5267 | OGB-LSC | Matthias Fey (TU Dortmund) | Paper, Code | 473,241 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
Leaderboard for WikiKG90Mv2
MRR on the test-dev and validation sets. The higher, the better.
Package: >=1.3.3
| Rank | Method | Ensemble | Test-dev MRR | Validation MRR | Team | Contact | References | #Params | Hardware | Date |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | BESS + diverse KGE ensemble | Yes | 0.2539 | 0.2919 | Graphcore | Daniel Justus (Graphcore) | Paper, Code | 23,356,393,344 | Graphcore Bow Pod16 | Dec 1, 2022 |
| 2 | DGLKE+RuleMining+ManualFeature | Yes | 0.2493 | 0.2916 | DNAKG | Xiaojun Ma (MSRA) | Paper, Code | 18,247,074,200 | 2x TitanV,800GB memory | Nov 20, 2022 |
| 3 | PIE-RM | Yes | 0.2115 | 0.2540 | USTCWiki | HongChao Gu (USTC / iflytek) | Paper, Code | 18,247,074,200 | 4 Telsla M40 | Oct 26, 2022 |
| 4 | TransE-ens | Yes | 0.1903 | 0.2931 | CogDL | Xiao Liu (Tsinghua / Zhipu.AI) | Paper, Code | 18,247,074,200 | 1 A100 | Oct 31, 2022 |
| 5 | TransE-Shallow-PIE | No | 0.1883 | 0.2342 | Ant Group KG&NLP | Linlin Chao (Ant Group) | Paper, Code | 18,247,074,200 | Tesla P100 (16GB GPU) | May 2, 2022 |
| 6 | TransE-Concat | No | 0.1761 | 0.2060 | OGB-LSC | Hongyu Ren (Stanford) | Paper, Code | 18,246,707,000 | 1 Quadro RTX 8000 (45GB GPU) | Oct 12, 2021 |
| 6 | ComplEx-Concat | No | 0.1761 | 0.2048 | OGB-LSC | Hongyu Ren (Stanford) | Paper, Code | 18,246,707,000 | 1 Quadro RTX 8000 (45GB GPU) | Oct 12, 2021 |
| 7 | ComplEx | No | 0.1412 | 0.1816 | GraphLARS | Ling Yue (EE, Tsinghua / 4paradigm) | Paper, Code | 18,246,707,000 | 1*A100 | Sep 1, 2022 |
| 8 | ComplEx-MPNet | No | 0.0988 | 0.1258 | OGB-LSC | Hongyu Ren (Stanford) | Paper, Code | 307,600 | 1 Quadro RTX 8000 (45GB GPU) | Oct 12, 2021 |
| 9 | ComplEx-Shallow | No | 0.0985 | 0.1150 | OGB-LSC | Hongyu Ren (Stanford) | Paper, Code | 18,246,399,400 | 1 Quadro RTX 8000 (45GB GPU) | Oct 12, 2021 |
| 10 | TransE-MPNet | No | 0.0860 | 0.1128 | OGB-LSC | Hongyu Ren (Stanford) | Paper, Code | 307,600 | 1 Quadro RTX 8000 (45GB GPU) | Oct 12, 2021 |
| 11 | TransE-Shallow | No | 0.0824 | 0.1103 | OGB-LSC | Hongyu Ren (Stanford) | Paper, Code | 18,246,399,400 | 1 Quadro RTX 8000 (45GB GPU) | Oct 12, 2021 |
Leaderboard for PCQM4Mv2
MAE on the test-dev and validation sets. The lower, the better.
Package: >=1.3.2
| Rank | Method | Ensemble | Test-dev MAE | Validation MAE | Team | Contact | References | #Params | Hardware | Date |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | EGT+Tri. Attn.+RDKit Coords. | No | 0.0683 | 0.0671 | EGT-AIRC | Md Shamim Hussain (RPI / IBM) | Paper, Code | 203,945,093 | 32 NVIDIA V100 (32 GB) | Nov 16, 2023 |
| 2 | EGT+Tri. Attn. (Pure Neural) | No | 0.0698 | 0.0686 | EGT-AIRC | Md Shamim Hussain (RPI / IBM) | Paper, Code | 203,894,787 | 32 NVIDIA V100 (32 GB) | Nov 23, 2023 |
| 3 | Uni-Mol+ (use 3D) | No | 0.0705 | 0.0693 | DP Technology | Guolin Ke (DP Technology) | Paper, Code | 77,025,812 | 8 NVIDIA A100 GPUs | Jul 6, 2023 |
| 4 | Uni-Mol+ base (use 3D) | No | 0.0708 | 0.0696 | DP Technology | Guolin Ke (DP Technology) | Paper, Code | 52,374,260 | 8 NVIDIA A100 GPUs | Jul 14, 2023 |
| 5 | GPS++ | Yes | 0.0720 | 0.0778 | GraphcoreValenceMILA | Dominic Masters (Graphcore/Valence/MILA) | Paper, Code | 44,291,413 | Graphcore BOW-POD16 | Nov 18, 2022 |
| 6 | MolNet_Ensemble | Yes | 0.0753 | 0.0797 | polixir.ai | zouxiaochuan (polixir.ai) | Paper, Code | 32,047,874 | 8 RTX3090 | Nov 1, 2022 |
| 7 | Global-ViSNet | No | 0.0766 | 0.0784 | ViSNet | Tong Wang (Microsoft Research AI4Science) | Paper, Code | 78,450,692 | 4 NVIDIA A100 GPUs | Oct 26, 2022 |
| 8 | Transformer-M | No | 0.0782 | 0.0772 | FML Lab@PKU | Shengjie Luo (Peking University) | Paper, Code | 68,957,249 | 4 NVIDIA Tesla A100 GPUs (40GB) | Oct 5, 2022 |
| 9 | GraphGPT(MLM tv0) | No | 0.0802 | 0.0800 | Alibaba-DT | Qifang Zhao (Alibaba) | Paper, Code | 453,388,801 | 4 Nvidia L40S | Aug 29, 2024 |
| 10 | GraphGPT(MLM tv10k) | No | 0.0804 | 0.0840 | Alibaba-DT | Qifang Zhao (Alibaba) | Paper, Code | 453,388,801 | 4 Nvidia L40S | Aug 21, 2024 |
| 11 | GEM-2 | No | 0.0806 | 0.0793 | PaddleHelix | Donglong He (Baidu) | Paper, Code | 32,086,707 | 16 NVIDIA A100 | Aug 11, 2022 |
| 12 | GPTrans-L | No | 0.0821 | 0.0809 | IMAGINE@NJU | Zhe Chen (Nanjing University) | Paper, Code | 85,995,713 | 8 NVIDIA A100 GPUs | Jun 6, 2023 |
| 13 | GPTrans-T | No | 0.0842 | 0.0833 | IMAGINE@NJU | Zhe Chen (Nanjing University) | Paper, Code | 6,577,697 | 8 NVIDIA A100 GPUs | Jun 14, 2023 |
| 14 | Deep graph transformer | Yes | 0.0843 | 0.0891 | NVIDIA-PCQM4Mv2 | Jiwei Liu (NVIDIA) | Paper, Code | 63,600,000 | 1 NVIDIA V100 GPU 32 GB | Oct 26, 2022 |
| 15 | Deep graph transformer | Yes | 0.0844 | 0.0891 | NVIDIA-PCQM4Mv2 | Jiwei Liu (NVIDIA) | Paper, Code | 63,600,000 | 1 NVIDIA V100 GPU 32 GB | Oct 19, 2022 |
| 16 | Deep graph transformer | Yes | 0.0852 | 0.0891 | NVIDIA-PCQM4Mv2 | Jiwei Liu (NVIDIA) | Paper, Code | 63,600,000 | 1 NVIDIA V100 GPU 32 GB | Oct 11, 2022 |
| 17 | GraphGPT(MLM pretrained) | No | 0.0856 | 0.0847 | Alibaba-DT | Qifang Zhao (Alibaba) | Paper, Code | 453,388,801 | 4 Nvidia L40S | Aug 11, 2024 |
| 18 | EGT | No | 0.0862 | 0.0857 | EFT-AIRC | Md Shamim Hussain (RPI / IBM) | Paper, Code | 89,326,465 | 8 Tesla V100 (32GB) | Jun 24, 2022 |
| 18 | GPS | No | 0.0862 | 0.0852 | Mila | Ladislav Rampasek (Mila / Universite de Montreal) | Paper, Code | 13,807,345 | 1 NVIDIA A100 (40GB) | Nov 10, 2022 |
| 19 | CoAtGIN-base | No | 0.0866 | 0.0859 | xfcui@sdu | Xuefeng Cui (Shandong University) | Paper, Code | 9,938,433 | 8 NVIDIA A40 | Sep 30, 2022 |
| 20 | EGT+LSPE+HIERA_clustering | No | 0.0868 | 0.0863 | rwlspegate2 | YEOM JE YOON (University of Seoul) | Paper, Code | 90,102,913 | NVIDIA A100 80GB HBM2E SXM4 | Feb 16, 2023 |
| 21 | EGT | No | 0.0872 | 0.0869 | EGT-AIRC | Md Shamim Hussain (RPI / IBM) | Paper, Code | 89,326,465 | 8 Tesla V100 (32GB) | Jan 26, 2022 |
| 22 | GRPE-Large | No | 0.0876 | 0.0867 | WonWoo | Wonpyo Park (SNU / Standigm) | Paper, Code | 118,300,000 | 8 A100-SXM4-40GB | Aug 14, 2022 |
| 23 | GraphSelfAttention | No | 0.0898 | 0.0890 | WonWoo | Wonpyo Park (Standigm) | Paper, Code | 46,199,041 | 4 A100 GPU | Nov 15, 2021 |
| 24 | Deep graph transformer | Yes | 0.0904 | 0.0891 | NVIDIA-PCQM4Mv2 | Jiwei Liu (NVIDIA) | Paper, Code | 63,600,000 | 1 NVIDIA V100 GPU 32 GB | Oct 3, 2022 |
| 25 | CoAtGIN-tiny | No | 0.0908 | 0.0901 | xfcui@sdu | Xuefeng Cui (Shandong University) | Paper, Code | 6,395,393 | 1 NVIDIA A40 | Sep 20, 2022 |
| 26 | TokenGT (Lap) | No | 0.0919 | 0.0910 | vl-kaist | Jinwoo Kim (KAIST) | Paper, Code | 48,492,289 | NVIDIA RTX 3090 x 8 | Aug 8, 2022 |
| 27 | CoAtGIN-tiny | No | 0.0921 | 0.0916 | xfcui@sdu | Xuefeng Cui (Shandong University) | Paper, Code | 6,183,425 | 1 NVIDIA A40 | Sep 8, 2022 |
| 28 | CoAtGIN-tiny | No | 0.0935 | 0.0933 | xfcui@sdu | Xuefeng Cui (Shandong University) | Paper, Code | 5,460,225 | NVIDIA A40 | Aug 29, 2022 |
| 29 | HFAGNN | No | 0.1010 | 0.1005 | Zhuque Zhejianglab | Can Xu (Zhejiang Lab) | Paper, Code | 3,949,959 | 1 NVIDIA TITAN V GPU(12GB memory) | Oct 30, 2022 |
| 30 | GIN-virtual | No | 0.1084 | 0.1083 | OGB-LSC | Weihua Hu (Stanford) | Paper, Code | 6,656,406 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 31 | GCN-virtual | No | 0.1152 | 0.1153 | OGB-LSC | Weihua Hu (Stanford) | Paper, Code | 4,850,401 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 32 | GIN | No | 0.1218 | 0.1195 | OGB-LSC | Weihua Hu (Stanford) | Paper, Code | 3,761,406 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 33 | GraphGPS without PE (subset) | No | 0.1395 | 0.1334 | GraphLARS | Xu Wang (4Paradigm && EE Tsinghua) | Paper, Code | 6,155,089 | 1 RTX3090 (24GB GPU) | Sep 2, 2022 |
| 34 | GCN | No | 0.1398 | 0.1379 | OGB-LSC | Weihua Hu (Stanford) | Paper, Code | 1,955,401 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 35 | MLP-Fingerprint | No | 0.1760 | 0.1753 | OGB-LSC | Weihua Hu (Stanford) | Paper, Code | 16,107,201 | 1 GeForce RTX 2080 (11GB GPU) | Sep 8, 2021 |
| 36 | MolNet_Ensemble | Yes | 0.9899 | 0.0847 | polixir.ai | xiaochuan zou (polixir.ai) | Paper, Code | 32,047,874 | 8 RTX3090 | Oct 24, 2022 |