Comments (6)
The memory consumption increases exponentially with hops. If hop=2 OOMs and you still want to use hop=2, you may keep decreasing max_nodes_per_hop to a smaller value, or further reduce the max-train-num.
from seal.
May I ask you @muhanzhang , what were the parameters of your computer setup when you were running SEAL with hops = 2 on the big datasets like (Arxiv, FB, Wikipedia, Blogpost) so that I know how much RAM I approximately need?
Seems like the building of the SEALDataset from TorchGeometric is the most demanding task. Even to build the PB dataset, it took me around 800 seconds to be done with hops = 1. I am using SEAL code from PTG Link Prediction demo.
Thanks,
Joe
from seal.
Pytorch Geometric default k-hop subgraph sampler is slow. I will release a new version (possibly merge to PyG) with much faster (~1000 times) speed up recently. Please stay tuned. I'll let you know when it is released.
from seal.
@muhanzhang That would be much appreciated! Even if you could share it somehow here as a gist in some beta version. I need it for my MSc thesis and so far I cannot run Arxiv and Facebook with PTG SEAL as it is really slow.
from seal.
Hi @joc32 , the new version of SEAL is here. It runs on OGB benchmark datasets.
from seal.
Thanks very much @muhanzhang , will look into it.
from seal.
Related Issues (20)
- System crash during subgraph extraction step HOT 2
- 是否可以拓展到有向图的比较? HOT 2
- about the row_data and .mat files HOT 1
- about subgraph extraction HOT 1
- sampling negative links HOT 2
- 关于模型预测值 HOT 2
- Training on really large graphs HOT 2
- Questions about the contents in txt and mat files. HOT 4
- Error when trying to run my own datasets HOT 6
- Scores besides acc and auc HOT 3
- Command line argument for running facebook dataset HOT 2
- __init__() got an unexpected keyword argument 'size' HOT 1
- Access the nodes and edges from the subgraph HOT 2
- About node labeling HOT 1
- Bug during trying on a large graph HOT 1
- Node with same index HOT 4
- OSError: [Winerror 193] %1 is not an effective win32 application. HOT 2
- NameError: name 'cmd_args' is not defined
- TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType' HOT 1
- How to understand learn automatically “heuristic”
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from seal.