palash1992 / gem Goto Github PK
View Code? Open in Web Editor NEWLicense: BSD 3-Clause "New" or "Revised" License
License: BSD 3-Clause "New" or "Revised" License
Dear authors,
I have an issue using your implementation of sdne algorithm. When I run embedding/sdne.py I get the following error:
Traceback (most recent call last):
File "embedding/sdne.py", line 260, in
embedding.learn_embedding(graph=G, edge_f=None, is_weighted=True, no_python=True)
File "embedding/sdne.py", line 151, in learn_embedding
self._model.compile(optimizer=sgd, loss=[weighted_mse_x, weighted_mse_x, weighted_mse_y], loss_weights=[1, 1, self._alpha])
File "/home/tilenmarc/.virtualenvs/virtual1/local/lib/python2.7/site-packages/keras/engine/training.py", line 910, in compile
sample_weight, mask)
File "/home/tilenmarc/.virtualenvs/virtual1/local/lib/python2.7/site-packages/keras/engine/training.py", line 436, in weighted
score_array = fn(y_true, y_pred)
File "embedding/sdne.py", line 145, in weighted_mse_y
return KBack.sum(KBack.square(y_pred), axis=-1).reshape([min_batch_size, 1]) * y_true
AttributeError: 'Tensor' object has no attribute 'reshape'
Have any idea how to avoid it?
Best regards,
Tilen
Hi,
I'm strongly appreciate your great project of network embedding, and the codes you proposed indeed help me in research.
However, I encounter problems in running the usage code in README.md. It occurs a ''UnboundLocalError'' when execute the ''embedding.learn_embedding'' method for SDNE model, and the detailed error messages are as follows.
Traceback (most recent call last):
File "/Users/Daniel/PycharmProjects/test/test/gem_test.py", line 45, in
Y, t = embedding.learn_embedding(graph=G, edge_f=None, is_weighted=True, no_python=True)
File "/Users/Daniel/anaconda/lib/python2.7/site-packages/gem-1.0.0-py2.7.egg/gem/embedding/sdne.py", line 160, in learn_embedding
verbose=1
File "/Users/Daniel/anaconda/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "/Users/Daniel/anaconda/lib/python2.7/site-packages/keras/engine/training.py", line 1828, in fit_generator
print(str(steps_done) + ' ' +str(steps_per_epoch))
UnboundLocalError: local variable 'steps_done' referenced before assignment
Looking forward to your reply. Thanks!
Could you please provide some instructions on how to use the Jamfile to compile the CPP part of your code? I really couldn't find much resource about this. I've been trying for days with my partner but still can't use the CPP code for Graph Factorization.... I saw your earlier post that suggested "bjam toolset=gcc cxxflags="-std=c++11" but got the error:
/usr/local/share/boost-build/src/build/project.jam:111: in load-parent from module project
error: Could not find parent for project at '..'
error: Did not find Jamfile.jam or Jamroot.jam in any parent directory.
/usr/local/share/boost-build/src/build/project.jam:449: in initialize from module project
/usr/local/share/boost-build/src/build/project.jam:301: in load-jamfile from module project
/usr/local/share/boost-build/src/build/project.jam:64: in load from module project
/usr/local/share/boost-build/src/build/project.jam:113: in load-parent from module project
/usr/local/share/boost-build/src/build/project.jam:449: in initialize from module project
/usr/local/share/boost-build/src/build/project.jam:301: in load-jamfile from module project
/usr/local/share/boost-build/src/build/project.jam:64: in load from module project
/usr/local/share/boost-build/src/build/project.jam:145: in project.find from module project
/usr/local/share/boost-build/src/build-system.jam:535: in load from module build-system
/usr/local/share/boost-build/src/kernel/modules.jam:289: in import from module modules
/usr/local/share/boost-build/src/kernel/bootstrap.jam:139: in boost-build from module
/usr/local/share/boost-build/boost-build.jam:8: in module scope from module
Any help is appreciated!
there is only KARATE dataset is available。
Continuing the thread from the JOSS review:
I think the path to karate.edgelist
needs to be specified in the test script. Now I get:
Using TensorFlow backend.
/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.7
return f(*args, **kwds)
Num nodes: 34, num edges: 77
Could not import C++ module for Graph Factorization. Reverting to python implementation. Please recompile graphFac_ext from graphFac.cpp using bjam
Iter id: 0, Objective: 77.0078, f1: 77.0002, f2: 0.00757339
Iter id: 10000, Objective: 76.9979, f1: 76.9957, f2: 0.00211602
Iter id: 20000, Objective: 76.9971, f1: 76.9949, f2: 0.00215114
Iter id: 30000, Objective: 76.9968, f1: 76.9946, f2: 0.00225228
Iter id: 40000, Objective: 76.9967, f1: 76.9944, f2: 0.00230059
graph_factor_sgd:
Training time: 33.805227
Traceback (most recent call last):
File "test.py", line 50, in <module>
MAP, prec_curv = gr.evaluateStaticGraphReconstruction(G, embedding, Y, None)
ValueError: too many values to unpack (expected 2)
Hello,
I tried to run the example given in the readme but get the following error:
Could not import C++ module for Graph Factorization. Reverting to python implementation. Please recompile graphFac_ext from graphFac.cpp using bjam
Traceback (most recent call last):
File "1.py", line 12, in
Y, t = em.learn_embedding(graph, edge_f=None, is_weighted=True, no_python=True)
File "/home/vinayak/Research/Graph/GEM/gem/embedding/gf.py", line 80, in learn_embedding
[f1, f2, f] = self._get_f_value(graph)
File "/home/vinayak/Research/Graph/GEM/gem/embedding/gf.py", line 46, in _get_f_value
for i, j, w in graph.edges_iter(data='weight', default=1):
AttributeError: 'DiGraph' object has no attribute 'edges_iter'
Hello.
When I try to run the default examples on the index page, I get the following error:
code:
GraphFactorization(2, 100000, 1*10**-4, 1.0)
This throws
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/GEM/gem/embedding/gf.py", line 44, in __init__
for key in dictionary:
TypeError: 'int' object is not iterable
Looking at the code, it seem the input is supposed to be a dictionary, what am I doing wrong?
Thank you!
def init(self, *hyper_dict, **kwargs):
''' Initialize the GraphFactorization class
Args:
d: dimension of the embedding
eta: learning rate of sgd
regu: regularization coefficient of magnitude of weights
max_iter: max iterations in sgd
print_step: #iterations to log the prgoress (step%print_step)
'''
hyper_params = {
'print_step': 10000,
'method_name': 'graph_factor_sgd'
}
hyper_params.update(kwargs)
for key in hyper_params.keys():
self.__setattr__('_%s' % key, hyper_params[key])
for dictionary in hyper_dict:
for key in dictionary:
self.__setattr__('_%s' % key, dictionary[key])
So, why for key in dictionary?
hyper_dict is just a tuple!
error: Unable to find file or target named
error: 'graphFacLoad.py'
error: referred to from project at
error: '.'
Traceback (most recent call last):
File "test_karate.py", line 58, in
MAP, prec_curv = lp.evaluateStaticLinkPrediction(G,embedding)
File "/home/public/anaconda3/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/evaluation/evaluate_link_prediction.py", line 47, in evaluateStaticLinkPrediction
X=graph_util.loadEmbedding('data/node_vectors1.txt')
File "/home/public/anaconda3/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/utils/graph_util.py", line 191, in loadEmbedding
X[int(emb[0]), :] = emb_fl
IndexError: index 469 is out of bounds for axis 0 with size 439
What do you think about porting GEM algorithms to AjguDB https://github.com/amirouche/AjguDB#ajgudb?
I have embeddings generated by deepwalk, which is similar to node2vec, but a lot faster. What format do I need to load the embeddings in as in order to run learn_embeddings?
I've tried the loadEmbeddings() function in graph_utils.py. It's returning the following error:
index 11023110 is out of bounds for axis 0 with size 11023110
The error occurs for this line in loadEmbeddings():
X[int(emb[0]), :] = emb_fl
Hi,
I am having trouble reproducing the result of blogcatalog using SDNE. It's similar to the issue mentioned here.
#35
would it be possible for you to share the hyperparameter for blogcatalog dataset?
Thanks a lot!
I compiled the graph factorization cpp files in c_src using bjam. For the compilation I made the following changes:
Post compilation two files were created - graphFac.o, graphFac_ext.so
I was unable to use it with your python gem interface. I am not sure whether this is the right way to compile and set it up . Can you please describe the steps for setting up graph factorization? I am hoping to speed up the learning process by using the code.
Aside issue : Hope implementation is very memory intense , How much memory did you need for running on YouTube graph in your experiment?
IOError: Unable to create file (Unable to open file: name = './intermediate/enc_weights.hdf5', errno = 2, error message = 'no such file or directory', flags = 13, o_flags = 242)
Hi,
Do you plan to convert the lib code to python 3?
Thanks
Hello,
What is expected run time for big datasets like YOUTUBE? Experimental setup described in the paper is very powerful, have you tried with average hardware?
Thanks.
Hi,
first: thanks for your great work! This library is really helpful!
Second: I have the following issue. My graph is extremely large, around 1.5 Mio nodes. Constructing the quadratic matrix in graph_util.py will of course fail, because memory is scarce.
def transform_DiGraph_to_adj(di_graph):
n = di_graph.number_of_nodes()
adj = np.zeros((n, n))
for st, ed, w in di_graph.edges(data='weight', default=1):
adj[st, ed] = w
return adj
Hence, SDNE fails.
Would it be possible to provide a sparse matrix solution of SDNE?
Cheers!
Hi Palash,
I think the loadEmbedding
is encountering some issues, though I train the graph smoothly. I anyway can load the tempGraph.emb
file but you may want to look at this traceback :)
Traceback (most recent call last): File "train.py", line 81, in <module> n2v_model = model.train_n2v_model(G, params) File "train.py", line 32, in train_n2v_model embedding.learn_embedding(g) File "/home/pdaixmor/.virtualenvs/word_embeddings/lib/python2.7/site-packages/gem-1.0.0-py2.7.egg/gem/embedding/node2vec.py", line 84, in learn_embedding self._X = graph_util.loadEmbedding('tempGraph.emb') File "/home/pdaixmor/.virtualenvs/word_embeddings/lib/python2.7/site-packages/gem-1.0.0-py2.7.egg/gem/utils/graph_util.py", line 170, in loadEmbedding X[int(emb[0]), :] = emb_fl IndexError: index 6256 is out of bounds for axis 0 with size 6224
I try to reproduce the SDNE visualization result in you Survey, but using different hyper-parameter value caused different result. Would you mind tell me the parameter value you set in SDNE visualization experiment?
Hey,
This is great repos!
I am wondering could you provide the optimal hyper-parameter for PPI network?
Best,
Shun
Hi,
It is just a query, not an issue as such.
Does the implementation of SDNE produce comparable results as given in the paper?
Hi,when I modified test.kerate.py to make link prediction, the following error occurred:
Traceback (most recent call last):
File "test_karate.py", line 54, in
MAP, prec_curv = lp.evaluateStaticLinkPrediction(G, embedding,0.8)
File "/home/public/anaconda3/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/evaluation/evaluate_link_prediction.py", line 24, in evaluateStaticLinkPrediction
is_undirected=is_undirected
File "/home/public/anaconda3/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/utils/evaluation_util.py", line 50, in splitDiGraphToTrainTest
train_digraph.remove_edge(ed, st)
File "/home/public/anaconda3/lib/python3.6/site-packages/networkx/classes/digraph.py", line 692, in remove_edge
raise NetworkXError("The edge %s-%s not in graph."%(u,v))
networkx.exception.NetworkXError: The edge 31-0 not in graph.
I modified it as follows:
MAP, prec_curv = lp.evaluateStaticLinkPrediction(G, embedding,0.8)
I would appreciate it if you could help me!
I am running the Example 1 from the main page. However, for any plot I get the error
module 'matplotlib.cbook' has no attribute 'is_string_like'
and I get
without any label and connections.
I have matplotlib version 3.0.0, and cbook.is_string_like was deprecated https://matplotlib.org/api/api_changes.html
When I try to run link prediction by adding the following code in test_karate.py
MAP, prec_curv = lp.evaluateStaticLinkPrediction(G, embedding, 0.8, is_undirected=False)
I got the error:
Traceback (most recent call last):
File "tests/test_karate.py", line 55, in
MAP, prec_curv = lp.evaluateStaticLinkPrediction(G, embedding, 0.8, is_undirected=False)
File "/home/cai.507/Documents/DeepLearning/deep-persistence/pythoncode/GEM/gem/evaluation/evaluate_link_prediction.py", line 24, in evaluateStaticLinkPrediction
is_undirected=is_undirected
File "/home/cai.507/Documents/DeepLearning/deep-persistence/pythoncode/GEM/gem/utils/evaluation_util.py", line 40, in splitDiGraphToTrainTest
for (st, ed, w) in di_graph.edges_iter(data='weight', default=1):
AttributeError: 'DiGraph' object has no attribute 'edges_iter'
In sdne.py:
def weighted_mse_x(y_true, y_pred):
''' Hack: This fn doesn't accept additional arguments. We use y_true to pass them.
y_pred: Contains x_hat - x
y_true: Contains [b, deg]
'''
return KBack.sum(KBack.square(y_pred * y_true[:,0:self._node_num]), axis=-1)/y_true[:,self._node_num]
This code shows that you consider the degree of nodes in the loss function of second-order proximity.
However, The authur don't make use of node's degree in the paper.
Are there any benefit if the loss is divided by the degree of nodes?
I have been running into errors running SDNE.
ValueError: Error when checking target: expected subtract_1 to have shape (None, 150) but got array with shape (500, 151)
I am trying to embed a graph with 150 nodes.
I have added some print lines to sdne.py
S = (S + S.T) / 2
print ("S graph adj mat ")
print (S.shape)
I can see that adjacency matrix inputted has the right dimensions
self._node_num = graph.number_of_nodes()
print ("S graph nodes")
print (self._node_num)
Both lines output [None,150]
However the following lines with print lines added in sdne_utils.py
a1 = np.append(B_i, deg_i, axis=1)
a2 = np.append(B_j, deg_j, axis=1)
OutData = [a1, a2, X_ij.T]
counter += 1
print a1.shape
print a2.shape
print X_ij.shape
output
(500, 151)
(500, 151)
(1, 500)
np.append is probably increasing the second dimension by 1
Could this be due to the batch size not being a multiple of the node_num, or something?
i run node2vec in win10, python3.6 and got a problem , i don't know what's mean , i run it as the example you give. hope you can help me
Traceback (most recent call last):
File "D:\Anaconda\lib\site-packages\gem-1.0.0-py3.6.egg\gem\embedding\node2vec.py", line 80, in learn_embedding
call(args)
File "D:\Anaconda\lib\subprocess.py", line 267, in call
with Popen(*popenargs, **kwargs) as p:
File "D:\Anaconda\lib\subprocess.py", line 709, in init
restore_signals, start_new_session)
File "D:\Anaconda\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件。
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "F:/project/network/networkEmbedding/node2vec.py", line 20, in
Y, t = embedding.learn_embedding(graph=G, is_weighted=True)
File "D:\Anaconda\lib\site-packages\gem-1.0.0-py3.6.egg\gem\embedding\node2vec.py", line 83, in learn_embedding
raise Exception('./node2vec not found. Please compile snap, place node2vec in the path and grant executable permission')
Exception: ./node2vec not found. Please compile snap, place node2vec in the path and grant executable permission
I get the following error when trying to execute node2vec.
Traceback (most recent call last):
File "/usr/share/pycharm/helpers/pydev/pydev_run_in_console.py", line 78, in
globals = run_file(file, None, None)
File "/usr/share/pycharm/helpers/pydev/pydev_run_in_console.py", line 35, in run_file
pydev_imports.execfile(file, globals, locals) # execute the script
File "/usr/share/pycharm/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/mj/git/TIA_Portal_Graph_Analysis/graph_analysis/graph_analysis.py", line 59, in
Y, t = embedding.learn_embedding(graph=g_netx, edge_f=None, is_weighted=False, no_python=True)
File "/home/mj/.local/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/node2vec.py", line 84, in learn_embedding
raise Exception('./node2vec not found. Please compile snap, place node2vec in the path and grant executable permission')
Exception: ./node2vec not found. Please compile snap, place node2vec in the path and grant executable permission
I looked in the installation package directory and I don't see the c_exe folder there. Should I need to manually copy it there?
Thanks
Output is:
Using TensorFlow backend.
/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.7
return f(*args, **kwds)
Traceback (most recent call last):
File "test_sbm.py", line 34, in <module>
G = nx.read_gpickle(file_prefix)
File "<decorator-gen-404>", line 2, in read_gpickle
File "/usr/local/lib/python3.7/site-packages/networkx/utils/decorators.py", line 205, in _open_file
fobj = _dispatch_dict[ext](path, mode=mode)
FileNotFoundError: [Errno 2] No such file or directory: 'sbm.gpickle'
Traceback (most recent call last):
File "C:/Users/hurun/PycharmProjects/GEM/Example1.py", line 52, in
Y, t = embedding.learn_embedding(graph=G, edge_f=None, is_weighted=True, no_python=True)
File "C:\Users\hurun\AppData\Local\Programs\Python\Python36\lib\site-packages\gem-1.0.0-py3.6.egg\gem\embedding\gf.py", line 98, in learn_embedding
[f1, f2, f] = self._get_f_value(graph)
File "C:\Users\hurun\AppData\Local\Programs\Python\Python36\lib\site-packages\gem-1.0.0-py3.6.egg\gem\embedding\gf.py", line 56, in _get_f_value
f1 += (w - np.dot(self._X[i, :], self._X[j, :]))**2
TypeError: unsupported operand type(s) for -: 'dict' and 'float'
Hi. The DiGraph link is broken - here is the correct one (for v 1.11, if this matters):
https://networkx.github.io/documentation/networkx-1.11/reference/classes.digraph.html
Hello,
replacing the input dataset in the test script with the karate edgelist, I can the following error upon execution:
python test.py
Using TensorFlow backend.
Couldn't import dot_parser, loading of dot files will not be possible.
Traceback (most recent call last):
File "test.py", line 28, in <module>
models.append(GraphFactorization(2, 50000, 1*10**-4, 1.0))
File "/code/GEM/gem/embedding/gf.py", line 44, in __init__
for key in dictionary:
TypeError: 'int' object is not iterable
python3.6, run exampl2
error here
Traceback (most recent call last):
File "C:/Users/hurun/PycharmProjects/GEM/Example1.py", line 47, in
print ('Num nodes: %d, num edges: %d' % (G.number_of_nodes(), G.number_of_edges()))
File "C:\Users\hurun\AppData\Local\Programs\Python\Python36\lib\site-packages\networkx\classes\graph.py", line 780, in number_of_nodes
return len(self._node)
AttributeError: 'DiGraph' object has no attribute '_node'
Hi, I want to try SDNE, but it gives me following error:
Using TensorFlow backend.
/Users/luning/.pyenv/versions/3.6.1/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
Num nodes: 34, num edges: 77
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:71: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(50, activation="relu", kernel_regularizer=<keras.reg...)`
W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y[i])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:71: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(15, activation="relu", kernel_regularizer=<keras.reg...)`
W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y[i])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:73: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(2, activation="relu", kernel_regularizer=<keras.reg...)`
W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y[K - 1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:75: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor("in..., outputs=Tensor("de...)`
encoder = Model(input=x, output=y[K])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:90: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(15, activation="relu", kernel_regularizer=<keras.reg...)`
W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y_hat[i + 1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:90: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(50, activation="relu", kernel_regularizer=<keras.reg...)`
W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y_hat[i + 1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:92: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(34, activation="relu", kernel_regularizer=<keras.reg...)`
W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y_hat[1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:96: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor("in..., outputs=Tensor("de...)`
decoder = Model(input=y, output=x_hat)
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py:108: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor("in..., outputs=[<tf.Tenso...)`
autoencoder = Model(input=x, output=[x_hat, y])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py:116: UserWarning: The `merge` function is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.
output_shape=lambda L: L[1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/keras/legacy/layers.py:456: UserWarning: The `Merge` layer is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.
name=name)
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py:119: UserWarning: The `merge` function is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.
output_shape=lambda L: L[1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py:122: UserWarning: The `merge` function is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.
output_shape=lambda L: L[1])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py:148: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor("x_..., outputs=[<tf.Tenso...)`
self._model = Model(input=x_in, output=[x_diff1, x_diff2, y_diff])
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py:161: UserWarning: The semantics of the Keras 2 argument `steps_per_epoch` is not the same as the Keras 1 argument `samples_per_epoch`. `steps_per_epoch` is the number of batches to draw from the generator at each epoch. Update your method calls accordingly.
verbose=1
/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py:161: UserWarning: Update your `fit_generator` call to the Keras 2 API: `fit_generator(generator=<generator..., verbose=1, steps_per_epoch=0, epochs=50)`
verbose=1
Epoch 1/50
Exception in thread Thread-1:
Traceback (most recent call last):
File "/Users/luning/.pyenv/versions/3.6.1/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/Users/luning/.pyenv/versions/3.6.1/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/keras/engine/training.py", line 606, in data_generator_task
generator_output = next(self._generator)
File "/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne_utils.py", line 50, in batch_generator_sdne
deg_i = np.sum(X_batch_v_i != 0, 1).reshape((batch_size, 1))
ValueError: cannot reshape array of size 154 into shape (500,1)
Traceback (most recent call last):
File "test.py", line 41, in <module>
Y, t = embedding.learn_embedding(graph=G, edge_f=None, is_weighted=True, no_python=True)
File "/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/gem-1.0.0-py3.6.egg/gem/embedding/sdne.py", line 161, in learn_embedding
verbose=1
File "/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "/Users/luning/.pyenv/versions/3.6.1/envs/neo4j-demo/lib/python3.6/site-packages/keras/engine/training.py", line 1913, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
UnboundLocalError: local variable 'epoch_logs' referenced before assignment
Hi Palash,
I have a question about your paper. I am aware this may not be the best place to ask, but I didn't know where else to post - hope you don't mind.
Maybe I missed it, but in your experiments you don't seem to report the values of the parameters p
and q
that you used for node2vec. Did you optimize those values, for each dataset? If yes, how? Note that if you used the default settings p=q=1
, then you were using DeepWalk, not node2vec.
BTW, in Table 1, you indicate the wrong reference for DeepWalk - it is not S. Skienna et al. but Perozzi et al. And, just out of curiosity, still in Table 1, how did you come up with the time complexity values for DeepWalk and node2vec? I couldn't find those estimates in the original papers.
Thanks!
Hi there, I am a little surprised to see exe files checked into to git, what exactly is that for?
Hey Palash,
Thanks so much for the work on this! I've been trying to replicate your results for SDNE on some of the larger graphs in your paper. However i always hit an out of memory error when the code is trying to allocate a numpy zeros array for the adjacency matrix.
I was wondering how you managed to over come this and if we could make improvements to the code to help running with larger datasets?
I encountered this problem --raise Exception('./node2vec not found. Please compile snap, place node2vec in the path and grant executable permission'), and I can not find the subprocess in ode2vec.py file
I can train system but get error when evaluate graph reconstruction as following:
MAP, prec_curv = gr.evaluateStaticGraphReconstruction(G, embedding, Y, None)
ValueError: too many values to unpack
Could anyone help me please?
Thanks a lot
Hello again!
Is it possible for you to post all of the parameters you used in each one of your experiments? I tried a lot of different combination of parameters but just couldn't recreate your results...
I also noticed you didn't mention the performance in your paper. I tried on my server and it looked like the code ran relatively slow even when the graph only had a moderate size (around 20k nodes). Is this expected? It'll be really helpful if you could give us an estimate on how long each method takes on each dataset.
Thank you so much for your help!
Line 66 in 2e1d217
The call to the eigensolver here utilizes which='SM'
, which is documented to be slow (see last paragraph under heading "Basic Functionality" here). Instead, the shift-invert mode can be used by using the arguments sigma=0, which='LM'
, as documented in the Examples heading here.
If appropriate, I can work on the relevant PR.
Can you provide data sets that are directly used by GEM ?
Thank you very much.
Hello,
It seems like the tool here creates an embedding of nodes or edges based on a single graph.
If I have multiple independent graphs, is it possible to learn an embedding of nodes over multiple graph instances?
Thanks!
rt
I am trying to use SDNE but I am getting the following traceback:
Traceback (most recent call last):
File "gem/embedding/sdne.py", line 260, in
embedding.learn_embedding(graph=G, edge_f=None, is_weighted=True, no_python=True)
File "gem/embedding/sdne.py", line 115, in learn_embedding
self._encoder = get_encoder(self._node_num, self._d, self._K, self._n_units, self._nu1, self._nu2, self._actfn)
File "GEM/gem/embedding/sdne_utils.py", line 16, in get_encoder
y[i+1] = Dense(n_units[i], activation=activation_fn, W_regularizer=Reg.l1_l2(l1=nu1, l2=nu2))(y[i])
AttributeError: 'module' object has no attribute 'l1_l2'
It seems like some sort of Keras error. Any help/insight you can give will be appreciated.
Additionally, why did you re-implement SDNE when a reference implementation is available online? I tried using it and the embeddings I am getting from it are quite poor so I am wondering if you had a similar experience.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.