markpkcollier / neuralturingmachine Goto Github PK

Will each data in the batch have a different M eventually?

Will M change during prediction?

I look forward to your answer.

Thank you

Hi!

I have been trying to replicate the copy-task results from your paper; however, after running python3 run_tasks.py --experiment_name=trial01 the copy task does not seem to converge.

I obtained an error of 31.89375 per sequence and it gave the following results:

I ran it now multiple times but I always got similar results (I believe that you ran the experiments 10 times and took the mean per plot).

I checked the defaults parameters but everything seems to be in accordance to your paper.

Is there something that I am missing?

By the way, great job with the project :-)!

Maybe you could use locality sensitive hash (LSH) associative memory for NTMs?
http://gamespace.eu5.org/associativememory/index.html

It looks there is a recurrence happening on all elements in the state. Wouldn't you need
to do a tf.stop_gradient() on each of these items (except controller_state) to be consistent with the paper? At the very least there should be no recurrence on the memory M, right?

Hi Dear @MarkPKCollier

I wanted to run the code using python 3.5 on PyCharm, an error occured in file 'Ncross_analysis.py'

''''
C:\Users\Eduline\PycharmProjects\python\venv\Scripts\python.exe C:/Users/Eduline/Desktop/GEOMETRİ1/Ncross_analysis.py
File "", line 15
def getLine_pixelPoint((x1, y1), (x2, y2)):
^
SyntaxError: invalid syntax
''''
and it seems like the error is because 'tuple unpacking is not supported in Python3'.

Could you please help me to solve this problem?
How can I change the written code (in the photo), so that it will not be problematic ?

Have a good day

Hi @MarkPKCollier

I wanted to run the code using python 3.6, an error occurs in file 'generate_data.py', Line: 263

File "D:\...\NeuralTuringMachine-master\generate_data.py", line 263
    outputs = map(lambda (source, dest, edge):
                         ^
SyntaxError: invalid syntax

and it seems that the error is because 'tuple unpacking is not supported in Python3'.

What can I do? switch to Python 2?

Thank you.

empty

Hi @MarkPKCollier

Thanks for your awesome implementation and making the code available in such a well packaged form!

You mentioned in the paper that you tried 3 of the 5 experiments. I was wondering whether you were also able to reproduce the other 2 experiments (in particular the sorting experiment), and whether you can share some of your thoughts on those as well.

Thanks

I am really struggling to understand how to integrate your code in my tf/keras projects, could you provide some basic examples showing how to compile and train a network?

Say we are considering the copy task and we want to give a sequence of length 10 to our model. So, following the code, we actually give our controller a sequence of length 21, where the 11th element is an end of sequence marker (a vector of 1s) and the last 10 elements are vectors of 0s. Where does the code specify that the output should start at the end of sequence marker?

Now, for the repeated copy task, it doesn't seem like we should provide the target output size the same way we do in the copy task. For example, given a sequence of length 10 that is to be copied 2 times, it wouldn't make sense to give as input a sequence of 31 vectors since we want the model to learn when to stop copying. So how should the input be structured in this case?

Using ntm as the default setting works fine on copy task, but it doesn't work on associative recall task.

I run for more than 20000 training steps with 5 hours and 1 GPU, and the sequence error doesn't drop.

Does associative recall task needs another setting?

sorry for the silly question. I am an engineer, not a researcher(I don't have a strong backgroud on theory and math).
I try to apply this external memory idea on my work. I read the original NTM paper: 1410.5401 and your paper, I can't find the formula that generate these head parameters:
k, beta, g, s gamma.

Do I miss something or there are some "common knowledge" for this kind of parameter?