Need some help getting Tensor Rematerialization to work about gpt-2 HOT 7 CLOSED

Ah, that's because you have a newer version of ghc than I used. Not to worry, I updated it to work with your version (ghc 9.0 which corresponds to base-4.15). Pull the repo and try again.

from gpt-2.

That worked perfectly, as far as I am aware, though now when I try to run it using Tensor Rematerialization it throws an error, seemingly about not being able to find it, is there a step I'm missing? I ran the command cabal v2-install --installdir=../bin and then added --twremat and --twremat_memlimit, but then it says this.

bin\twremat C:\Users\natha\AppData\Local\Temp\tmpsc906qwm C:\Users\natha\AppData\Local\Temp\tmp5lue0uvk
Traceback (most recent call last):
File "train.py", line 314, in
main()
File "train.py", line 146, in main
(train_loss, opt_grads) = tfremat.tf_remat((train_loss, opt_grads), memlimit=args.twremat_memlimit)
File "D:\Hobbies\AI\gpt-2-finetuning\src\tfremat.py", line 163, in tf_remat
steps = twremat.runtwremat(node_info, memlimit, {from_op[c] for c in compute_ops})
File "D:\Hobbies\AI\gpt-2-finetuning\src\twremat.py", line 47, in runtwremat
proc = Popen([TWREMAT, fname, outname])
File "C:\Users\natha\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 800, in init
restore_signals, start_new_session)
File "C:\Users\natha\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 1207, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

from gpt-2.

Hmm... That looks like some sort of difference with windows. I can only guess what because I don't do any development on windows. Maybe because it's called twremat.exe there? Try editing src/twremat.py and overriding TWREMAT to the full path of twremat.exe? Probably D:\Hobbies\AI\gpt-2-finetuning\bin\twremat.exe in your case.

from gpt-2.

Verify that the file is actually there too, I guess.

from gpt-2.

How would I do that? Would I simply replace TWREMAT=os.path.join(BINDIR, 'twremat') with TWREMAT=D:\Hobbies\AI\gpt-2-finetuning\bin\twremat.exe') or am I thinking about this totally wrong? And sorry for all the questions I'm completely new to all of this (asides from bashing my head against it multiple times over the years and this week). Also the file is indeed there so that's good.

from gpt-2.

TWREMAT=r'D:\Hobbies\AI\gpt-2-finetuning\bin\twremat.exe'

from gpt-2.

Ah, that worked! Thank you for all the help!

from gpt-2.