Experiment tracking for fastai. 🧩 Log, organize, visualize and compare model metrics, hyperparameters, dataset versions, and more.
Home Page: https://docs.neptune.ai/integrations/fastai/
License: Apache License 2.0
Learner for regression models does not have the vocab parameter as it is generally a list of labels. While tracking experiments with neptune-ai for fastai/tsai the experiment fails as the vocab parameter is expected by neptune callback.
This should be an optional parameter since fastai and tsai by default exclude _vocab for regression models.
from tsai.all import *
import neptune.new as neptune
from neptune.new.integrations.fastai import NeptuneCallback
def main():
neptune_run = neptune.init()
dsid = 'AppliancesEnergy'
X, y, splits = get_regression_data(dsid, split_data=False)
X.shape, y.shape, y[:10]
tfms = [None, [TSRegression()]]
batch_tfms = TSStandardize(by_sample=True, by_var=True)
dls = get_ts_dls(X, y, splits=splits, tfms=tfms, batch_tfms=batch_tfms, bs=128)
learn = ts_learner(dls, InceptionTime, metrics=[mae, rmse], cbs=[NeptuneCallback(neptune_run, 'experiment')])
learn.fit_one_cycle(50, 1e-2)
if __name__ == '__main__':
main()This code is taken from https://timeseriesai.github.io/tsai/tutorials.html#Time-series-regression with modification to include neptune tracking.
Run the above snippet.
The training starts and neptune logs everything normally.
For the above snippet
Partial Traceback:
AttributeError: 'InceptionTime' object has no attribute '_vocab'
Complete Traceback:
Info (NVML): NVML Shared Library Not Found. GPU usage metrics may not be reported. For more information, see https://docs.neptune.ai/you-should-know/what-can-you-log-and-display#hardware-consumption
119it [00:03, 39.47it/s]
66it [00:01, 43.77it/s]
epoch train_loss valid_loss mae _rmse time
Traceback (most recent call last):
File "/Users/abhijitramesh/Development/DeepLearning/example.py", line 20, in <module>
main()
File "/Users/abhijitramesh/Development/DeepLearning/example.py", line 16, in main
learn.fit_one_cycle(50, 1e-2)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastai/callback/schedule.py", line 116, in fit_one_cycle
self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastai/learner.py", line 221, in fit
self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastai/learner.py", line 163, in _with_events
try: self(f'before_{event_type}'); f()
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastai/learner.py", line 141, in __call__
def __call__(self, event_name): L(event_name).map(self._call_one)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastcore/foundation.py", line 155, in map
def map(self, f, *args, gen=False, **kwargs): return self._new(map_ex(self, f, *args, gen=gen, **kwargs))
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastcore/basics.py", line 698, in map_ex
return list(res)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastcore/basics.py", line 683, in __call__
return self.func(*fargs, **kwargs)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastai/learner.py", line 145, in _call_one
for cb in self.cbs.sorted('order'): cb(event_name)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastai/callback/core.py", line 45, in __call__
if self.run and _run: res = getattr(self, event_name, noop)()
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/neptune_fastai/impl/__init__.py", line 163, in before_fit
self._log_model_configuration()
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/neptune_fastai/impl/__init__.py", line 139, in _log_model_configuration
'details': self._vocab,
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastcore/basics.py", line 389, in __getattr__
if attr is not None: return getattr(attr,k)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/fastcore/basics.py", line 389, in __getattr__
if attr is not None: return getattr(attr,k)
File "/Users/abhijitramesh/Development/DeepLearning/Theta-Tech-ai/Semler-Consulting-local/Semler-Consulting-local/Semler-Consulting/integration/qf_plus_trained/env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in __getattr__
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'InceptionTime' object has no attribute '_vocab'
Shutting down background jobs, please wait a moment...
Done!
Waiting for the remaining 86 operations to synchronize with Neptune. Do not kill this process.
All 86 operations synced, thanks for waiting!
The output of pip list:
Package Version
---------------------- ---------
alembic 1.7.5
appnope 0.1.2
asttokens 2.0.5
attrs 21.4.0
autopage 0.4.0
backcall 0.2.0
black 21.12b0
blis 0.7.5
boto3 1.18.18
botocore 1.21.65
bravado 11.0.3
bravado-core 5.17.0
catalogue 2.0.6
certifi 2021.10.8
charset-normalizer 2.0.10
click 8.0.3
cliff 3.10.0
cloudpickle 2.0.0
cmaes 0.8.2
cmd2 2.3.3
colorlog 6.6.0
cycler 0.11.0
cymem 2.0.6
dask 2022.1.0
debugpy 1.5.1
decorator 5.1.1
Deprecated 1.2.13
distributed 2022.1.0
entrypoints 0.3
et-xmlfile 1.1.0
executing 0.8.2
fastai 2.5.3
fastcore 1.3.27
fastdownload 0.0.5
fastprogress 1.0.0
fonttools 4.28.5
fsspec 2022.1.0
future 0.18.2
gitdb 4.0.9
GitPython 3.1.26
greenlet 1.1.2
HeapDict 1.0.1
idna 3.3
imbalanced-learn 0.9.0
ipykernel 6.7.0
ipython 8.0.0
ipython-genutils 0.2.0
jedi 0.18.1
Jinja2 3.0.3
jmespath 0.10.0
joblib 1.1.0
jsonpointer 2.2
jsonref 0.2
jsonschema 3.2.0
jupyter-client 7.1.1
jupyter-core 4.9.1
kiwisolver 1.3.2
langcodes 3.3.0
llvmlite 0.36.0
locket 0.2.1
Mako 1.1.6
MarkupSafe 2.0.1
matplotlib 3.5.1
matplotlib-inline 0.1.3
matrixprofile 1.1.10
monotonic 1.6
msgpack 1.0.3
murmurhash 1.0.6
mypy-extensions 0.4.3
nbformat 5.1.3
neptune-client 0.14.2
neptune-fastai 0.10.0
neptune-optuna 0.9.13
nest-asyncio 1.5.4
numba 0.53.1
numpy 1.19.3
oauthlib 3.1.1
openpyxl 3.0.7
optuna 2.9.1
packaging 21.3
pandas 1.3.5
parso 0.8.3
partd 1.2.0
pathspec 0.9.0
pathy 0.6.1
patsy 0.5.2
pbr 5.8.0
pexpect 4.8.0
pickleshare 0.7.5
Pillow 9.0.0
pip 21.1.1
platformdirs 2.4.1
plotly 5.5.0
preshed 3.0.6
prettytable 3.0.0
prompt-toolkit 3.0.24
protobuf 3.11.2
psutil 5.9.0
psycopg2-binary 2.9.1
ptyprocess 0.7.0
pure-eval 0.2.1
pydantic 1.8.2
Pygments 2.11.2
PyJWT 2.3.0
pyparsing 3.0.6
pyperclip 1.8.2
pyrsistent 0.18.1
python-dateutil 2.8.2
pytorch-ranger 0.1.1
pyts 0.12.0
pytz 2021.3
PyWavelets 1.2.0
PyYAML 6.0
pyzmq 22.3.0
requests 2.27.1
requests-oauthlib 1.3.0
rfc3987 1.3.8
s3transfer 0.5.0
scikit-learn 1.0.2
scipy 1.7.0
seaborn 0.11.2
setuptools 56.0.0
simplejson 3.17.6
six 1.16.0
sklearn 0.0
sktime 0.9.0
smart-open 5.2.1
smmap 5.0.0
sortedcontainers 2.4.0
spacy 3.2.1
spacy-legacy 3.0.8
spacy-loggers 1.0.1
SQLAlchemy 1.4.29
srsly 2.4.2
stack-data 0.1.4
statsmodels 0.12.1
stevedore 3.5.0
strict-rfc3339 0.7
stumpy 1.10.2
swagger-spec-validator 2.7.4
tblib 1.7.0
tenacity 8.0.1
thinc 8.0.13
threadpoolctl 3.0.0
tomli 1.2.3
toolz 0.11.2
torch 1.9.1
torch-optimizer 0.3.0
torchvision 0.10.1
tornado 6.1
tqdm 4.62.3
traitlets 5.1.1
tsai 0.2.24
tsfresh 0.18.0
typer 0.4.0
typing-extensions 4.0.1
urllib3 1.26.8
wasabi 0.9.0
wcwidth 0.2.5
webcolors 1.11.1
websocket-client 1.2.3
wheel 0.37.1
wrapt 1.13.3
zict 2.0.0
The operating system you're using: macOS 12.0.1
The output of python --version: Python 3.9.5
I found hotfix's for the problem but its not ideal:
This is not ideal we need vocab for classification.
learn._vocab=[]These fixes are not ideal, vocab should be optional parameter in the init.py for the neptune_fastai implementation.
I can't export FastAI learner with NeptuneCallback
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-25-1903c9215920> in <cell line: 1>()
----> 1 learn.export(f"output/models/learn_bigger_data{str(i)}.pkl")
~/.local/lib/python3.8/site-packages/fastai/learner.py in export(self, fname, pickle_module, pickle_protocol)
402 #To avoid the warning that come from PyTorch about model not being checked
403 warnings.simplefilter("ignore")
--> 404 torch.save(self, self.path/fname, pickle_module=pickle_module, pickle_protocol=pickle_protocol)
405 self.create_opt()
406 if state is not None: self.opt.load_state_dict(state)
~/.local/lib/python3.8/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
377 if _use_new_zipfile_serialization:
378 with _open_zipfile_writer(opened_file) as opened_zipfile:
--> 379 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
380 return
381 _legacy_save(obj, opened_file, pickle_module, pickle_protocol)
~/.local/lib/python3.8/site-packages/torch/serialization.py in _save(obj, zip_file, pickle_module, pickle_protocol)
482 pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
483 pickler.persistent_id = persistent_id
--> 484 pickler.dump(obj)
485 data_value = data_buf.getvalue()
486 zip_file.write_record('data.pkl', data_value, len(data_value))
AttributeError: Can't pickle local object 'NeptuneAuthenticator.__init__.<locals>.session_factory'
I use language_model_learner.
Library versions:
fastai==2.7.7
neptune-fastai==0.10.3
The line here:
Learner.loss_func is an object with a func attribute. However loss_func can be any callable, like torch.nn.MSELoss or just a plain python function.
This causes is to crash with AttributeError
Create a Learner by passing loss_func=nn.MSELoss() or any other custom function. Together with NeptuneCallback
Model trains without crashing
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.