mli-lab / deepdewedge Goto Github PK
View Code? Open in Web Editor NEWSelf-supervised deep learning for denoising and missing wedge reconstruction of cryo-ET tomograms
License: BSD 2-Clause "Simplified" License
Self-supervised deep learning for denoising and missing wedge reconstruction of cryo-ET tomograms
License: BSD 2-Clause "Simplified" License
Hello,
I've been enjoying trying out DeepDeWedge to denoise/correct some of our tomograms. However, on our data I noticed that the losses (both training and validation) were very high compared to the tutorial data, and that after refining the tomograms it produces a weird, grid-like artifact. I thought it may be due to something wrong with input dimensions, etc and reassembling of the refined subtomograms into the full tomogram, however when I extracted subtomograms and reassembled them from an unrefined tomogram (using ddw functions), everything looked fine. This made me curious if the high loss of the model is just resulting in poor refinement of subtomogram edges?
I've attached screenshots below - the first is a screenshot of the training loss curves from fitting to the entire tomogram, the second is the training loss curves from fitting to a sub-region of the tomogram that contains mostly the cell of interest and excludes empty regions, and the third is a screenshot of the refined tomogram. As can be seen, removing the empty areas from the tomogram improves the trends of the training loss curves (I could have ran it for much longer), however the losses are still very high and it didn't seem to improve the refinement very much.
Training curve from the full tomogram
Training curve for the tomogram region with most empty ice exluded
Snapshot of the refined tomogram
I also noticed that the tutorial used bin 6 data, whereas this data is bin 4 (pixel size ~1.06 nm). I'm rerunning it now with bin 6 data (pixel size ~1.6 nm) and the loss looks better, but it is still hovering above 900 so I'm not sure how the results will look. Let me know if you have any advice about this issue!
Best,
Matt
Hi,
I just wanted to share that following the instructions provided for installation did not work for me. Here is how I got to install DeepDeWedge
:
First, the required Python version was not specified anywhere. After some trial and error (regarding the pytorch
dependency and such) I found that python=3.10
works on my system (CentOS 7, NVIDIA driver 545.23.08, CUDA 12.3)
git clone https://github.com/MLI-lab/DeepDeWedge
cd DeepDeWedge
mamba create -n DeepDeWedge_env python=3.10
Then I had to edit the requirements.txt
to be as follows:
torch==1.12.1
mrcfile
matplotlib
pandas
pytorch-lightning==1.8.0.post1
scikit-image
and then
pip install -r requirements.txt`
finally worked.
Hope this helps other people out there!
Thank you,
Ricardo
Hi,
it is a bit of a pain to erase all folders in subtomo-dir every time the run crashes and I want to re-run the fixed command. I am sure this is an easy fix.
When trying to resume ddw fit-model
from a previous checkpoint, it seems to ignore it and start fitting from scratch again. I am adding the resume-from-checkpoint
line to my config.yaml
file:
fit_model:
unet_params_dict:
chans: 32
num_downsample_layers: 3
drop_prob: 0.1
adam_params_dict:
lr: 0.0004
num_epochs: 600
batch_size: 10
update_subtomo_missing_wedges_every_n_epochs: 10
check_val_every_n_epochs: 10
save_n_models_with_lowest_val_loss: 5
save_n_models_with_lowest_fitting_loss: 5
save_model_every_n_epochs: 50
logger: "csv"
resume-from-checkpoint: "ddw_imod/logs/version_0/checkpoints/epoch/epoch=399.ckpt"
I know it's starting from scratch because the fitting loss at this epoch was 0.14867860078811646 in the previous run (which crashed), but upon starting this new run it says the fitting loss is 0.1634301096200943 in the first epoch. Does that make sense?
Thank you!
Hi Simon,
I am consistently observing the validation loss almost always being lower than the fitting loss, which is quite puzzling. Isn't the validation loss expected to be higher than the fitting loss, on average?
My guess is some issue with the normalization of the validation subtomograms, not sure. The curve can probably still be used to identify overfitting, regardless of the scale, I think.
Here are 4 examples from different cases (2 sets of tomograms reconstructed in 2 different ways each):
Thank you!
I installed the software yesterday (after implemented overwrite in prepare_data comment) and i got the following error in the tutorial run (provided config with updated : model_checkpoint_file: "./tutorial_project/logs/version_3/checkpoints/epoch/epoch=999.ckpt")
ddw refine-tomogram --config ./config.yaml
Running the second command returns this error. I am not clear on what is happening there.
Command: ddw fit-model --unet-params-dict chans 32 drop_prob 0.0 num_downsample_layers --adam-params-dict lr 0.0004 --num-epochs 100 --batch-size 5 --subtomo-size 64 --mw-angle 60 --gpu 0 --num-workers 3
Error:
Traceback (most recent call last) ────────────────────────────────╮
│ /cemcofsw/em/deepDeWedge/240507/bin/ddw:8 in │
│ │
│ 5 from ddw.app import main │
│ 6 if name == 'main': │
│ 7 │ sys.argv[0] = re.sub(r'(-script.pyw|.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(main()) │
│ 9 │
│ │
│ ╭───────────────────────────────────── locals ─────────────────────────────────────╮ │
│ │ main = <function main at 0x7f4db165f0a0> │ │
│ │ re = <module 're' from '/cemcofsw/em/deepDeWedge/240507/lib/python3.10/re.py'> │ │
│ │ sys = <module 'sys' (built-in)> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/ddw/app.py:14 in main │
│ │
│ 11 │
│ 12 │
│ 13 def main(): │
│ ❱ 14 │ app() │
│ 15 │
│ 16 │
│ 17 if name == "main": │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/typer/main.py:326 in call │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/typer/main.py:309 in call │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/click/core.py:1157 in call │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/typer/core.py:723 in main │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/typer/core.py:193 in _main │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/click/core.py:1686 in invoke │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/click/core.py:943 in make_context │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/click/core.py:1408 in parse_args │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/click/core.py:2400 in │
│ handle_parse_result │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/click/core.py:2362 in process_value │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/site-packages/typer/main.py:1001 in wrapper │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/ast.py:110 in literal_eval │
│ │
│ 107 │ │ │ │ else: │
│ 108 │ │ │ │ │ return left - right │
│ 109 │ │ return _convert_signed_num(node) │
│ ❱ 110 │ return _convert(node_or_string) │
│ 111 │
│ 112 │
│ 113 def dump(node, annotate_fields=True, include_attributes=False, *, indent=None): │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ _convert = <function literal_eval.._convert at 0x7f4c4099a680> │ │
│ │ _convert_num = <function literal_eval.._convert_num at 0x7f4c4099a560> │ │
│ │ _convert_signed_num = <function literal_eval.._convert_signed_num at │ │
│ │ 0x7f4c4099a5f0> │ │
│ │ _raise_malformed_node = <function literal_eval.._raise_malformed_node at │ │
│ │ 0x7f4c4099a440> │ │
│ │ node_or_string = <ast.Name object at 0x7f4c409b4850> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/ast.py:109 in _convert │
│ │
│ 106 │ │ │ │ │ return left + right │
│ 107 │ │ │ │ else: │
│ 108 │ │ │ │ │ return left - right │
│ ❱ 109 │ │ return _convert_signed_num(node) │
│ 110 │ return _convert(node_or_string) │
│ 111 │
│ 112 │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ _convert = <function literal_eval.._convert at 0x7f4c4099a680> │ │
│ │ _convert_num = <function literal_eval.._convert_num at 0x7f4c4099a560> │ │
│ │ _convert_signed_num = <function literal_eval.._convert_signed_num at │ │
│ │ 0x7f4c4099a5f0> │ │
│ │ _raise_malformed_node = <function literal_eval.._raise_malformed_node at │ │
│ │ 0x7f4c4099a440> │ │
│ │ node = <ast.Name object at 0x7f4c409b4850> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/ast.py:83 in _convert_signed_num │
│ │
│ 80 │ │ │ │ return + operand │
│ 81 │ │ │ else: │
│ 82 │ │ │ │ return - operand │
│ ❱ 83 │ │ return _convert_num(node) │
│ 84 │ def _convert(node): │
│ 85 │ │ if isinstance(node, Constant): │
│ 86 │ │ │ return node.value │
│ │
│ ╭──────────────────────────────────── locals ────────────────────────────────────╮ │
│ │ _convert_num = <function literal_eval.._convert_num at 0x7f4c4099a560> │ │
│ │ node = <ast.Name object at 0x7f4c409b4850> │ │
│ ╰────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/ast.py:74 in _convert_num │
│ │
│ 71 │ │ raise ValueError(msg + f': {node!r}') │
│ 72 │ def _convert_num(node): │
│ 73 │ │ if not isinstance(node, Constant) or type(node.value) not in (int, float, comple │
│ ❱ 74 │ │ │ _raise_malformed_node(node) │
│ 75 │ │ return node.value │
│ 76 │ def _convert_signed_num(node): │
│ 77 │ │ if isinstance(node, UnaryOp) and isinstance(node.op, (UAdd, USub)): │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ _raise_malformed_node = <function literal_eval.._raise_malformed_node at │ │
│ │ 0x7f4c4099a440> │ │
│ │ node = <ast.Name object at 0x7f4c409b4850> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /cemcofsw/em/deepDeWedge/240507/lib/python3.10/ast.py:71 in _raise_malformed_node │
│ │
│ 68 │ │ msg = "malformed node or string" │
│ 69 │ │ if lno := getattr(node, 'lineno', None): │
│ 70 │ │ │ msg += f' on line {lno}' │
│ ❱ 71 │ │ raise ValueError(msg + f': {node!r}') │
│ 72 │ def _convert_num(node): │
│ 73 │ │ if not isinstance(node, Constant) or type(node.value) not in (int, float, comple │
│ 74 │ │ │ _raise_malformed_node(node) │
│ │
│ ╭────────────────── locals ───────────────────╮ │
│ │ lno = 1 │ │
│ │ msg = 'malformed node or string on line 1' │ │
│ │ node = <ast.Name object at 0x7f4c409b4850> │ │
│ ╰─────────────────────────────────────────────╯
Originally posted by @Mishaczech in #3 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.