deci-ai / data-gradients Goto Github PK
View Code? Open in Web Editor NEWComputer Vision dataset analysis
License: Apache License 2.0
Computer Vision dataset analysis
License: Apache License 2.0
The data-gradients package is much more friendly to make reports for image datasets. Making some notebooks could help the developers to get started.
PS: Does the project is accepting any PR for the same?
Hey, I have a problem with importing DetectionAnalysisManager after installing data-gradients .So it's present in my site-packages. I tried using different PYTHONPATHes but it doesn't help. And I get ModuleNotFoundError
again.
from data_gradients.managers.detection_manager import DetectionAnalysisManager
And have an error
Traceback (most recent call last):
File "/home/user/data-gradients/data_gradients.py", line 1, in <module>
from data_gradients.managers.detection_manager import DetectionAnalysisManager
File "/home/user/data-gradients/data_gradients.py", line 1, in <module>
from data_gradients.managers.detection_manager import DetectionAnalysisManager
ModuleNotFoundError: No module named 'data_gradients.managers'; 'data_gradients' is not a package
python - 3.9.17. data-gradients - 0.1.4
How can this be resolved?
Thanks
No response
Image Shape: (256, 256)
Enter the channel format representing your image:
RGB : Red, Green, Blue
BGR : Blue, Green, Red
G : Grayscale
LAB : Luminance, A and B color channels
ADDITIONAL CHANNELS?
If your image contains channels other than the standard ones listed above (e.g., Depth, Heat), prefix them with 'O'.
For instance:
ORGBO: Can represent (Heat, Red, Green, Blue, Depth).
OBGR: Can represent (Alpha, Blue, Green, Red).
GO: Can represent (Gray, Depth).
IMPORTANT: Make sure that your answer represents all the image channels.
Enter your response >>> RGB : Red, Green, Blue
RGB : Red, Green, Blue
is not a valid input! Please check the instruction and try again.
Enter your response >>> rgb
rgb
is not a valid input! Please check the instruction and try again.
Enter your response >>> RGB
RGB
is not a valid input! Please check the instruction and try again.
Enter your response >>> RGB
RGB
is not a valid input! Please check the instruction and try again.
Enter your response >>> RGB : Red, Green, Blue
RGB : Red, Green, Blue
is not a valid input! Please check the instruction and try again.
No response
Is this data gradient not available for segmentation analysis of dataset in yolo format?
how i can use data-gradients in yolo-nas to show me a repport of accurancy,recall ,precision..etc
Hi, I am trying to run the detection_tinycoco.ipynb
notebook from examples in Google Colab, with a fresh install, but it fails at this step:
from data_gradients.managers.detection_manager import DetectionAnalysisManager
from data_gradients.datasets.detection.coco_detection_dataset import COCODetectionDataset
Error:
Downloading: "https://download.pytorch.org/models/mobilenet_v3_small-047dcff4.pth" to /root/.cache/torch/hub/checkpoints/mobilenet_v3_small-047dcff4.pth
100%|โโโโโโโโโโ| 9.83M/9.83M [00:00<00:00, 24.1MB/s]
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
[<ipython-input-2-5657470712ca>](https://localhost:8080/#) in <cell line: 2>()
1 from data_gradients.managers.detection_manager import DetectionAnalysisManager
----> 2 from data_gradients.datasets.detection.coco_detection_dataset import COCODetectionDataset
2 frames
[/usr/local/lib/python3.10/dist-packages/data_gradients/datasets/segmentation/voc_segmentation_dataset.py](https://localhost:8080/#) in <module>
2 from typing import Union
3
----> 4 from data_gradients.datasets.download.voc import download_VOC
5 from data_gradients.datasets.segmentation.voc_format_segmentation_dataset import VOCFormatSegmentationDataset
6
ModuleNotFoundError: No module named 'data_gradients.datasets.download'
---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.
To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------
I have tried the DG_Demo.ipynb notebook as well and the same issue happened there.
Although, the classification notebook under examples works fine.
Collecting environment information...
PyTorch version: 2.0.1+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.2 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.0-1ubuntu1.1
CMake version: version 3.27.4
Libc version: glibc-2.35
Python version: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-5.15.120+-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Vendor ID: GenuineIntel
Model name: Intel(R) Xeon(R) CPU @ 2.20GHz
CPU family: 6
Model: 79
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
Stepping: 0
BogoMIPS: 4399.99
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat md_clear arch_capabilities
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32 KiB (1 instance)
L1i cache: 32 KiB (1 instance)
L2 cache: 256 KiB (1 instance)
L3 cache: 55 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0,1
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Mitigation; PTE Inversion
Vulnerability Mds: Vulnerable; SMT Host state unknown
Vulnerability Meltdown: Vulnerable
Vulnerability Mmio stale data: Vulnerable
Vulnerability Retbleed: Vulnerable
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Vulnerable
Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==2.0.1+cu118
[pip3] torchaudio==2.0.2+cu118
[pip3] torchdata==0.6.1
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.15.2
[pip3] torchvision==0.15.2+cu118
[pip3] triton==2.0.0
[conda] Could not collect
I got this error:
cannot import name 'vit_b_16' from 'torchvision.models'
for this:
from data_gradients.managers.segmentation_manager import SegmentationAnalysisManager
Is there required version of torch/torchvision ?
No response
File "analyse_dataset.py", line 27, in
analyzer.run()
File "/usr/local/lib/python3.8/dist-packages/data_gradients/managers/abstract_manager.py", line 226, in run
self.execute()
File "/usr/local/lib/python3.8/dist-packages/data_gradients/managers/abstract_manager.py", line 114, in execute
for i, (train_batch, val_batch) in enumerate(datasets_tqdm):
File "/usr/local/lib/python3.8/dist-packages/tqdm/std.py", line 1195, in iter
for obj in iterable:
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 635, in next
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 679, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
return self.collate_fn(data)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 143, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 143, in
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 120, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 172, in collate_numpy_array_fn
return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 120, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 163, in collate_tensor_fn
return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [1, 5] at entry 0 and [4, 5] at entry 2
[pip3] numpy==1.22.2
[pip3] pytorch-quantization==2.1.2
[pip3] torch==1.14.0a0+44dac51
[pip3] torch-tensorrt==1.4.0.dev0
[pip3] torchtext==0.13.0a0+fae8e8c
[pip3] torchvision==0.15.0a0
[pip3] triton==2.0.0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.