rmdlo / cocopen-opencv Goto Github PK
View Code? Open in Web Editor NEWFirst-Prize winner in the Core Track of the 2022 OpenCV AI Competition
License: MIT License
First-Prize winner in the Core Track of the 2022 OpenCV AI Competition
License: MIT License
Note: The submission is the COCOpen-OpenCV github repository. The code submission does not include training detectron2. The code submission should include a demos/visualize.py
file which visualizes a subset of the generated dataset so that users can see for themselves the annotations produced by COCOpen. This file will be based on the Dataset_Verification.ipynb
, but it will generate the mask visualization images in a python script and save images to a folder (make sure to add the folder to the .gitignore
!). Will also need to update the environment.yml
file to install detectron2 dependencies.
The demo includes training detectron2, generating a model, and qualitatively (and quantitatively?) demonstrating the performance of the model. All code for this demo is not part of the COCOpen-OpenCV repository.
The demo video should be ~5 minutes in duration and must:
In the combine function, we have several augmentation methods all in one function. It would be helpful to set the boolean for these as a class variable (e.g., self.standard_scale_jittering=False) rather than hard-coded within the function (lines 391-401 in commit 42782ee
Explain functionality, variables, and I/O associated with modules, classes, and functions in cocopen.py
.
The 'wire' dataset was pruned one last time. We need to upload the final version of this dataset to Azure. The final wire
dataset is here.
Image dimensions are currently hard-coded throughout cocopen.py (height = 1080, width = 1920). Need to replace these with a class variable self.height and self.width. It would be helpful to have a function that checks to make sure all images used to generate the dataset are the same dimensions before beginning the image combination - should throw an exception if image dimensions are not consistent. This function could be included in a dataset preprocessing script (i.e., not in coconut.py).
Note: This affects the scale_image, random_operations, and color_augmentation functions in coconut.py where image dimensions are hard-coded
Train/val split is currently hard-coded as 0.8 in two locations. It would be better to let the user define their train/val split (some users may want a 0.9/0.1 split, for example). Assign this to a class variable and/or read it in from run.py.
if 0 <= rand < 0.8:
self.class_dict = {"device": 1,
"ethernet": 2,
"phone": 3,
"power": 4,
"hdmi": 5,
"coaxial": 6}
self.categories = [{"supercategory": "device", "id": self.class_dict["device"], "name": "device"},
{"supercategory": "cable", "id": self.class_dict["ethernet"], "name": "ethernet"},
{"supercategory": "cable", "id": self.class_dict["phone"], "name": "phone"},
{"supercategory": "cable", "id": self.class_dict["power"], "name": "power"},
{"supercategory": "cable", "id": self.class_dict["hdmi"], "name": "hdmi"},
{"supercategory": "cable", "id": self.class_dict["coaxial"], "name": "coaxial"}]
These categories correspond to data on labelbox. For our current submission, we should only deal with single wire images. We should rename the folder on azure from "single-wire" to "wire" and assign the folder name to the category id. A reach improvement would be to collect images of single devices and set up auto-labeling for them so that we can demonstrate the functionality of this method on more than one object type.
Because the edge pixels of every object may have been lighter than the color threshold (some mix of the color of the object and the color of the background - black), the edge pixels were not filtered out. Every generated object image in the dataset includes black outlines around the object. We could use alpha matting or a similar technique to remove this outline - similar to this post and this post.
Purpose: generate the method can scale to N object categories by demonstrating it works for two.
Device images show holes in the ethernet ports since these ports are shadowed and appear black. We should fill these holes as part of the auto-labeling process.
This link details how to do this:
https://learnopencv.com/filling-holes-in-an-image-using-opencv-python-c/
Add a dictionary key to the label .json file which, for each generated image, indicates which raw images were combined to produce the image (for example, train/1.png comprises single_wire/1099.png, single_wire/2351.png, and single_wire/4312.png)
This information will be added with a (key, value) pair to the coco_new_obj_seg_sem and coco_new_obj_sem dictionaries. Value = a list of image names, e.g., [“wire/2351.png”, “wire/2362.png”...]
***Note: add the folder the image was pulled from (which should correspond to the object’s category ID) to directory structure of this list
***Will need to set up reading the category name for the object from the folder name where the image is stored (so need to rename folder of “single_wire” images to “wire” images)
We have a dependency on pillow, hsv_img = Image.fromarray(rgb_img)
. It would be better to remove pillow dependencies and only use opencv dependencies. If we use opencv for this, we need to make sure to conda uninstall pillow
and upload new environment.yml
files.
We are currently building on a downloaded model configuration in the src/train.py
script rather than loading it in from a training configuration .yaml file which we set up. It would be better for usability and python syntax to load the config from a .yaml file and perform cfg.merge_from_file()
, deleting all the other cfg.X
calls in src/train.py
.
This is not related to structure of code, but is related to user-friendliness: it would be helpful to aggregate total computation time of each function to see which portions of dataset creation take the most time and to target dataset generation optimization.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.