Authors: Manny Ko, Ujjawal K. Panchal, Héctor Andrade Loarca, Andres Mendez-Vazquez

CoShNet Architecture

Architecture

CoShNet is a fully complex hybrid neural network. We use the CoShREM (now call SymFD) http://www.math.uni-bremen.de/cda/software.html signal transform to produce a stable embedding. The network operates entirely in $\mathbb{C}$ domain to take advantage of unique properties of CVNNs.

Architecture Brief

Input is any $32\times32 \in \mathbb{R}$ image.
Input is CoShREM transformed to produce a $32\times32\times20 \in \mathbb{C}$ output.
CoShREM output is convolved with the $2$ cplx-conv layers.
Each cplx-conv layer is composted of := $\mathbb{C}$-Conv + $\mathbb{C}$-ReLU + $\mathbb{C}$-AvgPool2d.
The response is flattened and passed through $2$ cplx-linear layers.

Each cplx-linear layer is composted of := $\mathbb{C}$-linear layer + $\mathbb{C}$-ReLU.
The $\mathbb{R}$, $\mathbb{I}$ components are stacked together (see shape) and passed through $1$ final $\mathbb{R}$-linear layer.

Setup

Python 3.8.x and newer are supported:

Automated Setup

Create a virtualenv at the root of the repo: python -m venv venv4coshnet
Activate venv4coshnet:
- Windows: venv4coshnet\Scripts\activate
- Linux/MacOS: source venv4coshnet/bin/activate
Run setup.py:
- with CUDA: python setup.py
- without CUDA: python setup.py --no-cuda
- use --no-venv to disable venv check (e.g. inside a docker)

Manual Setup

Please follow: setup.txt

Docker Setup

Build image: docker build -t coshnet-docker:latest . (Some systems might require running this in `sudo` mode.)

Contents List

code/: Contains all code essential to run experiments in this repo.
libs/: Contains all custom-made and collected libs and modules we use for our experiments. (Installed automatically in setup.txt)
data/: Folder where datasets are present. Created automatically when running for first time.
setup.txt: Steps for setting up repo.
requirements.txt: requirements file.
changelog.md: all changes relevant to releases, branch prs, or any other general notes needed for maintenance.

How to Run?

Running in Local

cd code/. Following are the possible cases:

Running our models: run: python test_fashion.py --help to see several arguments you are allowed to tune. (Default run (10k20E) gets 89.2% on RTX 2080 Super). The default will use the 10k test set of Fashion to train for 20 epochs, and the 60k training set to test.
Running resnet(18|50): run: python test_resnet.py --help to see several arguments you are allowed to set. (Default run (RN18, 10k20E) gets 88.3% on RTX 2080 Super).

Note: This code (shown in test_fashion.py,test_resnet.py) will not run in (jupyter|google colab) notebook(s). This is because our code defaults to using `asyncio` for batch generation for speed. Hence, if you absolutely have to run in a notebook, please create your own batch generation code.

Running in Docker

Run Image: docker run coshnet-docker:latest (Some systems might require running this in `sudo` mode.)

Note: The above is a brief demo for running our codebase in a docker. If you want to do something specific, e.g. deliver an API endpoint through a docker, you will have to edit the Dockerfile

Some Results

Model	Epochs	Parameters	Size Ratio	Top-1 Accuracy (60k)	Top-1 Accuracy (10k)
ResNet-18	100	11.18M	229	91.8%	88.3%
ResNet-50	100	23.53M	481	90.7%	87.8%
CoShNet(base)	20	1.37M	28	92.2%	89.2%
CoShNet (tiny)	20	49.99K	1	91.6%	88.0%

Note: 60k = train on train-set (60k observations), test on test-set (10k observations). 10k = vice-versa. K or k = 1000, M = Million.

Cite

@misc{coshnet2022,
  doi = {10.48550/ARXIV.2208.06882},
  url = {https://arxiv.org/abs/2208.06882},
  author = {Ko, Manny and Panchal, Ujjawal K. and Andrade-Loarca, Héctor and Mendez-Vazquez, Andres},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {CoShNet: A Hybird Complex Valued Neural Network using Shearlets},
  publisher = {arXiv},
  year = {2022},
  copyright = {Creative Commons Attribution Non Commercial Share Alike 4.0 International}
}