lddl / cnns Goto Github PK

Describe the bug

According this reference:
https://en.wikipedia.org/wiki/Hadamard_product_(matrices)#Definition
For matrices of different dimensions (m × n and p × q, where m ≠ p or n ≠ q) the Hadamard product is undefined.
there is no reason to write code like this:

for z := 0; z < t1.Size.Z; z++ {
	for y := 0; y < t1.Size.Y; y++ {
		for x := 0; x < t1.Size.X; x++ {
			ret.Set(x, y, z, t1.Get(x, y, z)*t2.Get(x, y, z))
		}
	}
}

To Reproduce
look at https://github.com/LdDl/cnns/blob/master/tensor/tmath.go#L76

Expected behavior
nope

Describe the solution you'd like and provide pseudocode examples if you can
Just iterate []float64 as flatten slice and do multiplication (check dimensions first of course)

for i := range ret.Data {
	ret.Data[i] = t1.Data[i] * t2.Data[i]
}

Additional context
This issue is labeled as a "BUG" because of wrong understanding of HadamardProduct.
This function isn't used anywhere (currently), so I have implemented it for "just cause" and forgotten about it a long time ago.
Also this function can help to optimize convolution operation in future.

Is your feature request related to a problem? Please describe.
See ref. https://en.wikipedia.org/wiki/Dilution_(neural_networks)
It should be applied to neural network only when it's in learning stage.

Describe the solution you'd like and provide pseudocode examples if you can
Just use existing best practices for this layer.
Pytorch may be? https://pytorch.org/docs/master/generated/torch.nn.Dropout.html

Describe alternatives you've considered and provide pseudocode examples if you can

Additional context
nope

I like it and it is so good :-p

Is your feature request related to a problem? Please describe.
Switch lddl/cnns/nns and lddl/cnns/nns/tensor to lddl/cnns and lddl/cnns/tensor respectively.
Remove main.go (it's just legacy while learning Go and ML)
Structure examples folder. Add datasets
Remove legacy files (no code, just files)

Describe the solution you'd like and provide pseudocode examples if you can
Just do it carefully and make tests and examples not break

Describe alternatives you've considered and provide pseudocode examples if you can
nope

Additional context
nope

Is your feature request related to a problem? Please describe.
This code:

import (
    t "github.com/LdDl/cnns/tensor"
)

is not a good practice since t is reserved for testing:

import "testing"

Describe the solution you'd like and provide pseudocode examples if you can
Just make bunch of renames

Describe alternatives you've considered and provide pseudocode examples if you can
nope

Additional context
nope

Is your feature request related to a problem? Please describe.
Currently math operations are really slow.
It's better just use battery included gonum

Describe the solution you'd like and provide pseudocode examples if you can

1. Update bunch of internal math operations
2. Operate with matrices (*mat.Dense), not [][][]float64. Stacked matrices is just []*mat.Dense
3. Update DoActivation() on layers.
4. Update BackPropagate() on layers.
5. Update constructors.

Describe alternatives you've considered and provide pseudocode examples if you can
...or just optimize Matrix operations (lot of time and reinvent the wheel)

Additional context
Need to add mention about gonum's authors and theirs license

Is your feature request related to a problem? Please describe.
Please see ref. https://en.wikipedia.org/wiki/Mathematics_of_artificial_neural_networks#Bias

Describe the solution you'd like and provide pseudocode examples if you can
Just slice of values, which would be used as shifting coefficients

Describe alternatives you've considered and provide pseudocode examples if you can
nope

Additional context
nope

Describe the bug
There are some log.Panicln(...) calls in code. I do not think that this is good at all.
Also error handling is not that good,

To Reproduce
nope

Expected behavior
nope

Describe the solution you'd like and provide pseudocode examples if you can
Every function which can cause error should return it as second returning argument.
For example : https://github.com/LdDl/cnns/blob/master/convolutional_layer.go#L104 can call panic when some indices mismatch detected

Additional context
nope

Hello,

How can I use CNN with Goroutines (parallelism)? Can it be used like that?

Thanks for answer.

Is your feature request related to a problem? Please describe.
It would be cool to have pretty print for network's architecture

Describe the solution you'd like and provide pseudocode examples if you can
See ref. https://graphviz.org
I haven't dived into graphviz stuff yet
Prepare text-based, which can be used on any Graphviz web-viewers.

Describe alternatives you've considered and provide pseudocode examples if you can
nope

Additional context
There are plenty of web-viewers to prepare pdf/png/svg images from graphivz text-based format.
So it could be easier to work on docs and examples.