deeptrackai / deeplay Goto Github PK

Continuation from #13

My suggestion would to have a subclassed

class GraphSequentialBlock(SequentialBlock):
....

where the interleaving of inputs is handled in the forward call. @JesusPinedaC thoughts?

When wrapping torch.nn.GRU, LSTM or RNN layers in DL Layers, the attributes are fetched as "args" and "kwargs" and the DLModule cannot be built. I.e. the following layerList:

blocks = LayerList()
rnn = Layer(torch.nn.GRU,1,1,1,0)
act = Layer(torch.nn.Identity)
blocks.append(LayerActivation(rnn,act))
print(rnn)

(blocks): LayerList(
(0): LayerActivation(
(layer): Layer[GRU](args=1, kwargs=1)
(activation): LayerIdentity

When built, returns the following TypeError:

rnn.build()

TypeError: RNNBase.init() got an unexpected keyword argument 'args'

In our latest pull request #31, we've added the ability to take dictionaries as inputs for the network. Now, we're figuring out how to smoothly blend layers that usually work with tensors and those set up for dictionaries.

So, here's the question: should we make it a rule that if just one layer in a Sequential model expects dictionaries, all layers roll with it and accept dictionary inputs automatically? Your thoughts on this would be super helpful!

There is a bug in nested calls to multi in blocks. Existing configurations cause errors, stating that some DeeplayModule is not attached to the root. Currently, we've attempted to fix this by removing all such configurations. This is a hotfix and should be properly fixed to address the root cause of the detached modules

I'm about to introduce a powerful new system for selecting which submodules to configure. It's based on __getitem__. The idea is to introduce powerful syntax to quickly select all items one wants to configure. I will first introduce the issue and then explain how the syntax works.

What do I need input on?

Consider the following two statements:

selection_1 = model[..., "blocks", 0:2]
selection_2 = model[..., "pool", 0:2]

Intuitively one would imagine that the first syntax would select the first two items of the list blocks. This means that slice should select the children of the previous selection.

However, that doesn't work for the second syntax. One does not want to select the first two children of every "pool" layer. One likely wants to select the first 2 "pool" layers.

I think the best solution is to have slices work select the children of the previous selection and introduce an alternative syntax for slicing the current selection directly. However, I'm not sure what that alternative syntax would be. This is what I need input on.

One idea is to have a string syntax like: model[..., "pool>0:2"]. But I'm very open to other suggestions and ideas.

What can be done

Consider the cnn. It has a child blocks with several children. Each child has the attributes pool, layer, activation, normalization.

• cnn["blocks"] would select the list of blocks. Equivalent to cnn.blocks
• cnn["blocks", 0] would select the first block of blocks. Equivalent to cnn.blocks[0].
• cnn[..., "layer"] would select every module named layer. Equivalent to block.layer for block in cnn.blocks
• cnn["blocks", 0, ...] would select every child of the first block. Equivalent to [cnn.blocks[0].pool, cnn.blocks[0].layer, cnn.blocks[0].activation, cnn.blocks[0].normalization]

you can of course mix and match the syntaxes. cnn[:, :, "layer"], cnn[..., "layer", ...] etc.

The method module.create() does't overwrite module and requires mymodule = module.create() whereas module.build(), module.configure() and module.replace() do.
It makes sense, since create must return a new instance but I believe it can easily produce mistakes.

There seem to be some problems with the RNN:

• during training the log messages appear empty as "...."
• the dropout as parameter seems not to be working correctly

See the example rnn.py in DeepTrackAI/DeepLearningCrashCourse#25

According to PEP 8, module names should be lowercase. Shall we change this consistently in deeplay?

The problem is that argspec = self.get_argspec() in dl.Layers, when applied on the torch recurrent layer, does not correctly access the init input parameters. The reason is that the init method of this family of layers is taged as @overload which makes it difficult to access positional arguments correctly.

It is not possible to configure the layers of a DeeplayModule inside the class definition of another DeeplayModule, i.e. this works in general:

rnn = RecurrentNeuralNetwork(in_features=10,hidden_features=[20,30,40],out_features=50)
rnn.blocks.layer.configure(torch.nn.LSTM)

But this does not:

class RNNModel(DeeplayModule):
    def __init__(self):
        super().__init__()
        rnn=RecurrentNeuralNetwork(in_features=10,hidden_features=[20,30,40],out_features=50)
        rnn.blocks.layer.configure(torch.nn.LSTM)

No properties or attributes of the internal DeeplayModule, i.e. blocks, layers, etc, can be accessed.

After some recent PRs, the requirements file is out of date. Looks like some of the packages that are used for lodestar application / model are not listed here.

After discussing with some of you, we agreed that there is a need for a simpler and more standard way to define final models in Deeplay. This would allow users to directly specify model attributes or layers during model definition.

The proposed solution is to create a folder within Deeplay called "models." Here, we would define functions that implement standard models. This way, users, especially newcomers, can easily call and use models without needing to delve into more complex Deeplay functionalities.

I believe this approach makes sense for the book and improves user ease. It also helps to clarify the distinction between "components" and "models" in Deeplay, which is currently somewhat ambiguous.

Here's an example for an RNN:

def RecurrentNN(
    in_features,
    out_features,
    hidden_sizes=[...],
    bidirectional=False,
    layer_type="LSTM",
   # more paremeters
):
    if layer_type == "LSTM":
        layer = nn.LSTM
    else:
        # Handle other layer types if needed

    rnn = dl.components.RecurrentNeuralNetwork(
        in_features,
        hidden_sizes
        # other parameters
    )
    rnn.blocks.layer.configure(layer, bidirectional=bidirectional, #other parameters)

    dense_top = dl.components.MultiLayerPerceptron(#parameters)

    return dl.sequential(rnn, dense_top)

The function would return a DeeplayModule that remains customizable for more advanced users!

What do you think @giovannivolpe @Henrik-KM @cmanzo @BenjaminMidtvedt @HarshithBachimanchi ?

I've encountered a problem where, if I create a model prior to starting a deeplay application, application.logs remains empty following inference. Logs are instead stored in model.logs. For example:

import torch
import torch.nn as nn

import deeplay as dl

model = dl.LayerActivation(
    layer=dl.Layer(nn.Linear, 1, 10),
    activation=dl.Layer(nn.Sigmoid),
)
model.layer.log_output("x_transformed")
model = model.create()

classifier = dl.BinaryClassifier(
    model=model,
    optimizer=dl.Adam(lr=0.01),
).create()

x = torch.randn(2, 1)
y = classifier(x)

print(classifier.logs) # same for classifier.model.logs
# {}

print(model.logs["x_transformed"].shape)
# (2, 10)

if model = model.create() is skipped, then classifier.logs is populated while model.logs does not exist. Is this an expected behavior?