Some issues when combining `deep_architect` and `ray.tune` about deep_architect HOT 4 OPEN

Hey there - this seems like a problem with Ray's documentation being unclear.

What if you just did:

class SimpleClassifierTrainable(tune.Trainable):
    def _setup(self, config):
        use_cuda = torch.cuda.is_available()
        self.device = torch.device("cuda" if use_cuda else "cpu")
        self.batch_size = config["batch_size"]
        self.learning_rate = config.get("lr", 0.01)
        self.train_loader, self.val_loader = get_dataloaders(self.batch_size)
        ##############################
		# CREATE MODEL HERE
        model = sample_model(in_features=784, num_classes=10))
        self.model = model.to(self.device)
        ###############################
        self.criterion = nn.CrossEntropyLoss()
        self.optimizer = optim.Adam(self.model.parameters(),
                                    lr=self.learning_rate)

from deep_architect.

Hi Iacopo. Apologies for the delay. Unfortunately, I haven't been able to dedicate much time to the DeepArchitect lately, but I'm looking to resume soon. I'm curious about whether how far did you go with DeepArchitect in your work. I'm not familiar with Ray but happy to integrate some functionality as it seems widely adopted now. I don't see any inherent problems in using DeepArchitect with Ray, provided that Ray does not need too much information about the workload that it is running (e.g., the exact architecture).

from deep_architect.

Hi @negrinho

No need to apologize :-) didn't have much time to work on this either.

The idea would be to be able to use DeepArchitect functions as a sampler for a Pytorch (or TF) model in the tune.run parameter config (see here: https://gist.github.com/iacolippo/3f815fa90c254f7a065bdc446406233a#file-ray_deep_architect_ex2-py-L201). This would make it really easy to scale an architecture search from a single machine to a cluster.

I will have a person working on a closely related project starting in October, so I will hopefully be able to give more detailed information soon.

from deep_architect.

from deep_architect.