Sparse Times/Parameter about cntk HOT 6 CLOSED

In most cases users would have sparse input but dense weight matrix. In the current version, sparse inputs with dense weight matrix is already supported. Can you tell us your scenario so that we can understand better what kind of sparse parameters are needed?

Thanks,
Dong

from cntk.

Hello Dong, I see. I saw the sparse input code.

Sure, so its a binary segmentation task where I wanna train a CNN end-to-end, i.e. one image = one sample. I also wanna have some kind of "global context", meaning that local layers (like a convolution, in a fully convolutional network) would not help me that much.

After the regular convolution-pooling layers, I could use a dense layer (fully connected) as my last layer but there would be a lot of weights in this case. So I wanna substitute this layer by a layer that follows a specific pattern of connections and this pattern is not rectangular so I can't use a convolution layer (or any other standard layer) to create it.

I was able to created this layer in theano, but there's no GPU implementation of what they call "sparse structured dot".

As for the pattern, it is inspired in semi-global stereo matching methods.

Best regards,
Caio Mendes

from cntk.

Is there a pointer to a sparse input example? I was looking at the UCI Fast Reader docs but didn't see how to configure and format for sparse inputs.

from cntk.

Bing DSSM tasks use it. We were talking about making this an official test case and sample in the public repo.

Dong, do you happen to know if we already have DSSM in our set of samples? I don’t see it.

From: Robert Sim [mailto:[email protected]]
Sent: Tuesday, February 9, 2016 13:42
To: Microsoft/CNTK [email protected]
Subject: Re: [CNTK] Sparse Times/Parameter (#22)

Is there a pointer to a sparse input example? I was looking at the UCI Fast Reader docs but didn't see how to configure and format for sparse inputs.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/22#issuecomment-182089390.

from cntk.

I think it’s not part of the examples. I have asked Bhaskar and Ryan whether they can provide an example.

Thanks,

Dong Yu (俞栋)

From: Frank Seide [mailto:[email protected]]
Sent: Tuesday, February 9, 2016 4:33 PM
To: Microsoft/CNTK [email protected]
Cc: Dong Yu [email protected]
Subject: Re: [CNTK] Sparse Times/Parameter (#22)

Bing DSSM tasks use it. We were talking about making this an official test case and sample in the public repo.

Dong, do you happen to know if we already have DSSM in our set of samples? I don’t see it.

From: Robert Sim [mailto:[email protected]]
Sent: Tuesday, February 9, 2016 13:42
To: Microsoft/CNTK <[email protected] mailto:[email protected] >
Subject: Re: [CNTK] Sparse Times/Parameter (#22)

Is there a pointer to a sparse input example? I was looking at the UCI Fast Reader docs but didn't see how to configure and format for sparse inputs.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/22#issuecomment-182089390.

—
Reply to this email directly or view it on GitHub #22 (comment) . https://github.com/notifications/beacon/AL5PcypvifDTLEQhWMqCjwolz-KNxetaks5pinzZgaJpZM4HMFXt.gif

from cntk.

We are waiting to switch to the new CNTKTextReader reader first since we don't have an encoder on github for the readers that were used for it.

Clemens

Am 10.02.2016 um 01:37 schrieb Dong Yu <[email protected]mailto:[email protected]>:

I think it's not part of the examples. I have asked Bhaskar and Ryan whether they can provide an example.

Thanks,

Dong Yu (??)

From: Frank Seide [mailto:[email protected]]
Sent: Tuesday, February 9, 2016 4:33 PM
To: Microsoft/CNTK <[email protected]mailto:[email protected]>
Cc: Dong Yu <[email protected]mailto:[email protected]>
Subject: Re: [CNTK] Sparse Times/Parameter (#22)

Bing DSSM tasks use it. We were talking about making this an official test case and sample in the public repo.

Dong, do you happen to know if we already have DSSM in our set of samples? I don't see it.

From: Robert Sim [mailto:[email protected]]
Sent: Tuesday, February 9, 2016 13:42
To: Microsoft/CNTK <[email protected]mailto:[email protected] mailto:[email protected] >
Subject: Re: [CNTK] Sparse Times/Parameter (#22)

Is there a pointer to a sparse input example? I was looking at the UCI Fast Reader docs but didn't see how to configure and format for sparse inputs.

Reply to this email directly or view it on GitHubhttps://github.com//issues/22#issuecomment-182089390.

Reply to this email directly or view it on GitHub #22 (comment) . https://github.com/notifications/beacon/AL5PcypvifDTLEQhWMqCjwolz-KNxetaks5pinzZgaJpZM4HMFXt.gif

Reply to this email directly or view it on GitHubhttps://github.com//issues/22#issuecomment-182148471.

from cntk.