piiswrong / dec Goto Github PK

Hi
I have import error when running (python make_mnist_data.pay) , it is related to importing leveldb library. Anyone have similar issue?

Inside the caffe folder
make -j8 && make pycaffe

When I run the code that you gave, I met some errors like data layer doesn't have the field seek and doesn't have the MULTI_T_LOSS type. Can you give me some advice how I solve these?

Hi,

I'm attempting to run pretrain.py on the mnist dataset out of the box, but within the function pretrain_main(), I hit an error affiliated with Cuda. The trace is below:

F0203 19:28:13.953368  9487 math_functions.cu:394] Check failed: status == CURAND_STATUS_SUCCESS (201 vs. 0)  CURAND_STATUS_LAUNCH_FAILURE
*** Check failure stack trace: ***
    @     0x7fb2a5554daa  (unknown)
    @     0x7fb2a5554ce4  (unknown)
    @     0x7fb2a55546e6  (unknown)
    @     0x7fb2a5557687  (unknown)
    @           0x4ae358  caffe::caffe_gpu_rng_uniform()
    @           0x4faff4  caffe::DropoutLayer<>::Forward_gpu()
    @           0x46a16b  caffe::Net<>::ForwardFromTo()
    @           0x46a597  caffe::Net<>::ForwardPrefilled()
    @           0x4aadae  caffe::Solver<>::Solve()
    @           0x417f62  train()
    @           0x4118f1  main
    @     0x7fb2a1f69f45  (unknown)
    @           0x416997  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

I'm running from the docker image. Maybe it has something to do with the cuda version? The image uses CUDA 7.0.

I think there's a bug in the code here. You're using data you're already training on as test data.

On line 446, the train data is using all the samples (i.e. X = X[:N]), where N is the number of samples.
On line 447, the test data is using the last 20% of the samples (i.e. X=X[4/5*N:N]).

I think this is a bug.

hi,

when i use my own images to pretrain the encodes , i found that at the process of pretrainMain function after first 1000 iteration the loss become nan for the first layer training.

At the fist layer pretrain the net includes all the encoder layers and the last layer of decoder, then the loss will be the euclidean distance between input data and the output of decoder.

Is this loss ok?

Hello,
I followed all the proper instructions to get the data (./get_data.sh), then (python make_mnist_data.py ) and now I get this error when running:

[ad@turing dec]$ python dec.py mnist
/usr/lib64/python2.7/site-packages/sklearn/lda.py:6: DeprecationWarning: lda.LDA has been moved to discriminant_analysis.LinearDiscriminantAnalysis in 0.17 and will be removed in 0.19
"in 0.17 and will be removed in 0.19", DeprecationWarning)
Traceback (most recent call last):
File "dec.py", line 768, in
DisKmeans(db, lam)
File "dec.py", line 684, in DisKmeans
ret, net = extract_feature('net.prototxt', 'exp/'+db+'/save_iter_100000.caffemodel', ['output'], N, True, 0)
File "dec.py", line 576, in extract_feature
caffe.Net.set_phase_train()
AttributeError: type object 'Net' has no attribute 'set_phase_train'

Could you help me understand what I need to do?

Running an experiment with reutersidf10k dataset via python dec.py reutersidf10k fails with an error

I0721 18:03:54.192920 24690 net.cpp:67] Creating Layer data
I0721 18:03:54.192975 24690 net.cpp:358] data -> data
I0721 18:03:54.193002 24690 net.cpp:96] Setting up data
I0721 18:03:54.193032 24690 data_layer.cpp:45] Opening leveldb reutersidf10k_total
python: db_iter.cc:68: virtual leveldb::Slice leveldb::<unnamed>::DBIter::value() const: Assertion `valid_' failed.

The same error occurs with the mnist and the reutersidf datasets. I haven't tried with the stl dataset.

I compiled the Caffe library provided in this repo, downloaded the dataset, and ran python make_reuters_data.py as given in the instruction.

The error happens when iter_->value() is called in: https://github.com/piiswrong/dec/blob/master/caffe/src/caffe/layers/data_layer.cpp#L122

It seems https://github.com/piiswrong/dec/blob/master/caffe/src/caffe/layers/data_layer.cpp#L53 creates an invalid iterator; putting a line CHECK(iter->Valid()) immediately after it fails.

My leveldb version is 1.0.7

Hello,
when running 'python dec.py mnist' I'm getting this problem:

Traceback (most recent call last):
File "dec.py", line 768, in
DisKmeans(db, lam)
File "dec.py", line 721, in DisKmeans
write_db(weight, np.zeros((weight.shape[0],)), 'train_weight')
File "dec.py", line 552, in write_db
db = leveldb.LevelDB(fname)
leveldb.LevelDBError: IO error: lock train_weight/LOCK: already held by process

Regards

Hi,

I am implementing dec and would like to try it on an unlabeled dataset of images. However, I noticed that labels are being used in the 'dec.py' script. Why are the labels required if this is unsupervised learning? How can I implement this for unlabeled dataset?

Thanks!

Hi Junyuan,
Basic, I downloaded two datasets: mnist and reuters by mnist(reuters)/get_data.sh then pyhton make_mnist(reuters)_data.py.
However, things is not so easy as I thought.
1. Parameter initialization Problem: Claimed " You can use dec/pretrain.py to train your own autoencoder. " on the https://github.com/piiswrong/dec
but in my practice, python pretrain.py came out of "segment fault on libleveldb.so when load data", which data is default on DB=mnist, and download by mnist/get_data.sh then pyhton make_mnist_data.py
2. Two(mnist/reuters10k) datasets are segmented fault on training using dec.py.
3. Since reutersidf is ok on training, I am looking forward to seeing the amazing result, but the acc is only about 0.35 not a impressive promance of 0.7, during the experiment procedure I assumed the reutersidf data is pretrained well.
Could you double check on if datesets is pretrained well ? Although, the loss is very low.
4. I have a confuse on the training process: during diff iteration, diff training data is loaded by seek operator, after the current iteration training(each has a 20 times caffe iterations), init weight and init.model are updated, then a fine-tuning of next iteration diff data is processed, is it right?

  Dtype qij = cpu_inv_sigma_prod[j]*std::pow(cpu_mask[i*N_+j], alpha_exp)/norm;
  cpu_proba[i*N_ + j] = qij;
  Dtype pij = cpu_label[i*N_+j];
  //Dtype pij = qij*qij/sqr_norm;
  cpu_mask[i*N_+j] = alpha_weight*cpu_mask[i*N_+j]*(pij - qij);
  cpu_coefm[i] += cpu_mask[i*N_+j];
  cpu_coefn[j] += cpu_mask[i*N_+j];
  loss += pij * std::log(pij/qij);

what is cpu_label? Is it the label information?

I have followed the way you mentioned at the "Useage" to pretrain the mnist encoder, but it failed to get mnist leveldb's lock.

Is there anyone who meet this kind of problem ?

Any ideas will be appreciated !

When I extract features of STL10 dataset, I ran make_stl_data.py which use setup_features.py.
But I can't find the file "features.pyx" in it.

setup(
name = "features",
ext_modules = cythonize("features.pyx"),
include_dirs = [np.get_include()]
)

and I got a
ValueError: 'features.pyx' doesn't match any files

So did I miss some steps and how can I fix it?
Thanks

Hello, great work!

In the paper and the code, you refer to an "annealing speed" lambda ranging over 10*(2^i) for i = 0, 1, ..., 8.

What does this refer to? Is it a learning rate annealer? Do you mean this is how often you reduce the learning rate?

I tried to read the code but could not figure out what seek does (the lambda parameter is used to update this data seek thing).

Thank you very much!

Hi ,
I run the program with my own data. I find that the first result is great having the highest accuracy. After finetuning the network, the result becomes worse.
I don't know the reason. Could you give me some suggestions?

Thanks

Hi @piiswrong,

I am having trouble to compile your caffe version. It seems that there some incompatibility with other libraries that it links to. Would please inform which version of the libraries you used? Like boost and opencv versions.

Thanks,

Doug

@piiswrong I'm trying to test dec in our environment but we use the latest version of caffe.
Could you kindly share what changes are made to caffe?
If would be great if we can known the commit hash of your base version so that a tree diff is possible.

Thank you!

Hi @piiswrong ,

I have change the path of caffe in dec.py as the path where I installed cafe. When I run python dec.py mnist, it reports "type object 'Net' has no attribute 'set_phase_train'", it seems to be caused by different versions of caffe. But the caffe version in your directory seems not be able to been built. Do you have any idea how I should solve this problem?

Thanks,
Rui

Hello,

In line no. 8, 'libatlas-base-dev' is mistyped as 'libatlas-base-de'. Please update the dockerfile.

Thank you

How to fix this problem?
When I do "python make_stl_data1.py", I got a errors. The error is:

root@hu-N551JW:/home/hu/dec-master/dec# python make_stl_data1.py
Building HOG feature extractor...
running build
running install
running build
running install_egg_info
Writing /usr/local/lib/python2.7/dist-packages/features-0.0.0.egg-info
Preparing stl data. This could take a while...
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers
while thread._state == RUN or (pool._cache and thread._state != TERMINATE):
AttributeError: '_MainThread' object has no attribute '_state'

Hi
Can you provide working link for the experimental implementation of DEC code in MXnet, as the one in the readme file is broken.

Thank you

Is the caff code given by the paper using CPU or GPU?
Thank you.

recently, I read your paper. And I think in is very cool.
I do as follows:

1. I installed caffe on windows 10.
2. git clone your code.
3. cd mnist; ./get_data.sh
4. cd dec; python make_mnist_data.py

then I got the error information:
H:\project\windows_caffe\dec\dec\dec>python make_mnist_data.py
Traceback (most recent call last):
File "make_mnist_data.py", line 1, in
import dec
File "H:\project\windows_caffe\dec\dec\dec\dec.py", line 12, in
import caffe
File "../caffe/python\caffe__init__.py", line 1, in
from .pycaffe import Net, SGDSolver
File "../caffe/python\caffe\pycaffe.py", line 10, in
from ._caffe import Net, SGDSolver
ImportError: No module named _caffe

Could you please tell me what's wrong with it . Thank you very much.

Hello,
Your link https://mxnet.readthedocs.org/en/latest/build.html does not work. Could you please provide the correct and updated link for the readthedocs?
Thank you!

Hi, do you have any idea about why the author define the target distribution in such a format?

The author defined the target distribution because it can 1) improve cluster purity, 2) put more emphasis on data points assigned with high confidence, and 3) normalize the loss contribution of each centroid to prevent large clusters from distorting the hidden feature space.

But I feel confused about why such a definition can achieve the above goals? I would be extremely grateful if you could give me an answer.

Thanks for your code.
I followed your tips in Readme.md. And I also run dec.py by your dec caffe.But it occured this error. How can I solve this?