Failed to include caffe_pb2 about channel-pruning HOT 19 CLOSED

@tianyuandu @yihui-he protobuf problem fixed, because of the c++ version pb not matching the python version, reinstall them has fixed. Thanks two guys!

from channel-pruning.

@tianyuandu @LearnerInGithub Hello， I downloaded the latest version of caffe, installed anaconda, and installed both python 2.7 and 3.5. The protoc in usr/lib is version 2.5. If I don't install(conda install) protoc 3.2 (or later) in python and use protoc2.5 , then caffe can be made all. But once protoc3.2 (or later) is installed in python, i cannot make all caffe. I hope you can provide some advice, thank you.

from channel-pruning.

I encountered this before, however solved it with upgrade to protobuf 3.2
my version:

$ pip3 show protobuf
Name: protobuf
Version: 3.2.0
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: [email protected]
Author-email: [email protected]
License: New BSD License
Location: /usr/local/lib/python3.5/dist-packages
Requires: setuptools, six

If you still suffer this problem, the alternative is install my ImageData layer to your own Caffe.
Replace the following parts:

/src/caffe/layers/image_data_layer.cpp
/include/caffe/layers/image_data_layer.hpp
/src/caffe/util/io.cpp
/include/caffe/util/io.hpp
src/caffe/data_transformer.cpp

and https://github.com/yihui-he/caffe-pro/blob/master/src/caffe/proto/caffe.proto#L783-L814

from channel-pruning.

whether these parts still works well in current caffe is not granted

from channel-pruning.

According to BVLC/caffe#2464, this problem is solved in latest Caffe version. Since I can't reproduce this problem, I've merge my Caffe to the latest BVLC/caffe. Please update and try remake.

from channel-pruning.

@yihui-he Thanks for your suggestion.
I've tried the same setting you mentioned with caffe in your repository but still didn't work.
Then I cloned the latest Caffe (before official Caffe solved this problem) and replaced the modified files, and built with protobuf 3.2 and python3.5, I can successfully include caffe_pb2 and run your code.

from channel-pruning.

@tianyuandu I clone the official caffe, and replaced the files you mentioned, but still face this problem. Does anything I'm missing? I installed the protobuf 3.2 and python3 is Python3.5.

from channel-pruning.

Can you built official caffe successfully in the first place? If not, please ask this question in Caffe.

from channel-pruning.

@yihui-he Yes, I can compile caffe successfully, problem will happy while I run the command:
python3 train.py -action c3 -caffe 0

from channel-pruning.

@LearnerInGithub
I'm not sure what your error message is. You can paste error message here.
After built official caffe, can you import caffe and import caffe.proto?
If import caffe worked and import caffe.proto failed, you can try to add these codes at the start of train.py after from __future__ import print_function:
import sys
sys.path.insert(0, "path/to/caffe/python")

Hope that can help you.

from channel-pruning.

@tianyuandu Thanks for your reply, when I import caffe, error will occurs:

Traceback (most recent call last):
File "", line 1, in
File "/channel-pruning/caffe/python/caffe/init.py", line 1, in
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "/cephfs/person//channel-pruning/caffe/python/caffe/pycaffe.py", line 15, in
import caffe.io
File "/channel-pruning/caffe/python/caffe/io.py", line 2, in
import skimage.io
File "/usr/local/lib/python3.5/site-packages/skimage/io/init.py", line 15, in
reset_plugins()
File "/usr/local/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 93, in reset_plugins
_load_preferred_plugins()
File "/usr/local/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 73, in _load_preferred_plugins
_set_plugin(p_type, preferred_plugins['all'])
File "/usr/local/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 85, in _set_plugin
use_plugin(plugin, kind=plugin_type)
File "/usr/local/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 255, in use_plugin
_load(name)
File "/usr/local/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 299, in _load
fromlist=[modname])
File "/usr/local/lib/python3.5/site-packages/skimage/io/_plugins/matplotlib_plugin.py", line 3, in
import matplotlib.pyplot as plt
File "/usr/local/lib/python3.5/site-packages/matplotlib/pyplot.py", line 37, in
from matplotlib.figure import Figure, figaspect
File "/usr/local/lib/python3.5/site-packages/matplotlib/figure.py", line 40, in
from matplotlib.axes import Axes, SubplotBase, subplot_class_factory
File "/usr/local/lib/python3.5/site-packages/matplotlib/axes/init.py", line 4, in
from ._subplots import *
File "/usr/local/lib/python3.5/site-packages/matplotlib/axes/_subplots.py", line 10, in
from matplotlib.axes._axes import Axes
File "/usr/local/lib/python3.5/site-packages/matplotlib/axes/_axes.py", line 24, in
import matplotlib.dates as _ # <-registers a date unit converter
File "/usr/local/lib/python3.5/site-packages/matplotlib/dates.py", line 125, in
from dateutil.rrule import (rrule, MO, TU, WE, TH, FR, SA, SU, YEARLY,
File "/usr/local/lib/python3.5/site-packages/dateutil/rrule.py", line 55
raise ValueError, "Can't create weekday with n == 0"
^
SyntaxError: invalid syntax

from channel-pruning.

@LearnerInGithub
Actually I haven't meet this problem.
I just found a similar error in http://blog.csdn.net/yzf0011/article/details/73250107
You can try this:sudo pip3 install --upgrade python-util

from channel-pruning.

@tianyuandu I execute this command: sudo pip3 install --upgrade python-util:
Requirement already up-to-date: python-util in /usr/local/lib/python3.5/site-packages
Requirement already up-to-date: docopt in /usr/local/lib/python3.5/site-packages (from python-util)

However, the import caffe still have problem:

import caffe
Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
File "", line 1, in
File "/cephfs/person/mingliu/channel-pruning/caffe/python/caffe/init.py", line 4, in
from .proto.caffe_pb2 import TRAIN, TEST
File "/cephfs/person/mingliu/channel-pruning/caffe/python/caffe/proto/caffe_pb2.py", line 831, in
options=_descriptor._ParseOptions(descriptor_pb2.FieldOptions(), _b('\020\001')), file=DESCRIPTOR),
TypeError: init() got an unexpected keyword argument 'file'

from channel-pruning.

@LearnerInGithub Seem like caffe_pb2.py has some problem. Is your protobuf 3 install by pip3?
And when you are building caffe, the protobuf in system was chosen and maybe the version is mismatch with pip's protobuf.
I used cmake to build caffe so that I can see the exact version of protobuf, boost and cuda. You can also try this.

from channel-pruning.

@tianyuandu Sounds a good idea, I will try it. Thanks you!

from channel-pruning.

@LearnerInGithub ,you mentioned you reinstall protobuf and python ? which version of protobuf and python you used? I had the same problem and my protobuf is 3.5.0 ,python version is 2.7.13 of anaconda. what should I do to fix this problem?Thanks .

from channel-pruning.

solved with sudo pip3 install sklearn-contrib-lightning

from channel-pruning.

@tianyuandu @LearnerInGithub Hello， I downloaded the latest version of caffe, installed anaconda, and installed both python 2.7 and 3.5. The protoc in usr/lib is version 2.5. If I don't install(conda install) protoc 3.2 (or later) in python and use protoc2.5 , then caffe can be made all. But once protoc3.2 (or later) is installed in python, i cannot make all caffe. I hope you can provide some advice, thank you.

i am facing same problem,have you solved the problem?

from channel-pruning.

I have solved this problem. I update protobuf from v2.5.0 to v3.2.0, it seems work! ps: I use python 3.5.0.

from channel-pruning.