Re-implement CVPR2017 paper: "dense captioning with joint inference and visual context" and minor changes in Tensorflow. (mAP 8.296 after 500k iters of training)
License: MIT License
Hi, can you tell me why you use the training data to build vocabulary and not use the complete data?
Hi
I'm trying to training, but this error comes out.
ImportError: /home/bg/BG/densecap-tensorflow/tools/../lib/nms/gpu_nms.so: undefined symbol: _Py_ZeroStruct
i'm using python3.
How should i fix it?
I cannot train the model. Can you provide the pretrain model?
Thanks in advance
Hi, I am running a code on ubuntu server. I uploaded this project in my home directory and trying to generate captions on custom images using
$ bash scripts/dense_cap_demo.sh output/ckpt output/ckpt/vocabulary.txt
I created the data/demo and output file in this project's directory i-e
/home/densecap-tensorflow-master/data/demo
When I run
bash scripts/dense_cap_demo.sh output/ckpt output/ckpt/vocabulary.txt
it gives me following error
+ set -e
+ ckpt=output/ckpt
+ vocab=output/ckpt/vocabulary.txt
+ '[' -d /home/joe ']'
+ python ./tools/demo.py --ckpt output/ckpt --cfg scripts/dense_cap_config.yml --vocab output/ckpt/vocabulary.txt --set TEST.USE_BEAM_SEARCH False EMBED_DIM 512 TEST.LN_FACTOR 1. TEST.RPN_NMS_THRESH 0.7 TEST.NMS 0.3
Traceback (most recent call last):
File "./tools/demo.py", line 28, in <module>
import tensorflow as tf
File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 99, in <module>
from tensorflow_core import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/__init__.py", line 34, in <module>
from tensorflow._api.v1 import autograph
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/_api/v1/autograph/__init__.py", line 22, in <module>
from tensorflow._api.v1.autograph import experimental
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/_api/v1/autograph/experimental/__init__.py", line 10, in <module>
from tensorflow.python.autograph.core.converter import Feature
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/autograph/__init__.py", line 35, in <module>
from tensorflow.python.autograph import operators
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/autograph/operators/__init__.py", line 40, in <module>
from tensorflow.python.autograph.operators.control_flow import for_stmt
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/autograph/operators/control_flow.py", line 65, in <module>
from tensorflow.python.autograph.operators import py_builtins
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/autograph/operators/py_builtins.py", line 30, in <module>
from tensorflow.python.data.ops import dataset_ops
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/data/__init__.py", line 25, in <module>
from tensorflow.python.data import experimental
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/data/experimental/__init__.py", line 89, in <module>
from tensorflow.python.data.experimental.ops.batching import dense_to_sparse_batch
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/data/experimental/ops/batching.py", line 20, in <module>
from tensorflow.python.data.ops import dataset_ops
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 70, in <module>
from tensorflow.python.training.tracking import tracking
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/training/tracking/tracking.py", line 29, in <module>
from tensorflow.python.training.tracking import data_structures
File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/training/tracking/data_structures.py", line 30, in <module>
from .....third_party import wrapt
ValueError: Attempted relative import beyond toplevel package
how can i solve this issue. I cannot create directories in root folder as there are other users as well. I have access to my home directory only.
I found the error line is from 'lib.utils.cython_bbox import bbox_overlaps'.
And there is no 'cython_bbox' in 'lib.utils'.
How can I fix this problem?
Thank you very much.
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/dense_cap/train.py", line 356, in train_model
blobs = self.data_layer.forward()
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/fast_rcnn/layer.py", line 99, in forward
blobs = self._get_next_minibatch()
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/fast_rcnn/layer.py", line 95, in _get_next_minibatch
return get_minibatch(minibatch_db)
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/fast_rcnn/minibatch.py", line 41, in get_minibatch
im_blob, im_scales, roidb = _get_image_blob(roidb, random_scale_inds)
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/fast_rcnn/minibatch.py", line 101, in _get_image_blob
cfg.TRAIN.MAX_SIZE)
File "/raid/wwb/densecap-tensorflow-master/tools/../lib/utils/blob.py", line 38, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
i'm interested in captioning. so i've tried to train.
but following message was output.
File "./tools/train_net.py", line 32, in
from lib.datasets.factory import get_imdb
File "/DataStorage/dhshin/image_caption/densecap-tensorflow/tools/../lib/datasets/factory.py", line 11, in
from visual_genome import visual_genome
File "/DataStorage/dhshin/image_caption/densecap-tensorflow/tools/../lib/datasets/visual_genome.py", line 20, in
from lib.datasets.imdb import imdb
File "/DataStorage/dhshin/image_caption/densecap-tensorflow/tools/../lib/datasets/imdb.py", line 24, in
from lib.utils.bbox import bbox_overlaps
ImportError: /DataStorage/dhshin/image_caption/densecap-tensorflow/tools/../lib/utils/bbox.so: failed to map segment from shared object
i refered
"bash scripts/dense_cap_train.sh [dataset] [net] [ckpt_to_init] [data_dir] [step]"
And my command is
"bash scripts/dense_cap_train.sh visual_genome_1.2 res101 /DataStorage/dhshin/image_caption /DataStorage/dhshin/ic_dataset/visual_genome/VG_100K/ 1"
dataset: visual_genome_1.2
net: res101
ckpt_to_init: /DataStorage/dhshin/image_capton
data_dir: /DataStorage/dhshin/ic_dataset/visual_genome/VG_100K
resnet101.ckpt file exists in"/DataStorage/dhshin/image_capton"
image files exist in "/DataStorage/dhshin/ic_dataset/visual_genome/VG_100K"
i don't know why this error occure. i need your help.
Thank you for your kind answers.
@InnerPeace-Wu hi, i am trying to train densecap-tensorflow project on my own dataset.But i 'm encountered a weired problem: the caption loss has been dropped from 0.4 to 0.1 by 250k iterations; but the caption result is way out of good; as i have increased the total label class from 2 to 8, so i guess that added label classes may lead to this problem
Could you give me some suggestions? Thanks!
Ｈｉ,
After the preparing data process, I train the model with the command:
"bash scripts/dense_cap_train.sh [dataset] [net] [ckpt_to_init] [data_dir] [step]"
An error appears:
Traceback (most recent call last):
File "/data/yanghui/.pycharm_helpers/pydev/pydevd.py", line 1664, in
main()
File "/data/yanghui/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/data/yanghui/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/yanghui/densecap-tensorflow/tools/train_net.py", line 214, in
main()
File "/home/yanghui/densecap-tensorflow/tools/train_net.py", line 168, in main
imdb, roidb = get_roidb_limit_ram(args.imdb_name)
File "/home/yanghui/densecap-tensorflow/tools/train_net.py", line 123, in get_roidb_limit_ram
roidb = imdb.roidb
File "/home/yanghui/densecap-tensorflow/lib/datasets/imdb.py", line 82, in roidb
self._roidb = self.roidb_handler()
File "/home/yanghui/densecap-tensorflow/lib/datasets/visual_genome.py", line 144, in gt_roidb
gt_roidb = self.gt_roidb_limit_ram()
File "/home/yanghui/densecap-tensorflow/lib/datasets/visual_genome.py", line 196, in gt_roidb_limit_ram
dictionary = self._load_vg_annotation(idx)
File "/home/yanghui/densecap-tensorflow/lib/datasets/visual_genome.py", line 273, in _load_vg_annotation
with open(self.region_imset_path + '/%s.json' % index, 'r') as f:
IOError: [Errno 2] No such file or directory: '/home/yanghui/densecap-tensorflow/VG/1.2/train_gt_regions/2375025.json'
I check the directory of '/home/yanghui/densecap-tensorflow/VG/1.2/', there is no folder 'train_gt_regions'. How does this folder generate?
Hi,
I wanted to ask about the LSTM implementation as it looks a bit different to the original implementation, for example it is actually not necessary to use an tf.nn.dynamic_rnn for variable sized sequences, as the inputs are already padded to have the same length, right?
Hi.
When reading your code, I confused that why the context feature is implemented as in the code: $ROOT/lib/nets/network/py line:340
global_feature, region_features = tf.split(region_features, [1, -1], axis=0)
around codes as follows
region_features = slim.fully_connected(fc7, cfg.EMBED_DIM,
weights_initializer=initializer,
trainable=is_training,
activation_fn=None, scope='region_features')
if cfg.CONTEXT_FUSION:
# global_feature [1, cfg.EMBED_DIM(512)]
global_feature, region_features = tf.split(region_features, [1, -1], axis=0)
batch_size = tf.shape(region_features)[0]
# global_feature_rep [batch_size(256), cfg.EMBED_DIM(512)]
global_feature_rep = tf.tile(global_feature, [batch_size, 1])
gfeat_lstm_cell = rnn.BasicLSTMCell(cfg.EMBED_DIM, forget_bias=1.0,
state_is_tuple=True)
else:
batch_size = tf.shape(region_features)[0]
I do not understand why
thanks a lot.
Loaded.
Fix Resnet V1 layers..
Fixed.
Ckpt path: ckpt/./res50_densecap_iter_500000.ckpt
Found pickle file, restore training process.
Last snapshot iters:100000
Traceback (most recent call last):
File "./tools/train_net.py", line 214, in
main()
File "./tools/train_net.py", line 210, in main
max_iters=args.max_iters)
File "/home/liuxiaoxiao/docment/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/liuxiaoxiao/docment/tools/../lib/dense_cap/train.py", line 356, in train_model
blobs = self.data_layer.forward()
File "/home/liuxiaoxiao/docment/tools/../lib/fast_rcnn/layer.py", line 99, in forward
blobs = self._get_next_minibatch()
File "/home/liuxiaoxiao/docment/tools/../lib/fast_rcnn/layer.py", line 95, in _get_next_minibatch
return get_minibatch(minibatch_db)
File "/home/liuxiaoxiao/docment/tools/../lib/fast_rcnn/minibatch.py", line 41, in get_minibatch
im_blob, im_scales, roidb = _get_image_blob(roidb, random_scale_inds)
File "/home/liuxiaoxiao/docment/tools/../lib/fast_rcnn/minibatch.py", line 101, in _get_image_blob
cfg.TRAIN.MAX_SIZE)
File "/home/liuxiaoxiao/docment/tools/../lib/utils/blob.py", line 37, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
First of all, thank you for sharing you project! I have played with it a little bit and found out that inference is quit slow - something about 50 seconds per image. Is this ok?
I got this error when I test a image following your instruction in README.md. Did I miss something? Can you help me please?（I'm so sorry that I paste the wrong picture before）
How to run the dense_cap_test.sh？
@InnerPeace-Wu Hi, I'm trying to train according to your given step, but I met some mistake:
densecap-tensorflow bash ./scripts/dense_cap_train.sh visual_genome res101 /home/dleam/Project/graduate/densecap-tensorflow/model/res101_faster_rcnn_iter_110000 /media/dleam/c2824c7e-ccb2-43d5-9193-8c6a4bbb28d2/dleam/datasets/VG/images_processed 1
Have I done something wrong? The previous step I have done seems to be all right ... I'm looking forward to your reply, thx~
Loaded.
Fix Resnet V1 layers..
Fixed.
Ckpt path: /home/xiewei/densecap-tensorflow/slim_models/res50.ckpt
Traceback (most recent call last):
File "./tools/train_net.py", line 214, in
main()
File "./tools/train_net.py", line 210, in main
max_iters=args.max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 356, in train_model
blobs = self.data_layer.forward()
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/layer.py", line 99, in forward
blobs = self._get_next_minibatch()
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/layer.py", line 95, in _get_next_minibatch
return get_minibatch(minibatch_db)
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/minibatch.py", line 41, in get_minibatch
im_blob, im_scales, roidb = _get_image_blob(roidb, random_scale_inds)
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/minibatch.py", line 101, in _get_image_blob
cfg.TRAIN.MAX_SIZE)
File "/home/xiewei/densecap-tensorflow/tools/../lib/utils/blob.py", line 37, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
请问，如何解决
Hi,
It takes more than 1 minute to process 1 image.
I ran the bash scripts/dense_cap_demo.sh ./output/ckpt ./output/ckpt/vocabulary.txt command.
I wonder if I am doing anything wrong. I am using an aws p2.xlarge gpu. Is this normal ?
Or do have to do another configuration I am not aware of. Please any help is appreciated.
Thanks
@InnerPeace-Wu 使用resnet_v2作为基本网络，在训练时出现如下错误，希望能得到你的建议，非常感谢！！！
:~/densecap-tensorflow$ bash scripts/dense_cap_train.sh visual_genome_1.2 resnet_v2_50 /home/xiewei/densecap-tensorflow/slim_models/resnet_v2_50.ckpt /home/xiewei/densecap-tensorflow/visual_genome 1
float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type./home/xiewei/densecap-tensorflow/output/dc_conv_fixed/vg_1.2_train/home/xiewei/densecap-tensorflow/output/dc_conv_fixed/tb/vg_1.2_train/defaultCaused by op u'save_1/Assign_217', defined at:
File "./tools/train_net.py", line 221, in
main()
File "./tools/train_net.py", line 217, in main
max_iters=args.max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 325, in train_model
rate, last_snapshot_iter, stepsizes, np_paths, ss_paths = self.initialize(sess)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 236, in initialize
restorer = tf.train.Saver(variables_to_restore)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1218, in init
self.build()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1227, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1263, in _build
build_save=build_save, build_restore=build_restore)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 439, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 160, in restore
self.op.get_shape().is_fully_defined())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/state_ops.py", line 276, in assign
validate_shape=validate_shape)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 57, in assign
use_locking=use_locking, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1024] rhs shape= [2048]
[[Node: save_1/Assign_217 = Assign[T=DT_FLOAT, _class=["loc:@resnet_v2_50/postnorm/moving_variance"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](resnet_v2_50/postnorm/moving_variance, save_1/RestoreV2_217/_9)]]
Li feifei has another paper, generate coherent long sentences to describe an image using hierarchical attention method. It seems that the hierarchical attention method is extended from the dense cap method, thus I am wondering whether it is possible to implement the hierarchical attention model based on your current code.
Do you have any suggestion?
Hi,
what is the spec of the training and the testing set of the model used
Hi,
This issue seems more of stemming from coco caption tool, but would you like to comment any possible way to get around it? So, the error raises within meteor.py as follows,
Traceback (most recent call last):
File "../tools/test_net.py", line 142, in <module>
vis=args.vis, use_box_at=args.use_box_at, save_html_path=args.save_html_dir)
File "/scratch/work/xxxx/Projects/densecap/tools/../lib/dense_cap/test.py", line 481, in test_net
vg_evaluator.evaluate()
File "/scratch/work/xxxx/Projects/densecap/tools/../lib/pycocoevalcap/vg_eval.py", line 118, in evaluate
score_match, scores_match = Meteor().compute_score(gts_tokens_match, res_tokens_match, imgIds=all_keys)
File "/scratch/work/xxxx/Projects/densecap/tools/../lib/pycocoevalcap/meteor/meteor.py", line 47, in compute_score
stat = self._stat(res[i][0], gts[i])
File "/scratch/work/xxxx/Projects/densecap/tools/../lib/pycocoevalcap/meteor/meteor.py", line 96, in _stat
self.meteor_p.stdin.write('{}\n'.format(score_line))
IOError: [Errno 32] Broken pipe
It happens in a for loop given below (also in meteor.py),
for i in imgIds:
print 'img IDs:', i
assert(len(res[i]) == 1)
stat = self._stat(res[i][0], gts[i])
print 'stat:', stat
eval_line += ' ||| {}'.format(stat)
The first execution of the for loop is fine, but the second time it enters the loop, specifically, getting into stat = self._stat(res[i][0], gts[i]), then the error pops up.
I am aware of there had been posts asking the exact same issue (e.g. elliottd/satyrid#2), but just couldn't find myself a workaround after spending few hours on it. Have you ever encountered this?
Thanks in advance, and any idea would be really appreciated :)
Hi , I got an IOError [Errno 32] Broken pipe when I use dense_cap_test.sh .
It looks like a multithread error or not ?
Could you help me ?
how to get ckpt_to_init in details
According to Li Feifei's paper on densecap, they mention that they were using the bilinear sampling method, thus the gradients can be propagated backward into predicted region proposal coordinates.
It seems that your code is using stop_gradient before crop_and_resize. I am curious whether Li Feifei's strategy really matters.
How do you think about it?
Thanks!
Hello, I am a new TF-boy and help some help :)
In your tutorial,
Trainning model, Add or modify configurations in
root/scripts/dense_cap_config.yml, refer to'lib/config.py'for more configuration details.
$ cd $ROOT
$ bash scripts/dense_cap_train.sh [dataset] [net] [ckpt_to_init] [data_dir] [step]
when i try to train my model by typing ：
sudo CUDA_VISIBLE_DEVICES="0" bash scripts/dense_cap_train.sh visual_genome_1.2 res50 resnet_v1_50/res50.ckpt git/visual_genome 1
I get the ERROR:
+ set -e
+ export PYTHONUNBUFFERED=True
+ PYTHONUNBUFFERED=True
+ DATASET=visual_genome_1.2
+ NET=res50
+ ckpt_path=resnet_v1_50/res50.ckpt
+ data_dir=git/visual_genome
+ step=0
+ '[' -d /home/joe ']'
+ case $DATASET in
+ TRAIN_IMDB=vg_1.2_train
+ TEST_IMDB=vg_1.2_val
+ PT_DIR=dense_cap
+ FINETUNE_AFTER1=200000
+ FINETUNE_AFTER2=100000
+ ITERS1=400000
+ ITERS2=300000
+ '[' -d /valohai/outputs ']'
++ date +%Y-%m-%d_%H-%M-%S
+ LOG=logs/s0_res50_vg_1.2_train.txt.2018-04-07_22-09-27
+ exec
++ tee -a logs/s0_res50_vg_1.2_train.txt.2018-04-07_22-09-27
+ echo Logging output to logs/s0_res50_vg_1.2_train.txt.2018-04-07_22-09-27
Logging output to logs/s0_res50_vg_1.2_train.txt.2018-04-07_22-09-27
+ '[' 0 -lt 2 ']'
+ python ./tools/train_net.py --weights resnet_v1_50/res50.ckpt --imdb vg_1.2_train --imdbval vg_1.2_val --iters 200000 --cfg scripts/dense_cap_config.yml --data_dir git/visual_genome --net res50 --set EXP_DIR dc_conv_fixed CONTEXT_FUSION False RESNET.FIXED_BLOCKS 3
------ called with args: -------
Namespace(cfg_file='scripts/dense_cap_config.yml', context_fusion=False, data_dir='git/visual_genome', device='gpu', device_id=0, embed_dim=512, imdb_name='vg_1.2_train', imdbval_name='vg_1.2_val', max_iters=200000, net='res50', randomize=False, set_cfgs=['EXP_DIR', 'dc_conv_fixed', 'CONTEXT_FUSION', 'False', 'RESNET.FIXED_BLOCKS', '3'], tag=None, weights='resnet_v1_50/res50.ckpt')
runing with LIMIT_RAM: True
Using config:
{'ALL_TEST': False,
'ALL_TEST_NUM_TEST': 1000,
'ALL_TEST_NUM_TRAIN': 100,
'ALL_TEST_NUM_VAL': 100,
'ANCHOR_RATIOS': [0.5, 1, 2],
'ANCHOR_SCALES': [4, 8, 16, 32],
'CACHE_DIR': '/home/joe/git/visual_genome/1.2',
'CONTEXT_FUSION': False,
'CONTEXT_FUSION_MODE': 'sum',
'CONTEXT_MODE': 'concat',
'DATA_DIR': 'git/visual_genome',
'DEBUG_ALL': False,
'EMBED_DIM': 512,
'END_INDEX': 2,
'EXP_DIR': 'dc_conv_fixed',
'FILTER_SMALL_BOX': False,
'GLOVE_DIM': 300,
'GPU_ID': 0,
'INIT_BY_GLOVE': False,
'KEEP_AS_GLOVE_DIM': False,
'LIMIT_RAM': True,
'LOG_DIR': '/home/XXX/docment/densecap-tensorflow/logs',
'LOSS': {'BBOX_W': 0.01,
'CAP_W': 1.0,
'CLS_W': 0.1,
'RPN_BBOX_W': 0.05,
'RPN_CLS_W': 0.1},
'MAX_WORDS': 10,
'PIXEL_MEANS': array([[[ 102.9801, 115.9465, 122.7717]]]),
'POOLING_MODE': 'crop',
'POOLING_SIZE': 7,
'RESNET': {'FIXED_BLOCKS': 3, 'MAX_POOL': False},
'RNG_SEED': 3,
'ROOT_DIR': '/home/XXX/docment/densecap-tensorflow',
'RPN_CHANNELS': 512,
'SAMPLE_NUM_FIXED_REGIONS': False,
'SPLIT_DIR': '/home/XXX/docment/densecap-tensorflow/info',
'TEST': {'BBOX_REG': True,
'BEAM_SIZE': 3,
'HAS_RPN': True,
'LN_FACTOR': 0.0,
'MAX_SIZE': 720,
'MODE': 'nms',
'NMS': 0.5,
'PROPOSAL_METHOD': 'gt',
'RPN_MIN_SIZE': 16,
'RPN_NMS_THRESH': 0.6,
'RPN_POST_NMS_TOP_N': 300,
'RPN_PRE_NMS_TOP_N': 6000,
'RPN_TOP_N': 5000,
'SCALES': [600],
'SVM': False,
'USE_BEAM_SEARCH': False},
'TIME_STEPS': 12,
'TRAIN': {'ASPECT_GROUPING': True,
'BATCH_SIZE': 256,
'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
'BBOX_NORMALIZE_TARGETS': True,
'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True,
'BBOX_REG': True,
'BBOX_THRESH': 0.5,
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0.0,
'BIAS_DECAY': False,
'CLIP_NORM': 40.0,
'DISPLAY': 10,
'DOUBLE_BIAS': False,
'EXP_DECAY_RATE': 0.9,
'EXP_DECAY_STEPS': 5000,
'FG_FRACTION': 0.5,
'FG_THRESH': 0.5,
'GAMMA': 0.5,
'HAS_RPN': True,
'IMS_PER_BATCH': 1,
'LEARNING_RATE': 0.001,
'LR_DIY_DECAY': True,
'MAX_SIZE': 720,
'MOMENTUM': 0.98,
'OPTIMIZER': 'sgd_m',
'PROPOSAL_METHOD': 'gt',
'RPN_BATCHSIZE': 256,
'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'RPN_CLOBBER_POSITIVES': False,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 16,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POSITIVE_WEIGHT': -1.0,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 12000,
'SCALES': [600],
'SNAPSHOT_ITERS': 5000,
'SNAPSHOT_KEPT': 3,
'SNAPSHOT_PREFIX': 'res50_densecap',
'STEPSIZE': [100000],
'SUMMARY_INTERVAL': 10,
'USE_FLIPPED': True,
'USE_GT': False,
'WEIGHT_DECAY': 0.0001,
'WEIGHT_INITIALIZER': 'normal'},
'TRAIN_GLOVE': False,
'USE_GPU_NMS': True,
'VOCAB_END_ID': 2,
'VOCAB_SIZE': 10000,
'VOCAB_START_ID': 1}
data_path: git/visual_genome/1.2
loading splits from /home/XXX/docment/densecap-tensorflow/info/densecap_splits.json
Number of examples: 77398
train gt roidb could be loaded from git/visual_genome/1.2_cache/train_gt_roidb
Getting gt roidb and number of examples is:154796
output will be saved to `/home/XXX/docment/densecap-tensorflow/output/dc_conv_fixed/vg_1.2_train`
TensorFlow summaries will be saved to `/home/zxf/docment/densecap-tensorflow/output/dc_conv_fixed/tb/vg_1.2_train/default`
data_path: git/visual_genome/1.2
loading splits from /home/XXX/docment/densecap-tensorflow/info/densecap_splits.json
Number of examples: 5000
val gt roidb could be loaded from git/visual_genome/1.2_cache/val_gt_roidb
Getting gt roidb and number of examples is:5000
2018-04-07 22:09:29.615617: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-04-07 22:09:29.765218: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX TITAN major: 3 minor: 5 memoryClockRate(GHz): 0.8755
pciBusID: 0000:04:00.0
totalMemory: 5.94GiB freeMemory: 5.86GiB
2018-04-07 22:09:29.765255: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX TITAN, pci bus id: 0000:04:00.0, compute capability: 3.5)
Solving...
LIMIT_RAM version and load index from git/visual_genome/1.2_cache/train_gt_roidb/image_index.json
LIMIT_RAM version and load index from git/visual_genome/1.2_cache/val_gt_roidb/image_index.json
Fixing 3 blocks.
Initialize embedding vectors with default initializer.
Shape of embedding is (10003, 512)
learning rate 0.001
/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gradients_impl.py:96: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Loading initial model weights from resnet_v1_50/res50.ckpt
Variables restored: resnet_v1_50/conv1/BatchNorm/gamma:0
.....
.....
.....
Variables restored: resnet_v1_50/block4/unit_3/bottleneck_v1/conv3/BatchNorm/beta:0
Variables restored: resnet_v1_50/block4/unit_3/bottleneck_v1/conv3/BatchNorm/moving_mean:0
Variables restored: resnet_v1_50/block4/unit_3/bottleneck_v1/conv3/BatchNorm/moving_variance:0
Loaded.
Fix Resnet V1 layers..
Fixed.
Ckpt path: resnet_v1_50/res50.ckpt
Traceback (most recent call last):
File "./tools/train_net.py", line 214, in <module>
main()
File "./tools/train_net.py", line 210, in main
max_iters=args.max_iters)
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 356, in train_model
blobs = self.data_layer.forward()
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/fast_rcnn/layer.py", line 99, in forward
blobs = self._get_next_minibatch()
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/fast_rcnn/layer.py", line 95, in _get_next_minibatch
return get_minibatch(minibatch_db)
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/fast_rcnn/minibatch.py", line 41, in get_minibatch
im_blob, im_scales, roidb = _get_image_blob(roidb, random_scale_inds)
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/fast_rcnn/minibatch.py", line 101, in _get_image_blob
cfg.TRAIN.MAX_SIZE)
File "/home/XXX/docment/densecap-tensorflow/tools/../lib/utils/blob.py", line 37, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
There are my sys-info:
ubuntu16.04
GeForce GTX TITAN
RAM:8G
CUDA:8.0
cudnn:6.0
TF:1.4
I review the code and think the net wasn't feed any data, I locate the issues in this file:
/lib/utils/blob.py
def prep_im_for_blob(im, pixel_means, target_size, max_size):
"""Mean subtract and scale an image for use in a blob."""
im = im.astype(np.float32, copy=False)
im -= pixel_means
im_shape = im.shape
im_size_min = np.min(im_shape[0:2])
im_size_max = np.max(im_shape[0:2])
im_scale = float(target_size) / float(im_size_min)
# Prevent the biggest axis from being more than MAX_SIZE
if np.round(im_scale * im_size_max) > max_size:
im_scale = float(max_size) / float(im_size_max)
im = cv2.resize(im, None, None, fx=im_scale, fy=im_scale,
interpolation=cv2.INTER_LINEAR)
return im, im_scale
**so, i guess the net read nothing from roidb. and how do i fix it ? **
PS: the tree of my workspace:
├── data
│ ├── ckpt_ori
│ ├── imagenet_weights
│ ├── resnet_v1_50_2016_08_28.tar.gz
│ ├── resnet_v2_101_2017_04_14.tar.gz
│ └── resnet_v2_50_2017_04_14.tar.gz
├── git
│ └── visual_genome
├── git_0
│ └── visual_genome
├── info
│ ├── densecap_splits.json
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── read_regions.py
│ ├── read_splits.py
│ ├── read_splits.pyc
│ ├── test.txt
│ ├── train.txt
│ └── val.txt
├── __init__.py
├── lib
│ ├── config.py
│ ├── config.pyc
│ ├── datasets
│ ├── dense_cap
│ ├── download_data_vh.sh
│ ├── fast_rcnn
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── layers
│ ├── limit_ram
│ ├── Makefile
│ ├── nets
│ ├── nms
│ ├── pre_glove.py
│ ├── preprocess.py
│ ├── preprocess.sh
│ ├── pycocoevalcap
│ ├── setup.py
│ └── utils
├── LICENSE
├── logs
│ ├── densecap.png
│ ├── funny.png
│ ├── s0_res50_vg_1.2_train.txt.2018-04-07_16-02-00
├── Note.md
├── output
│ ├── dc_context
│ ├── dc_conv_fixed
│ ├── dc_tune_context
│ └── dc_tune_conv
├── README.md
├── requirements.txt
├── res50
│ └── res50.ckpt
├── scripts
│ ├── dense_cap_config.yml
│ ├── dense_cap_demo.sh
│ ├── dense_cap_test.sh
│ ├── dense_cap_train.sh
│ └── old_dense_cap_train.sh
├── tests
│ ├── architecture_test.py
│ ├── bash_log_test
│ ├── ckpt_restore_test.py
│ ├── dencap_oa_test.sh
│ ├── __init__.py
│ ├── logs
│ ├── pickle_read_test.py
│ ├── README.md
│ ├── read_regions_json
│ ├── roidata_test.py
│ ├── sentence_data_layer_test.py
│ └── vh_train_command.sh
├── tools
│ ├── demo.py
│ ├── _init_paths.py
│ ├── _init_paths.pyc
│ ├── __init__.py
│ ├── test_net.py
│ └── train_net.py
├── valohai.yaml
├── VG
│ ├── 1.2
│ └── images
└── vis
├── d3.min.js
├── jquery-1.8.3.min.js
├── README.md
├── style.css
├── utils.js
└── view_results.html
any help will be appreciated. THX : )
I came into this error:
File "./tools/demo.py", line 129, in
with open(save_path + '/results.json', 'w') as f:
IOError: [Errno 2] No such file or directory: './vis/data/results.json'
It is clearly slow, but is it better? Why should we use beam search, or not?
当运行bash scripts/dense_cap_train.sh [dataset] [net] [ckpt_to_init] [data_dir] [step]时，出现如下bug:
.......
.......
Loaded.
Fix Resnet V1 layers..
Fixed.
Ckpt path: /home/xiewei/densecap-tensorflow/slim_models/res50.ckpt
Traceback (most recent call last):
File "./tools/train_net.py", line 214, in
main()
File "./tools/train_net.py", line 210, in main
max_iters=args.max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 356, in train_model
blobs = self.data_layer.forward()
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/layer.py", line 99, in forward
blobs = self._get_next_minibatch()
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/layer.py", line 95, in _get_next_minibatch
return get_minibatch(minibatch_db)
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/minibatch.py", line 41, in get_minibatch
im_blob, im_scales, roidb = _get_image_blob(roidb, random_scale_inds)
File "/home/xiewei/densecap-tensorflow/tools/../lib/fast_rcnn/minibatch.py", line 101, in _get_image_blob
cfg.TRAIN.MAX_SIZE)
File "/home/xiewei/densecap-tensorflow/tools/../lib/utils/blob.py", line 37, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
请问，如何解决？谢谢！
Hi,
when I train the model with version 1.2 of Visual Genome Dataset. An error appears:
Traceback (most recent call last):
File "/home/XXX/Documents/densecap-tensorflow/tools/train_net.py", line 215, in
main()
File "/home/XXX/Documents/densecap-tensorflow/tools/train_net.py", line 211, in main
max_iters=args.max_iters)
File "/home/XXX/Documents/densecap-tensorflow/lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/XXX/Documents/densecap-tensorflow/lib/dense_cap/train.py", line 356, in train_model
blobs = self.data_layer.forward()
File "/home/XXX/Documents/densecap-tensorflow/lib/fast_rcnn/layer.py", line 99, in forward
blobs = self._get_next_minibatch()
File "/home/XXX/Documents/densecap-tensorflow/lib/fast_rcnn/layer.py", line 95, in _get_next_minibatch
return get_minibatch(minibatch_db)
File "/home/XXX/Documents/densecap-tensorflow/lib/fast_rcnn/minibatch.py", line 54, in get_minibatch
blobs['gt_phrases'] = _process_gt_phrases(roidb[0]['gt_phrases'])
File "/home/XXX/Documents/densecap-tensorflow/lib/fast_rcnn/minibatch.py", line 77, in _process_gt_phrases
gt_phrases[ix, :l] = phra
ValueError: cannot copy sequence with size 11 to array axis with dimension 10
Some describe words of bounding boxes are more than 10 words. Then I update the __C.MAX_WORDS to be 15, similar error appear. How can I set the parameter of __C.MAX_WORDS? Does anyone else meet this problem?
hello
when i try to this
$ cd $ROOT/lib
$ python preprocess.py --version [version] --path [raw_data_path]
--output_dir [dir] --max_words [max_len]$ cd $ROOT/lib ,
i have type error..
split image number: 77398 for split name: train
start loading json files...
10.529908 seconds for loading
train: 0%| | 0/108077 [00:00<?, ?it/s]
Traceback (most recent call last):
File "preprocess.py", line 226, in
process_vg()
File "preprocess.py", line 222, in process_vg
vocab = process_dataset(split_name, vocab=vocab)
File "preprocess.py", line 196, in process_dataset
split_ids=split_image_ids, vocab=vocab)
File "preprocess.py", line 117, in init
obj['phrase_tokens'] = phrase.translate(None, string.punctuation).split()
TypeError: a bytes-like object is required, not 'str'
how to fix it?
In the context fusion mode, _global_roi is used as the context feature . However, I don't quite understand how you calculate the _global_roi from scratch. Can you give me any hint?
Thanks!
Can I use this code?
i am working on the densecap and have a little confusion about the embedding as i wanted to train the same model on diffrent language. and also glove embedding is only for english so,
i have to use some other embedding?
if yes please suggest where to use the same?
thank you
你好，因为 tensorflow/contrib/slim/python/slim/nets/resnet_v1.py 和tensorflow/contrib/slim/python/slim/nets/resnet_v2.py 极其相似，所以我将 lib/nets/resnet_v1.py 中的出现resnet_v1的地方替换为resnet_v2，作为resnet_v2.py，但是resnet_v1.py 依旧保留。
另外，我在 tools/train_net.py 中，也做了相应的修改，导入了resnetv2（from lib.nets.resnet_v2 import resnetv2）。在#load network也做了些修改，如下：
if args.net == 'vgg16':
net = vgg16()
elif args.net == 'resnet_v1_50':
net = resnetv1(num_layers=50)
elif args.net == 'resnet_v1_101':
net = resnetv1(num_layers=101)
elif args.net == 'resnet_v1_152':
net = resnetv1(num_layers=152)
elif args.net == 'resnet_v2_50':
net = resnetv2(num_layers=50)
elif args.net == 'resnet_v2_101':
net = resnetv2(num_layers=101)
elif args.net == 'resnet_v2_152':
net = resnetv2(num_layers=152)
else:
raise NotImplementedError
当训练模型时，出现以下错误，希望能得到您的建议，谢谢！！@InnerPeace-Wu
~/densecap-tensorflow$ bash scripts/dense_cap_train.sh visual_genome_1.2 resnet_v2_50 /home/xiewei/densecap-tensorflow/slim_models/resnet_v2_50.ckpt /home/xiewei/densecap-tensorflow/visual_genome 1
set -e
export PYTHONUNBUFFERED=True
PYTHONUNBUFFERED=True
DATASET=visual_genome_1.2
NET=resnet_v2_50
ckpt_path=/home/xiewei/densecap-tensorflow/slim_models/resnet_v2_50.ckpt
data_dir=/home/xiewei/densecap-tensorflow/visual_genome
step=1
'[' -d /home/joe ']'
case $DATASET in
TRAIN_IMDB=vg_1.2_train
TEST_IMDB=vg_1.2_val
PT_DIR=dense_cap
FINETUNE_AFTER1=200000
FINETUNE_AFTER2=100000
ITERS1=400000
ITERS2=300000
'[' -d /valohai/outputs ']'
++ date +%Y-%m-%d_%H-%M-%S
LOG=logs/s1_resnet_v2_50_vg_1.2_train.txt.2018-08-03_09-38-38
exec
++ tee -a logs/s1_resnet_v2_50_vg_1.2_train.txt.2018-08-03_09-38-38
echo Logging output to logs/s1_resnet_v2_50_vg_1.2_train.txt.2018-08-03_09-38-38
Logging output to logs/s1_resnet_v2_50_vg_1.2_train.txt.2018-08-03_09-38-38
'[' 1 -lt 2 ']'
python ./tools/train_net.py --weights /home/xiewei/densecap-tensorflow/slim_models/resnet_v2_50.ckpt --imdb vg_1.2_train --imdbval vg_1.2_val --iters 200000 --cfg scripts/dense_cap_config.yml --data_dir /home/xiewei/densecap-tensorflow/visual_genome --net resnet_v2_50 --set EXP_DIR dc_conv_fixed CONTEXT_FUSION False RESNET.FIXED_BLOCKS 3
/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Couldn't import dot_parser, loading of dot files will not be possible.
------ called with args: -------
Namespace(cfg_file='scripts/dense_cap_config.yml', context_fusion=False, data_dir='/home/xiewei/densecap-tensorflow/visual_genome', device='gpu', device_id=0, embed_dim=512, imdb_name='vg_1.2_train', imdbval_name='vg_1.2_val', max_iters=200000, net='resnet_v2_50', randomize=False, set_cfgs=['EXP_DIR', 'dc_conv_fixed', 'CONTEXT_FUSION', 'False', 'RESNET.FIXED_BLOCKS', '3'], tag=None, weights='/home/xiewei/densecap-tensorflow/slim_models/resnet_v2_50.ckpt')
runing with LIMIT_RAM: True
Using config:
{'ALL_TEST': False,
'ALL_TEST_NUM_TEST': 1000,
'ALL_TEST_NUM_TRAIN': 100,
'ALL_TEST_NUM_VAL': 100,
'ANCHOR_RATIOS': [0.5, 1, 2],
'ANCHOR_SCALES': [4, 8, 16, 32],
'CACHE_DIR': '/home/xiewei/densecap-tensorflow/visual_genome/1.2',
'CONTEXT_FUSION': False,
'CONTEXT_FUSION_MODE': 'sum',
'CONTEXT_MODE': 'concat',
'DATA_DIR': '/home/xiewei/densecap-tensorflow/visual_genome',
'DEBUG_ALL': False,
'EMBED_DIM': 512,
'END_INDEX': 2,
'EXP_DIR': 'dc_conv_fixed',
'FILTER_SMALL_BOX': False,
'GLOVE_DIM': 300,
'GPU_ID': 0,
'INIT_BY_GLOVE': False,
'KEEP_AS_GLOVE_DIM': False,
'LIMIT_RAM': True,
'LOG_DIR': '/home/xiewei/densecap-tensorflow/logs',
'LOSS': {'BBOX_W': 0.01,
'CAP_W': 1.0,
'CLS_W': 0.1,
'RPN_BBOX_W': 0.05,
'RPN_CLS_W': 0.1},
'MAX_WORDS': 10,
'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]),
'POOLING_MODE': 'crop',
'POOLING_SIZE': 7,
'RESNET': {'FIXED_BLOCKS': 3, 'MAX_POOL': False},
'RNG_SEED': 3,
'ROOT_DIR': '/home/xiewei/densecap-tensorflow',
'RPN_CHANNELS': 512,
'SAMPLE_NUM_FIXED_REGIONS': False,
'SPLIT_DIR': '/home/xiewei/densecap-tensorflow/info',
'TEST': {'BBOX_REG': True,
'BEAM_SIZE': 3,
'HAS_RPN': True,
'LN_FACTOR': 0.0,
'MAX_SIZE': 720,
'MODE': 'nms',
'NMS': 0.5,
'PROPOSAL_METHOD': 'gt',
'RPN_MIN_SIZE': 16,
'RPN_NMS_THRESH': 0.6,
'RPN_POST_NMS_TOP_N': 300,
'RPN_PRE_NMS_TOP_N': 6000,
'RPN_TOP_N': 5000,
'SCALES': [600],
'SVM': False,
'USE_BEAM_SEARCH': False},
'TIME_STEPS': 12,
'TRAIN': {'ASPECT_GROUPING': True,
'BATCH_SIZE': 256,
'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
'BBOX_NORMALIZE_TARGETS': True,
'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True,
'BBOX_REG': True,
'BBOX_THRESH': 0.5,
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0.0,
'BIAS_DECAY': False,
'CLIP_NORM': 40.0,
'DISPLAY': 10,
'DOUBLE_BIAS': False,
'EXP_DECAY_RATE': 0.9,
'EXP_DECAY_STEPS': 5000,
'FG_FRACTION': 0.5,
'FG_THRESH': 0.5,
'GAMMA': 0.5,
'HAS_RPN': True,
'IMS_PER_BATCH': 1,
'LEARNING_RATE': 0.001,
'LR_DIY_DECAY': True,
'MAX_SIZE': 720,
'MOMENTUM': 0.98,
'OPTIMIZER': 'sgd_m',
'PROPOSAL_METHOD': 'gt',
'RPN_BATCHSIZE': 256,
'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'RPN_CLOBBER_POSITIVES': False,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 16,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POSITIVE_WEIGHT': -1.0,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 12000,
'SCALES': [600],
'SNAPSHOT_ITERS': 5000,
'SNAPSHOT_KEPT': 3,
'SNAPSHOT_PREFIX': 'res50_densecap',
'STEPSIZE': [100000],
'SUMMARY_INTERVAL': 10,
'USE_FLIPPED': True,
'USE_GT': False,
'WEIGHT_DECAY': 0.0001,
'WEIGHT_INITIALIZER': 'normal'},
'TRAIN_GLOVE': False,
'USE_GPU_NMS': True,
'VOCAB_END_ID': 2,
'VOCAB_SIZE': 10000,
'VOCAB_START_ID': 1}
data_path: /home/xiewei/densecap-tensorflow/visual_genome/1.2
loading splits from /home/xiewei/densecap-tensorflow/info/densecap_splits.json
Number of examples: 77398
train gt roidb could be loaded from /home/xiewei/densecap-tensorflow/visual_genome/1.2_cache/train_gt_roidb
Getting gt roidb and number of examples is:154796
output will be saved to /home/xiewei/densecap-tensorflow/output/dc_conv_fixed/vg_1.2_train
TensorFlow summaries will be saved to /home/xiewei/densecap-tensorflow/output/dc_conv_fixed/tb/vg_1.2_train/default
data_path: /home/xiewei/densecap-tensorflow/visual_genome/1.2
loading splits from /home/xiewei/densecap-tensorflow/info/densecap_splits.json
Number of examples: 5000
val gt roidb could be loaded from /home/xiewei/densecap-tensorflow/visual_genome/1.2_cache/val_gt_roidb
Getting gt roidb and number of examples is:5000
2018-08-03 09:38:39.720107: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-08-03 09:38:40.880941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: Tesla P4 major: 6 minor: 1 memoryClockRate(GHz): 1.1135
pciBusID: 0000:01:00.0
totalMemory: 7.43GiB freeMemory: 7.32GiB
2018-08-03 09:38:40.881009: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Tesla P4, pci bus id: 0000:01:00.0, compute capability: 6.1)
Solving...
LIMIT_RAM version and load index from /home/xiewei/densecap-tensorflow/visual_genome/1.2_cache/train_gt_roidb/image_index.json
LIMIT_RAM version and load index from /home/xiewei/densecap-tensorflow/visual_genome/1.2_cache/val_gt_roidb/image_index.json
Fixing 3 blocks.
Traceback (most recent call last):
File "./tools/train_net.py", line 221, in
main()
File "./tools/train_net.py", line 217, in main
max_iters=args.max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 485, in train_net
sw.train_model(sess, max_iters)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 318, in train_model
lr, train_op = self.construct_graph(sess)
File "/home/xiewei/densecap-tensorflow/tools/../lib/dense_cap/train.py", line 148, in construct_graph
layers = self.net.create_architecture('TRAIN', num_classes=1, tag='default')
File "/home/xiewei/densecap-tensorflow/tools/../lib/nets/network.py", line 812, in create_architecture
rois, cls_prob = self._build_network(training)
File "/home/xiewei/densecap-tensorflow/tools/../lib/nets/network.py", line 601, in _build_network
fc7 = self._head_to_tail(pool5, is_training)
File "/home/xiewei/densecap-tensorflow/tools/../lib/nets/resnet_v2.py", line 131, in _head_to_tail
scope=self._resnet_scope)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/nets/resnet_v2.py", line 221, in resnet_v2
net, activation_fn=nn_ops.relu, scope='postnorm')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 643, in batch_norm
outputs = layer.apply(inputs, training=is_training)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 671, in apply
return self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 559, in call
self.build(input_shapes[0])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/normalization.py", line 189, in build
trainable=True)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 458, in add_variable
trainable=trainable and self.trainable)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
constraint=constraint)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
constraint=constraint)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 417, in get_variable
return custom_getter(**custom_getter_kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1539, in layer_variable_getter
return _model_variable_getter(getter, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1531, in _model_variable_getter
custom_getter=getter, use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py", line 262, in model_variable
use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py", line 217, in variable
use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
use_resource=use_resource, constraint=constraint)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 742, in _get_single_variable
name, "".join(traceback.format_list(tb))))
ValueError: Variable resnet_v2_50/postnorm/gamma already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py", line 217, in variable
use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py", line 262, in model_variable
use_resource=use_resource)
Hello I just wanted some insight on the context features and using attention for the language model.Can anyone help?
Hi,
Can you provide pretrained weights? Thanks!
Jiakui Wang
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.