yzhangcs / sotu Goto Github PK

View Code? Open in Web Editor NEW

86.0 7.0 36.0 855 KB

A flask-based cbir system

License: MIT License

Python 58.98% CSS 15.02% JavaScript 10.94% HTML 15.05%

flask cbir bof he wgc reranking

sotu's Introduction

sotu

利用Flask¹框架实现的基于内容的图像检索(Content Based Image Retrieval, CBIR)系统

要求

python版本：3.5.2

pip版本： 9.0.3

初始化

程序在虚拟环境下运行，首先确保安装virtualenv，之后创建并激活虚拟环境，接着在虚拟环境下安装所有必要的组件，并设置相关环境变量的值.

在运行应用之前，需要提取图像特征，其中初始数据集使用的是ukbench²数据集的前4096幅图，提取的特征是SIFT的改进版，即RootSIFT特征.

在Linux下所有的命令为：

# 复制仓库并进入相应目录
$ git clone https://github.com/zysite/SoTu.git
$ cd SoTu

# 激活虚拟环境并安装所有必要的组件
$ virtualenv venv
$ . venv/bin/activate
$ pip install -r requirements.txt

# 设置环境变量FLASK_APP和FLASK_ENV的值
$ export FLASK_APP=sotu.py
$ export FLASK_ENV=development

# 设置最大线程数
# $ export OMP_NUM_THREADS=8

# 提取图像特征
$ flask extract

其中激活虚拟环境的命令在Windows的环境下有所不同：

$ venv\Scripts\activate

如果用cmd设置环境变量，需要用set代替上面的export.

如果用powershell设置，则命令为：

$ $env:FLASK_APP="sotu.py"

最后退出虚拟环境的命令为：

$ deactivate

运行

检索系统的实现基于特征袋模型(Bag of Feature, BoF)，并在此基础上利用了汉明嵌入(Hamming Embedding, HE)方法、弱几何一致性(Weak Geometric Consistency, WGC)约束和基于RANSAC算法的几何重排.

运行web应用使用下面的命令，可以指定主机和端口：

$ flask run -h localhost -p 8080

检索系统支持文件上传、拖拽上传和URL上传三种图片上传方式.

评估

这里使用的评估指标是mAP(mean Average Precision, mAP)指标，执行评估使用下面的命令：

$ flask evaluate

不同方法的评价结果如下表，其中BoF模型设置的聚类数为5000，HE的阈值ht为23：

methods	mAP
BoF	0.713298
BoF+HE	0.878229
BoF+HE+Reranking	0.898573

Jégou提到对于ukbench数据集而言，WGC方法的效果较差³，因此评估没有采用WGC方法.

参考文献

Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110.
Zisserman A. Three things everyone should know to improve object retrieval [C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Rhode Island, USA, 2012:2911-2918.
Sivic J, Zisserman A. Video google: A text retrieval approach to object matching in videos [C]. IEEE International Conference on Computer Vision, 2003, 2(1470): 1470-1477.
Jégou H, Douze M, Schmid C. Hamming Embedding and Weak Geometry Consistency for Large Scale Image Search[J]. Proc Eccv, 2008, 5302:304-317.
Jégou H, Douze M, Schmid C. Improving bag-of-features for large scale image search[J]. International journal of computer vision, 2010, 87(3): 316-336.
Zhao W L, Wu X, Ngo C W. On the Annotation of Web Videos by Efficient Near-Duplicate Search[J]. IEEE Transactions on Multimedia, 2010, 12(5):448-461.
Philbin J, Chum O, Isard M, et al. Object retrieval with large vocabularies and fast spatial matching [C]. IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA, 2007: 1-8.

sotu's People

Contributors

Stargazers

Watchers

Forkers

willard-yuan ztf-ucas apple1987 allensmile yunhai0920 liuheng0111 poonono zhudaoruyi cv-apps ruizewang nizvoo abrams90 daonancai zhoujunhaohao cenchaojun andishafira macxy2018 fancyxun shineway14

sotu's Issues

多次flask extract的话，旧的就被新的覆盖了？可以累加起来吗？

想请问，这是啥意思？

Traceback (most recent call last):
File "c:\users\administrator\appdata\local\programs\python\python35\lib\runpy.py", line 184, in _run_module_as_main
"main", mod_spec)
File "c:\users\administrator\appdata\local\programs\python\python35\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "D:\SoTu\venv\Scripts\flask.exe_main.py", line 7, in
File "d:\sotu\venv\lib\site-packages\flask\cli.py", line 966, in main
cli.main(prog_name="python -m flask" if as_module else None)
File "d:\sotu\venv\lib\site-packages\flask\cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "d:\sotu\venv\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "d:\sotu\venv\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "d:\sotu\venv\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "d:\sotu\venv\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "D:\SoTu\venv\vision\bof.py", line 32, in extract
self.extract()
File "D:\SoTu\venv\vision\bof.py", line 81, in extract
prj_all = np.vstack([prj for prj in projections])
File "d:\sotu\venv\lib\site-packages\numpy\core\shape_base.py", line 234, in vstack
return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
MemoryError

browse http://localhost:8080/ error: 'Blueprint' object has no attribute 'json_encoder'

File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/werkzeug/serving.py", line 270, in run_wsgi
execute(self.server.app)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/werkzeug/serving.py", line 258, in execute
application_iter = app(environ, start_response)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/app.py", line 1997, in call
return self.wsgi_app(environ, start_response)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/app.py", line 1978, in wsgi_app
ctx.push()
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/ctx.py", line 332, in push
self.session = self.app.open_session(self.request)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/app.py", line 912, in open_session
return self.session_interface.open_session(self, request)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/sessions.py", line 324, in open_session
s = self.get_signing_serializer(app)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/sessions.py", line 321, in get_signing_serializer
signer_kwargs=signer_kwargs)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/itsdangerous.py", line 519, in init
self.is_text_serializer = is_text_serializer(serializer)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/itsdangerous.py", line 69, in is_text_serializer
return isinstance(serializer.dumps({}), text_type)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/sessions.py", line 85, in dumps
return json.dumps(_tag(value), separators=(',', ':'))
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/json/init.py", line 177, in dumps
_dump_arg_defaults(kwargs)
File "/home/brandon/SoTu/venv/lib/python3.6/site-packages/flask/json/init.py", line 98, in _dump_arg_defaults
bp.json_encoder if bp and bp.json_encoder
AttributeError: 'Blueprint' object has no attribute 'json_encoder'

multi image detection / multiple occurences

Hi,

Hope you are all well !

I was wondering if SoTu is able to make these 2 points:

Detection of multiple occurrences of the same picture

For example, based on find-object, being able to detect multiple occurences of the same object.

Many objects detection

Another example, still based on find-object](https://github.com/introlab/find-object), to detect many objects in a scene.

Thanks in advance for your insights and inputs on these topics.

Cheers,
X