cubicasa / cubicasa5k Goto Github PK

Hi,

I've been visualizing the floor plans and they are very interesting. I was wondering if you had real-world scale information associated with the floor plans? I would like to get metric information for the object and room sizes.

Hi!

In fig 1. on your paper you display the automatic prediction compared to the actual SVG labels. How do you reproduce this since the models output is a segmentation map and running the post-processor outputs only the polygons without the actual "automatic prediction"? I assume that the result here is the floor plan after parsing it through some CAD tool, if so could you elaborate on the methods you used? Great work otherwise! @kaleaht

Hello! Thank you for a great work with awesome datasets and nice implementation!!

I've found an issue while trying to build the Dockerfile.
It seems that something is wrong when install Python packages while conducting pip install -r requirements.txt.

Here's the error log of the failing position.

Successfully built backcall lmdb networkx pandocfilters prometheus-client pyrsistent toolz tornado visdom
mkl-random 1.0.1 has requirement intel-numpy<1.15,>=1.14, but you'll have intel-numpy 1.15.1 which is incompatible.
Installing collected packages: attrs, backcall, webencodings, bleach, certifi, cffi, cloudpickle, cycler, Cython, dask, decorator, defusedxml, entrypoints, idna, tornado, ipython-genutils, traitlets, python-dateutil, pyzmq, jupyter-core, jupyter-client, ptyprocess, pexpect, Pygments, pickleshare, wcwidth, prompt-toolkit, parso, jedi, ipython, ipykernel, MarkupSafe, Jinja2, joblib, pyrsistent, jsonschema, terminado, Send2Trash, nbformat, testpath, pandocfilters, mistune, nbconvert, prometheus-client, notebook, jupyterlab-server, jupyterlab, kiwisolver, lmdb, numpy, pyparsing, matplotlib, intel-openmp, icc-rt, tbb, tbb4py, mkl, mkl-random, intel-numpy, mkl-fft, networkx, pytz, pandas, Pillow, protobuf, pyOpenSSL, PyWavelets, requests, scipy, scikit-image, Shapely, svgwrite, svgpathtools, tensorboardX, toolz, tqdm, websocket-client, visdom
  Found existing installation: certifi 2019.3.9
Cannot uninstall 'certifi'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

I've dealt with the problem by changing the statement in requirements.txt from

certifi==2018.10.15

to

certifi==2019.3.9

but it may be better not to fix the version of certifi module.

Hi,
I have with 6GB memory. but still I am getting the below exception when I train.
I set the batch-size to 1 and num_workers to 0. but could not avoid this exception.
Any solution?

Thanks in advance.

RuntimeError: CUDA out of memory. Tried to allocate 646.13 MiB (GPU 0; 6.00 GiB total capacity; 3.25 GiB already allocated; 289.57 MiB free; 1.05 GiB cached)

Thanks for sharing this amazing work!

I want to automate 3D model generation.
How do I feed the predictions after post processing to CAD as you mentioned in fig 5 (https://arxiv.org/pdf/1904.01920.pdf)

As I understand the dataset, there is always an original image of plan as well as a scaled version. The scaled version can have larger dimensions then the svg, but it is scaled to match the model.svg annotation.

Through parsing the plans myself, I found out that some of them are not scaled accordingly. For example high_quality_architectural/20107 is such a case (and there are a some more).

Both png (original and scaled) are the same size: w: 485px h: 430px
While the svg document has a size of w: 1066px h: 744px

Even just resizing the png to the svg won't help, because the svg is a bit larger then the matching scaling as you see here:

upscale png to svg size

upscale png to fit svg floorplan

Hi @ccmarkus ,
In your paper, you said the annotation is produced by a special CAD tool. Could you show me the name of this tool cause I want to annotate my own data and produce it to svg format ?
Thank you so much !

Hey there!

I would potentially be interested in using the given model as well as post processing code for a commercial application. Is it possible to obtain a commercial license for this?

Thanks!

Hello,
I'm trying to run the sample notebook on a new laptop with Ubuntu 20.04, RTX2000 GPU, and nvidia-driver-535.
When trying to execute following section in samples.ipynb

Networks prediction for the segmentation

I get following error in the notebook immediately with model():

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-6-d742f71e52a2> in <module>
     11         # We rotate first the image
     12         rot_image = rot(image, 'tensor', forward)
---> 13         pred = model(rot_image)
     14         # We rotate prediction back
     15         pred = rot(pred, 'tensor', back)

~/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

/app/floortrans/models/hg_furukawa_original.py in forward(self, x)
    134 
    135     def forward(self, x):
--> 136         out = self.conv1_(x)
    137         out = self.bn1(out)
    138         out = self.relu1(out)

~/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

~/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/modules/conv.py in forward(self, input)
    318     def forward(self, input):
    319         return F.conv2d(input, self.weight, self.bias, self.stride,
--> 320                         self.padding, self.dilation, self.groups)
    321 
    322 

RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Terminal running docker shows:

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1544174967633/work/aten/src/THC/THCGeneral.cpp line=405 error=8 : invalid device function

Could I get help to resolve this issue,
Thank you!

Hi

I found the method get_polygons from floortrans.post_prosessing can only post prosessing the horizontal or vertical walls, how can this method prosess the sloping walls?

eg.
img by prediction:

but after post prosessing by the get_polygons from floortrans.post_prosessing

the sloping walls in the prediction img are filtered , can this method remain the sloping walls by change some params or any other way?

Thinks ,waiting for your answer!
Best wishes!

Hi,

I was trying to create the database in the required format for a few files. I tried running the script create_lmdb.py but getting error as below:

I have tried replacing / with \ and also tried couple of other ways to get the location right but still facing the same issue. Kindly assist me in making the correct changes to be able to execute this script successfully.

Since this work contains both a model and data, do they both follow the license mentioned?(https://github.com/CubiCasa/CubiCasa5k/blob/master/LICENSE) or are they released under terms. If so, please link that.

Hi, I try to reproduce your results (with the goal of then try a different training approach as part of my master's thesis).

However, when I run the train.py script it takes forever to train. Each epoch takes about one hour but in your paper you have stated that in total (400 epochs) the training took you "... three hours on a Nvidia GeForce GTX TitanX GPU card"

I want to understand what might be the reason for this huge discrepancy.
What I have found is that the loading (including augmentations) is what takes the most time, please see table.

Specs

• I have created a lmdb database using create_lmdb.py
• I am using google colab and as far as my understanding goes, the performance should be similar to yours.
  • GPU: Tesla K80
  • CPU: Intel(R) Xeon(R) CPU @ 2.20GHz
  • SSD
• I have tried to set use num_workers=0 and 1, 2 8 etc.

Am I missing something obvious or do you have an idea what might be going on?
Thankfull for any help

Hello, I've downloaded all the data and the pretrained model and I've succesfully ran the shared notebook.

But I can't understand how to pass a new floorplan image from a custom site and get the model prediction.

When previewing the svg file and floorplan image, we can observe that the svg file misses part of the floorplan. However, the labels extracted from the svg file seems to cover the elements correctly. Why does this happen?
Thanks!

Thanks for sharing your work.

I am able to parse the floor plan with the categories you have provided. However, I am working to parse other categories such as sofa, tables etc. I see the icons for these are present in the actual floorplans (like F1_original.png) but not included the model.svg.

Could you please provide a way to parse these icons as well?

What is the correct way to interpret the House modules wall representation? This wasn't clear whilist looking at the source code for house.py. In the source it seems to be generated by the following block

for w in self.new_walls:
      end_points = w.end_points.round().astype('int').tolist()
      if w.name == "Wall":
            self.representation['walls'].append([end_points, ['wall', 1, 1]])
      else:
            self.representation['walls'].append([end_points, ['wall', 2, 1]])

Trying this with a sample data point I got that

house.representation['walls']

returns

[[[[25, 260], [474, 260]], ['wall', 1, 1]],
 [[[474, 260], [474, 680]], ['wall', 1, 1]],
 [[[468, 787], [468, 997]], ['wall', 1, 1]],
 [[[235, 997], [468, 997]], ['wall', 1, 1]],
 [[[25, 1003], [230, 1003]], ['wall', 1, 1]],
 [[[25, 260], [25, 1003]], ['wall', 1, 1]],
 [[[24, 62], [24, 241]], ['wall', 1, 1]],
 [[[474, 62], [474, 241]], ['wall', 1, 1]],
 [[[39, 71], [460, 71]], ['wall', 2, 1]],
 [[[235, 745], [235, 997]], ['wall', 1, 1]],
 [[[235, 745], [400, 745]], ['wall', 1, 1]],
 [[[40, 745], [230, 745]], ['wall', 1, 1]],
 [[[411, 691], [411, 776]], ['wall', 1, 1]],
 [[[411, 776], [477, 776]], ['wall', 1, 1]],
 [[[477, 691], [477, 776]], ['wall', 1, 1]],
 [[[411, 691], [477, 691]], ['wall', 1, 1]]]

so the question is that should I interpret the first two lists as (xmin, ymin), (xmax, ymax)?

Hello,
I am a researcher from Carnegie Mellon University. I am looking into building an indoor navigation system with some new techniques and I am hoping to explore how SVG formats of floor plans, like that which is outputted from your model, may integrate into and benefit my research project. I am trying to create the docker container for your project but I am getting the following error:

106.3 Cannot uninstall 'certifi'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
------
Dockerfile:10
--------------------
   8 |     COPY requirements.txt /app/.
   9 |     
  10 | >>> RUN pip install -r requirements.txt
  11 |     
  12 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements.txt" did not complete successfully: exit code: 1

Could you guide me on how to fix this issue?

What is the machine configuration used in your experiment, what is the configuration of the cpu, is it multi-core? How many cores?

Hi,

Thanks for the great contribution! I've been learning allot from this project.

This brought me to the following question: how can you create your own dataset? Or add custom data/ images?

So far I've been looking at the SVG's (in the provided dataset) and the corresponding structure of them. This makes me wonder how you have created them: did you use any specific tool or method for it?

It seems to be inaccurate and allot of work when doing this by directly manipulating the raw SVG files.

Kind regards,
Mateo

Could you please illustrate how to annotate our own data and make it as .svg to use the model? I am really stuck at this :/

Dear
I run the code, however, I can not reproduce the result in the paper, Is there anything else I could do to reproduce the results

I want to train this dataset with yolov8(segmentation and prediction) model. One model will trained to segment the rooms only and another model will predict the door, window, bed, commode, basin etc furniture.

For this kind of task need annotated .txt file. Is this dataset have .txt file after annotation?

Hi,
I am getting these two exceptions when I train. Can you please help me in fixing them?
Thanks in advance.

D:\Professional\GitHub\CubiCasa5k>python train.py --batch-size=1
D:\Professional\GitHub\CubiCasa5k\floortrans\models\hg_furukawa_original.py:227: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if y.shape != x.shape:
Traceback (most recent call last):
File "train.py", line 416, in
train(args, log_dir, writer, logger)
File "train.py", line 147, in train
for i, samples in tqdm(enumerate(trainloader), total=len(trainloader),
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 819, in iter
return _DataLoaderIter(self)
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 560, in init
w.start()
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle Environment objects

D:\Professional\GitHub\CubiCasa5k>Traceback (most recent call last):
File "", line 1, in
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\USER\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

We are interested in utilizing training code which i understand is a within a commercial setting, specifically.We understand that the current license is CC-BY-NC, which does not permit commercial use. Therefore, we would like to formally request the possibility of obtaining a commercial license. We are also following Raster-to-Vector: Revisiting Floorplan Transformation as much as possible.

I am not well versed with '.svg' data format [so apologies, if I miss anything obvious; do point it out to me] and how the ground truth is begin used.

Is there a module in your code that can dump (or a place I can easily insert code to dump) the data in a standard CSV format?
csv = [ image, symbol, polygon.positions [(px1,py1),(px2,py2)...] ]

This will make it easy to compare the performance of other networks on this database.