Comments (10)
Hi, the outputs of the inference should be a list of triplets including subjects, objects and their relations. Not only entities.
These triplets make up the scene graph.
Do you mean the automatic visualization? or just get the semantic results? If you want to visualize the scene graph, Graphviz is a good tool for you.
from reltr.
Hi, thanks for your response.
It seems the work : RelTR is excellent. I am trying to using it to generate a scene graph, which is similar to the "demo.png". I am a freshman in this field, so I wonder whether there is scripts of generating a scene graph as the demo image which could be shared? That would be wonderful.
Thanks!
from reltr.
You could use Graphviz if you want to draw scene graphs automatically.
But there is no tool that can draw scene graphs like the figures in papers. They are all hand drawn.
from reltr.
Hi, I still confuse about the scene graph generation, the question is how to use the outputs data to construct a Scene Graph?
when execute the command to run the inference, outputs of the model will be generated as below:
python inference.py --img_path ./demo/vg3.jpg --resume ./ckpt/checkpoint0149.pth
propagate through the model
outputs = model(img)
outputs:
{'pred_logits': tensor([[[-14.0701, -6.9378, -3.6805, ..., -2.2955, -8.1152, 9.3173],
...
[-11.5406, -4.8710, -5.6274, ..., -5.3334, -11.0269, 8.4589]]]), 'pred_boxes': tensor([[[0.9285, 0.2640, 0.1396, 0.3936],
...
[0.5625, 0.2458, 0.1401, 0.2452]]]), 'sub_logits': tensor([[[-10.0269, -5.9606, -2.0709, ..., -2.8782, -11.7878, 6.2765],
...
[-14.5083, -7.4713, -5.7031, ..., -5.1850, -9.3440, 8.3625]]]), 'sub_boxes': tensor([[[0.4524, 0.2413, 0.4110, 0.3153],
...
[0.2999, 0.2770, 0.6007, 0.5570]]]), 'obj_logits': tensor([[[-11.4217, -6.2552, -5.2713, ..., -3.9685, -14.2370, 6.4583],
...
[-11.3257, -3.0029, -1.1787, ..., -3.2346, -1.6754, 7.0650]]]), 'obj_boxes': tensor([[[0.4553, 0.2449, 0.4414, 0.3307],
...
Thanks.
from reltr.
Hi, could you please kindly help to let me know how to use the outputs data generated by the model to create a Scene Graph?
Thanks
from reltr.
Hi, I still have the same question of how to create a Scene Graph from output data,
Since it is urgent, could you share the function script?
In the output data, how to know the relationship between subject and object?
Output dict with the following elements:
- "pred_logits",
- "pred_boxes",
- "sub_logits",
- "obj_logits",
- "sub_boxes",
- "obj_boxes",
- "aux_outputs"
Thanks.
from reltr.
"pred_logits": the entity class probability (this is not related to SGG)
"pred_boxes": the entity bounding box (this is not related to SGG)
"sub_logits": N * subject class probability
"obj_logits": N * object class probability
"sub_boxes": N * subject bounding box
"obj_boxes": N * object bounding box
"rel_logits": N * relationship class probability
"aux_outputs": output for auxiliary loss (this is not related to SGG)
For example, the information of the k-th triplet proposal should be:
output["sub_logits"][k], output["sub_boxes"][k], output["obj_logits"][k], output["obj_boxes"][k], output["rel_logits"][k]
As you see in the inference.py, you can filter some proposals when the confidence scores (subject/object/relationship) are low.
from reltr.
Hi, thanks for your response very much. Here following one question:
i.e. : the data of the value: output["sub_logits"][k] is numeral value, such as -11.7209,
how to associate it with VG classes, like 'airplane'?
Does there any formula or function doing the transformation?
Thanks
from reltr.
Something is wrong...
output["sub_logits"] should be a tensor with the shape [query_number, class_num+1]!
So output["sub_logits"][k] should be a 1- d tensor. The shape should be [152] (or if 151 I forgot if there is a padding class). This is the probability corresponding to the 151 VG entity classes (with "background").
I think inference.py is a good demo. You see the output figure has shown the triplets. It's better to read it line by line. I think it is easy to understand the output structure.
from reltr.
Thanks very much!
from reltr.
Related Issues (20)
- Model not getting trained on single GPU HOT 4
- Evaluation on VRD dataset HOT 1
- Hi, the training results are lower than reported results. HOT 3
- Inquiry about code behavior in relation to constraints of evaluation metrics HOT 1
- About training strategy HOT 1
- Can ur work train on VidOR? HOT 1
- Some questions about evaluation HOT 2
- RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. HOT 4
- About evaluate_rel_batch() function HOT 7
- name 'train_stats' is not defined HOT 2
- convert the reltr model to onnx forma
- about Predcls HOT 1
- Evaluation HOT 3
- checkpoint should be updated with enhanced version HOT 1
- Some misunderstanding about the heat map using to predict Relationship HOT 2
- Error during training in bbox.pyx : ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'double' HOT 1
- 请问如何将inference.py得到的场景图保存成一个json文件呢 HOT 1
- What happens when there are no relations in a sample? HOT 2
- When I was training data, I encountered an error HOT 1
- About OpenV6 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from reltr.