Comments (3)
That's a good point!
As you mentioned, our model does not use text information for both training and inference, and using the text information in STR model is not trivial approach as your guess.
Actually, we want to use the text information in visual clue masking which is our special strategy to boost STR performance. In this approach, we might not worry about using the text information since masking is not used in inference time.
from matrn.
Oh, sorry for the confusion. Actually, I meant that the visual clue masking is our special strategy to boost STR in our model, MATRN. But, in now, we just pick a position randomly and mask the top-K visual features relevant to the chosen position.
I want to say that the position picking process might be improved with text information, which is not need in inference stage.
If we develop this idea, we will keep you posted!
from matrn.
It sounds really mind-blowing to me.
Will you post the code to show your "special strategy to boost STR" soon? Can't wait to see the implementation.
from matrn.
Related Issues (20)
- Demo to add? HOT 2
- handwriting HOT 1
- Question about code HOT 1
- Question about the performance of pre-trained model that the link contains. HOT 4
- Has your paper been accepted? HOT 1
- vision and language model training HOT 2
- Question about reproducing. HOT 2
- Error about the model when using resnet as the backbone. HOT 1
- About Code in Line 81 of main.py HOT 1
- Predict More Characters HOT 4
- Training always ends HOT 3
- Training on Non-Latin Languages HOT 1
- Question about metrics HOT 2
- Question about pretraining on language model HOT 5
- change input image size error HOT 2
- Question about evaluation on each dataset HOT 2
- The calculation of accuracy prediction in evaluation dataset HOT 2
- 作者你好,请问你论文里多模态中交叉注意力可视化在代码哪个部分 HOT 6
- LMDB Dataset & Loss Function Calculation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from matrn.