Question about the usage of text input about matrn HOT 3 CLOSED

That's a good point!
As you mentioned, our model does not use text information for both training and inference, and using the text information in STR model is not trivial approach as your guess.
Actually, we want to use the text information in visual clue masking which is our special strategy to boost STR performance. In this approach, we might not worry about using the text information since masking is not used in inference time.

from matrn.

Oh, sorry for the confusion. Actually, I meant that the visual clue masking is our special strategy to boost STR in our model, MATRN. But, in now, we just pick a position randomly and mask the top-K visual features relevant to the chosen position.
I want to say that the position picking process might be improved with text information, which is not need in inference stage.
If we develop this idea, we will keep you posted!

from matrn.

It sounds really mind-blowing to me.
Will you post the code to show your "special strategy to boost STR" soon? Can't wait to see the implementation.

from matrn.