Comments (3)
hello,koukou
from kobe.
Hi @jiqiujia ,
Thanks for your interest in this work!
I'm glad that someone finally noticed that part of code! Indeed we've tried to add RL objective and optimize with policy gradient. But unfortunately, they didn't work well enough, so we gave it up before the submission of the paper.
Specifically, we've tried two types of reward, they are:
BLEU reward
As many people did (e.g., https://arxiv.org/pdf/1705.04304), BLEU is straight-forward for combining RL and NLP. We expected an improved BLEU after adding this objective. However, it doesn't work and slows down the training. Also no significant improvement is shown in the generated descriptions, so we gave up BLEU reward.
Attractiveness reward
Our ultimate objective in this paper is to generate good product descriptions, which means the more users are interested and attracted, the better. However, the current objective is merely maximizing the likelihood of generated product descriptions in the dataset, which ignores the users' reaction (e.g., click through rate, page stay time).
Considering this issue, we designed an "attractiveness reward function", which is a trained classifier. This classifier is trained to predict user CTR (Click Through Rate) of a product description. With this reward, we expect the generated descriptions to be more attractive to users (in Chinese, more like 标题党). However, the model tends to overfit the reward function, (i.e., the reward function gives a very high reward but the generated description is not actually attractive judged by human).
To address this problem, we considered adversarial training of the seq2seq model and the classifier (reward giver) to be necessary. We didn't have enough time back then and left that part for future work.
If you are interested in this direction and want further discussions, contact me on my Messenger or add my WeChat: chenqibin99
Best,
Qibin Chen
from kobe.
I get it. Thank you for your reply.
from kobe.
Related Issues (20)
- Issue with evaluating model with beam search HOT 4
- Named entity to match product title with knowledge graph HOT 2
- Explaination about your preprocessed data HOT 3
- Failed to download the processed training data HOT 1
- detail user category HOT 1
- At the end of the run, the bleu score can no longer be improved at around 6. HOT 5
- Is there any dataset in English? HOT 1
- how long does the training process last?Could u please provide the training result? HOT 3
- How do you build the fact file in raw data? HOT 8
- Good name! HOT 1
- inference HOT 2
- 请问怎么在自己的数据集上做finetune和推理? HOT 4
- Error when preprocessing dataset HOT 1
- Why did you delete bi-attention code? HOT 2
- English language dataset HOT 1
- 关于模型的generate的API试用问题 HOT 1
- ignore this issue
- 关于24-V2训练
- WandB Link?
- 请问有图生文的部分吗
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kobe.