bojone / cosent Goto Github PK
View Code? Open in Web Editor NEW比Sentence-BERT更有效的句向量方案
比Sentence-BERT更有效的句向量方案
数据集是 0-1 格式的数据集,0 表示不相似,1 表示相似,各有10 万对。语料库中总共有 200 万个句子,也就是说有些样本没参与训练。不太一样的就是:第一个句子很短,5 6 个字左右,第二个句子很长,50字左右。比如淘宝搜索:XXX零食,推荐的结果会有:XXX商店XXX口味XXX面包。
我使用 CoSENT 进行训练,在 bert 后接入一个降维层,生成文本的 128 维度特征向量,期待相似样本的距离近,不相似样本距离远。微调 3 个 epoch 左右,spearman 得分在 0.86 左右。
而后,将所有文本生成特征向量,构建向量索引(这里用的别人成熟的框架,不是构建索引出错),并查询距离最近的向量,发现很难查回正样本,MRR 指标也很差,这个问题我怀疑是距离还是没有拉开,请问您在使用这个方法的时候有没有遇到过类似问题呢?仿佛偏题了
源码如下:
def optimal_threshold(y_true, y_pred):
"""最优阈值的自动搜索
"""
loss = lambda t: -np.mean((y_true > 0.5) == (y_pred > np.tanh(t)))
result = minimize(loss, 1, method='Powell')
return np.tanh(result.x), -result.fun
老师好,想问问为什么powell里边针对Acc的最小值优化需要加个np.tanh(t),如果直接为 (y_pred > t) 为什么不好呢? 想知道两个的区别,想了一晚上没想明白,希望老师能点一下思路
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.