Multi-process BLEU evaluation tool.
多进程BLEU评估工具。
Modified from the bleu scoring tool of huggingface evaluation.
改自 huggingface evaluate 的 bleu 评分工具。
https://github.com/huggingface/evaluate/blob/main/metrics/bleu/bleu.py
pip
pip install -U bleu-mp
dev
git clone https://github.com/One-sixth/bleu-mp
cd bleu-mp
pip install -e .
- Faster!
- The built-in multi-process implementation of python is not used. I use my own multi-process implementation, which is very friendly to the compatibility of Windows and Linux. The memory occupied by each calculation subprocess is very low.
- 更快！
- 不使用python内置的多进程实现。使用我自己的多进程实现，从而对windows和linux的兼容性非常友好，每个计算子进程占用的内存非常低。
Both string and integer sequences are supported for bleu calculation.
同时支持 字符串和整数序列 进行bleu计算。
Test code is in unittest/test.py.
测试代码位于 unittest/test.py。
CPU：i7-8750H
# short str / 短字符串
score (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 2200000, 2200000) (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 2200000, 2200000)
1 process cost time 16.979528665542603
10 process cost time 3.5354034900665283
# long str / 长字符串
score (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 22000000, 22000000) (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 22000000, 22000000)
1 process cost time 103.8217351436615
10 process cost time 22.66322374343872
# short int list / 短整数序列
score (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 800000, 800000) (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 800000, 800000)
1 process cost time 4.874496936798096
10 process cost time 1.1751139163970947
# long int list / 长整数序列
score (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 16000000, 16000000) (1.0, [1.0, 1.0, 1.0, 1.0], 1.0, 1.0, 16000000, 16000000)
1 process cost time 47.34107685089111
10 process cost time 10.046519994735718
Don't input pytorch's tensor type. It will causes unnecessary memory consumption, and a lot of performance loss.
Please convert to numpy array or list type first.
不要传入 pytorch 的 tensor 类型，这会导致额外的内存消耗和大量的性能损失。
请先转换到 numpy数组 或 list类型。
from bleu_mp import compute_bleu
# str
pred_data = ['床前明月光，疑是地上霜', '举头望明月，低头思故乡'] * 1000
tgt_data = [['床前明月光，疑是地上霜'], ['举头望明月，低头思故乡', '静夜思']] * 1000
result = compute_bleu(pred_data, tgt_data)
print('bleu score', result[0])
# int list
pred_data = [[1, 2, 3, 4], [2, 3, 4, 5]] * 1000
tgt_data = [[[1, 2, 3, 4]], [[2, 3, 4, 5], [4, 5, 6]]] * 1000
result = compute_bleu(pred_data, tgt_data)
print('bleu score', result[0])
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent