DAM model in serving about dialogue HOT 3 CLOSED

Hi,

I guess you may want to fetch top-n responses from m candidates, where m >> n.

If yes, you can try a min-heap with size n and iterate all candidates, this would be much faster than sorting all elements.

Xiangyang

from dialogue.

Hi, Xiangyang,

Thanks for the tips. We were actually using the same technique to get top-k, with a much smaller complexity than sorting (which is nlogn). Beyond that, we were also trying to group response candidates to different clusters, to reduce the number of requests sending to the tf serving backend.

p.s. Nice presentation on the ACL18. I was there, but did not have a chance to talk to you.

Cheers

from dialogue.

from dialogue.