Reactive AI

TensorBLEU - GPU-based vectorized BLEU score for in-training optimization Today I published my next paper, that's introducing TensorBLEU - https://lnkd.in/dcRrPFPf, the optimization dedicated for Reinforcement Learning rewards based BLEU score. It achieved over 10x speed improvement over NLTK's version on small T4 GPU and even 40x improvement with A100 GPU. That's not exactly linguistically correct BLEU, because it's not based on text n-grams, but on token ids. It's a conscious choice to skip computationally expensive token decoding, in case when it serves only as a reward signal. It was previously possible with NLTK's `sentence_bleu`, but required moving tensors with token ids from GPU to CPU, converting them to lists and calculating in python loop, creating significant performance bottlenecks. In our case, in Reactive AI (https://lnkd.in/dSiFCWSQ) we are using BLEU as a part of the reward in Memory Reinforcement Learning (MRL) of Reactive Transformer models (https://lnkd.in/dxKiJ_iD), combined with cosine similarity. To rate memory quality, we calculate BLEU and cosine similarity between generated answer and reference answer from dataset, as well as between generated answer and previous interaction(s), to ensure that current answer includes some information from previous time-steps. Cosine similarity is calculated on GPU, but BLEU calculation with NLTK have to be performed on CPU, with a lot of data moving and conversion. When all the episode (generating batch of answers, memory updates and reward calculation) takes i.e. 6 seconds, even 0.5s for the reward is noticeable, so we decided to optimize it. TensorBLEU calculation is performed on GPU for all the batch on sentence or corpus level - `tensor_sentence_bleu` or `tensor_corpus_bleu` from `rxlm.metrics.tensorbleu` (https://lnkd.in/dwHChCyj) Please check the paper and upvote it, if you like it :)