Computing Embeddings from Transformer? about biophi HOT 3 CLOSED

I'll take a look at this and submit a PR! Always happy to contribute :) I'll get around to it in a couple of weeks, if that's okay.

from biophi.

Hi @nickbhat there's a function intended for this:

However, looks like the return_all_hiddens arg is actually ignored :) Do you want to create a pull request by any chance? I'm always happy to get more contributors.

It would be a simple fix here:

Btw, this way we can only get the embeddings, if you also wanted the attention weights, that would require some changes to fairseq code.

from biophi.

Fixed by #21

Example usage:

from biophi.humanization.methods.sapiens.predict import sapiens_predict_seq

pred, extra = sapiens_predict_seq(
    seq=seq, # seq should be the variable region sequence only
    chain_type='H', # chain type is H or L
    return_all_hiddens=True
)

embeddings_per_layer = extra['inner_states']

from biophi.