How to use the model with sentence-transformer for inference? about consert HOT 2 OPEN

From the error message, it seems that the transformers and sentence_transformers packages used by your code are the ones installed in the python library (site-packages), rather than the modified version in my repository. You should run your code in the root dir of my repository or copy the modified transformers and sentence_transformers folders to your root dir.

If you want to encode the sentence by averaging the embeddings of last two layers (last2avg) or the first and the last layers (firstlastavg), you can use the following function in eval.py:

def load_model(model_path: str, last2avg: bool = False, firstlastavg: bool = False):
    model = SentenceTransformer(model_path)
    if last2avg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_last_2_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    if firstlastavg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_first_last_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    logging.info("Model successfully loaded")
    return model

from consert.

From the error message, it seems that the transformers and sentence_transformers packages used by your code are the ones installed in the python library (site-packages), rather than the modified version in my repository. You should run your code in the root dir of my repository or copy the modified transformers and sentence_transformers folders to your root dir.

If you want to encode the sentence by averaging the embeddings of last two layers (last2avg) or the first and the last layers (firstlastavg), you can use the following function in eval.py:

def load_model(model_path: str, last2avg: bool = False, firstlastavg: bool = False):
    model = SentenceTransformer(model_path)
    if last2avg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_last_2_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    if firstlastavg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_first_last_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    logging.info("Model successfully loaded")
    return model

Thanks a lot!

from consert.