Code Monkey home page Code Monkey logo

azure-search-query-classification's Introduction

Azure Search - Query Classification

Search experiences are all around us, empowering us to quickly find the documents, websites, products and answers that we're looking for.

For years, search engines have employed complex machine learning techniques to more deeply understand what users are searching for and to help them find it. These techniques enable search engines to semantically understand a user's query, to know that Debussy was a classical musician, that a dog is a pet, and that Python and JavaScript are programming languages.

In catalog search, semantically understanding the intent and extracting the entities in the query is crucial to provide useful results. Whenever the users type "blue girl cardigan", they expect the search engine to correctly identify the color as well as the specific product type and their intent to get back only female products.

Though these techniques help us find what we're looking for, they have not been available to enterprises who want to build their own semantic search capabilities.

Azure Cognitive Search aims to provide semantic understanding capabilities to search, so that any enterprise can build much more natural search experiences.

About this repo

This sample demonstrates how we can use word embeddings from Large Language Models (LLMs) with Azure Cognitive Search to create search experiences that understand the relationships between the entities in a query and the products in a catalog.

The project combines OpenAI embedding models and Azure Cognitive Search to enhance search relevancy by adding a semantic understanding layer in query composition. Whenever a user searches for something, it is crucial to understand his/her intent to provide a tailored result set, considering both the entities in the query and the semantic similarity to the available products.

The solution creates an embedding representation of a product catalog to improve search relevancy by applying an implicit semantic classification step whenever a user submits a new query. The code extracts product data from a search index, computes and stores the embeddings based on OpenAI model (ada, babbage or curie) and applies the same embedding technique to every new unseen query coming to the engine. The query embeddings are evaluated against the embeddings matrix to determine the semantic similarity score and return possible filters criteria, boosting criteria or query expansion with confidence thresholds.

This project also converts natural language into Lucene queries using a few-shot approach and OpenAI text generation models as an alternative for query composition.

The code is split into two different logical components to facilitate re-use and an easy integration in any application layer of your choice:

  • docker-api: core component computing the embedding matrix for the product catalog and exposing an API for query embedding and semantic scoring
  • docker-web: a sample experimentation UI to visualize embeddings results and threshold to test your dataset and defining the right filtering, boosting and query expansion logic

Both components are dockerized for easy deployment on multiple platforms and integration with existing applications.

Statement of Purpose

The purpose of this repository is to grow the understanding of using Large Language Models in Azure Cognitive Search by providing an example of implementation and references to support the Microsoft Build conference in 2022. It is not intended to be a released product. Therefore, this repository is not for discussing OpenAI API, Azure Cognitive Search or requesting new features.

How semantic search changes the results set

Let's see how semantic understanding changes the results set with a sample query.

The "girl yellow cardigan" query is analyzed and three entities are extracted as the most probable match, also with their attribute key. So, "yellow" is classified as color, "cardigan" as a product type and "girl" clearly indicates the need for "female" as product gender. semantic-understanding

Using this information, a filter on the product gender is applied to return just products classified as "Female" and a boost is pushing "yellow" products on top of the results set. You can easily compare the results in the following image. semantic-understanding

Requirements

Running the App Locally (without Docker)

  1. git clone the repo: git clone https://github.com/microsoft/azure-search-query-classification and open the project folder
  2. Create a .env file in the root directory of the project, copying the contents of the .env.example file See Configure the .env file section
  3. Gather the projects' dependencies for the backend component
    • Windows users: Run pip install -r docker-api\code\requirements.txt
    • MacOs users: Run pip install -r docker-api/code/requirements.txt
  4. Gather the projects' dependencies for the frontend component
    • Windows users: Run pip install -r docker-web\code\requirements.txt
    • MacOs users: Run pip install -r docker-web/code/requirements.txt
  5. Run python local-run.py to serve the backend and launch the web application. NOTE for MacOs users: by default no alias for python is defined. In this case, please run python3 local-run.py

Running the App Locally (with Docker)

  1. Check whether Docker is running
  2. git clone the repo: git clone https://github.com/microsoft/azure-search-query-classification and open the project folder
  3. Create a .env file in the root directory of the project, copying the contents of the .env.example file See Configure the .env file section
  4. Create the server docker image
    • Windows users: Run docker build docker-api\. -t YOUR_REGISTRY/YOUR_REPO/YOUR_API:TAG
    • MacOs users: Run docker build docker-api/. -t YOUR_REGISTRY/YOUR_REPO/YOUR_API:TAG
  5. Create the client docker image
    • Windows users: Run docker build docker-web\. -t YOUR_REGISTRY/YOUR_REPO/YOUR_WEB:TAG
    • MacOs users: Run docker build docker-web/. -t YOUR_REGISTRY/YOUR_REPO/YOUR_WEB:TAG
  6. Modify the docker-compose.yml to point to your images tags as defined in step 3 and step 4
  7. Run docker compose up to serve the backend and launch the web application.

Configure the .env file

Please use your own settings in the fields marked as "TO UPDATE" in the Note column in the following table.


App Setting Value Note
search_service YOUR_AZURE_COGNITIVE_SEARCH_SERVICE TO UPDATE Azure Cognitive Search service name e.g. https://XXX.search.windows.net use just XXX.
index_name YOUR_AZURE_COGNITIVE_SEARCH_INDEX TO UPDATE A new index that will be created by the code in your Azure Cognitive Search resource
api_key YOUR_AZURE_COGNITIVE_SEARCH_API_KEY TO UPDATE Azure Cognitive Search Admin Key
api_version 2021-04-30-Preview Azure Cognitive Search API Version
LoadSampleData true Load sample data in Azure Cognitive Search index
sample_data_url Link to download sample data if LoadSampleData is true
SentenceTransformer msmarco-distilbert-dot-v5, all-mpnet-base-v2, nq-distilbert-base-v1, all-MiniLM-L6-v2 List of all the models for compute the embeddings. You can add any Sentence Transformers
GPT_3 text-search-curie-query-001, text-search-babbage-query-001, text-search-ada-query-001 List of all the models for compute the embeddings. You can add any GPT-3 embedding model
OPENAI_API_KEY YOUR_OPENAI_API_KEY TO UPDATE OpenAI GPT-3 Key
fields 
 [
    {
      "name": "name",
      "type": "Edm.String"
    },
    {
      "name": "quality",
      "type": "Collection(Edm.String)"
    },
    {
      "name": "style",
      "type": "Collection(Edm.String)"
    },
    {
      "name": "gender",
      "type": "Edm.String"
    },
    {
      "name": "colors",
      "type": "Collection(Edm.String)"
    },
    {
      "name": "type",
      "type": "Edm.String"
    }
 ]
JSON version of all the fileds to be used for computing embeddings
select articleId,type,name,description,quality,style,gender,colors List of all Azure Cognitive Search index fields to visualize in the UI results table
searchFields articleId,type,name,description,quality,style,gender,colors List of all Azure Cognitive Search fields to search in
boost type,name,quality,style,gender,colors List of all Azure Cognivite Search fields to be used for attribute boosting in the UI
filters type,name,quality,style,gender,colors List of all Azure Cognivite Search fields to be used for attribute filtering in the UI
prod_url http://api:80 or http://localhost:8000 URL for the Server side component (docker-api microservices) as defined in the docker compose file When executing locally without Docker, use http://localhost:8000 instead
gpt3_prompt girl yellow cardian -> $search=girl yellow cardigan&$filter=color eq 'Yellow' and productTypeName eq 'Cardigan' and productGender eq 'Female'\nblu man t-shirt -> $search=blu man t-shirt&$filter=color eq 'Blue' and productTypeName eq 'T-shirt' and productGender eq 'Male'\nblack hoodie -> $search=black hoodie&$filter=color eq 'Black' and productTypeName eq 'Hoodie'\ncotton black pant -> $search=cotton black pant&$filter=color eq 'black' and productTypeName eq 'Black' and productQuality eq 'cotton'\nlight blue cotton polo shirt -> $search=light blue cotton polo shirt&$filter=color eq 'Light Blue' and productQuality eq 'Cotton' and productTypeName eq 'Polo Shirt'\ngreen cashmere polo shirt -> $search=green cashmere polo shirt&$filter=color eq 'Green' and productQuality eq 'Cashmere' and productTypeName eq 'Polo Shirt'\n OpenAI Prompt for query generation

Using the App

Example queries:

  • girl yellow cardigan
  • women white belted coat

When running the application, you can use the UI to tweak the filters and boosts logic according to your needs.

You can enable filtering and/or boosting using the menu on the left-hand side or modify the threshold for each field in the "Threshold definition" section.

In the following image, you can compare the results on the left-hand side ("Keyword-based Search") with those on the right-hand side ("Search with Semantic Understanding"), using the following settings:

  • Boost on extracted colors with confidence > 0.85 (light blue marker)
  • Filter on extracted gender with confidence > 0.75 (green marker)

semantic-understanding

The UI provides a sample playground to tweak the threshold parameters and define your own logic to be embedded in your application query layer.

How to use embeddings

The core of the repo is using the encode function from sentence_transformers library and get_embeddings from openai library to compute the embedding representations of the product attributes available in a sample product catalog.

# docker-api\code\utilities\utils.py
def compute_terms_embeddings(model,terms, path=os.getcwd()):
    if model['family'] == SentenceTransformer:
        start_time = datetime.datetime.now()
        terms_embedding = model['model'].encode(list(map(lambda x: x['value'], terms)))
        end_time = datetime.datetime.now()
        difference_in_ms = (end_time - start_time).total_seconds()*1000
        logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
        np.save(os.path.join(path, 'embeddings' , model['name']), terms_embedding)
        return terms_embedding

    elif model['family'] == 'GPT-3':
        df_terms = pd.DataFrame(terms)        
        start_time = datetime.datetime.now()
        df_terms['gpt_3_search'] = df_terms.value.apply(lambda x: get_embedding(x, engine= model['name']))
        end_time = datetime.datetime.now()
        difference_in_ms = (end_time - start_time).total_seconds()*1000
        logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
        df_terms.to_csv(os.path.join(path, 'embeddings' , f"{model['name']}.csv"))
        return df_terms

Whenever a user submits a query, the same libraries are used to compute the embeddings for the whole sentence and the single words in it and semantically rank this terms list against all available product attributes.

# docker-api\code\utilities\utils.py
def process_query(query, terms_embedding, model, terms):
    if model['family'] == SentenceTransformer:
        start_time = datetime.datetime.now()
        query_embedding = model['model'].encode(query)
        end_time = datetime.datetime.now()
        difference_in_ms = (end_time - start_time).total_seconds()*1000
        logging.info(query, '(encoded in', difference_in_ms, 'ms)')

        logging.info(f'Using cos_similarity for {model["name"]}')
        scores = util.cos_sim(query_embedding, terms_embedding).numpy()[0]

        terms_score = []
        counter = 0
        for score in scores:
            terms_score.append(
                {
                    "term": terms[counter]['value'],
                    "key" : terms[counter]['key'],
                    "score" : str(score),
                    "query" : query,
                    "model_name" : model['name']
                }
            )
            counter+=1

        return sorted(terms_score, key = lambda i: i['score'],reverse=True)

    else:

        start_time = datetime.datetime.now()
        query_embedding = get_embedding(query, engine=model['name'])
        end_time = datetime.datetime.now()
        difference_in_ms = (end_time - start_time).total_seconds()*1000
        logging.info(query, '(encoded in', difference_in_ms, 'ms)')
        
        terms_embedding['score'] = terms_embedding.gpt_3_search.apply(lambda x: cosine_similarity(x, query_embedding))
        
        res = terms_embedding.sort_values('score', ascending=False).head(5)

        res.drop(['gpt_3_search'], axis=1, inplace=True)
        res['query'] = query
        res['model_name'] = model['name']
        
        return res.to_dict('records')

Build a Docker image

Server

  1. docker build docker-api\. -t YOUR_REGISTRY/YOUR_REPO/YOUR_API:TAG
  2. docker run -p 80:80 --env-file .env -t YOUR_REGISTRY/YOUR_REPO/YOUR_API:TAG

Client

  1. docker build docker-web\. -t YOUR_REGISTRY/YOUR_REPO/YOUR_WEB:TAG
  2. docker run -p 80:80 --env-file .env -t YOUR_REGISTRY/YOUR_REPO/YOUR_WEB:TAG

Debugging

To debug the web application, you can debug with VSCode debugger.

Server

FastAPI Debugging tool

Client

Streamlit Debugging

Undestand the Code

Server (docker-api)

  • api.py is the main entry point for the app, it uses FastAPI to serve RESTful APIs.
  • utilities is the module with utilities to extract product catalog data, compute embeddings and the semantic similarity score for a new query

Client (docker-web)

  • ui.py is the entry to bootstrap the Streamlit web application
  • utilities is the module with utilities for interact with the search index and the server-side

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

azure-search-query-classification's People

Contributors

microsoft-github-operations[bot] avatar microsoftopensource avatar ruoccofabrizio avatar vivihung avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

isabella232 trentbrucegithub adiazcan dearborn-open-ai

azure-search-query-classification's Issues

Error finding pretrained model

Trying to run the docker container out of the box on Windows and getting an error where it is not able to find the path to pretrained model:

C:\Users\johwu\Documents\playground\azure-search-query-classification>docker compose up
[+] Running 2/2
 - Container azure-search-query-classification-api-1  Recreated                                                                                                   0.9s
 - Container azure-search-query-classification-web-1  Recreated                                                                                                   0.2s
Attaching to azure-search-query-classification-api-1, azure-search-query-classification-web-1
azure-search-query-classification-api-1  | 2022-05-10 00:37:10,762 [INFO] Request URL: 'https://csciblob.blob.core.windows.net/product-data/output.json?sv=REDACTED&st=REDACTED&se=REDACTED&sr=REDACTED&sp=REDACTED&sig=REDACTED'
azure-search-query-classification-api-1  | Request method: 'GET'
azure-search-query-classification-api-1  | Request headers:
azure-search-query-classification-api-1  |     'x-ms-range': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-version': 'REDACTED'
azure-search-query-classification-api-1  |     'Accept': 'application/xml'
azure-search-query-classification-api-1  |     'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.10 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.28)'
azure-search-query-classification-api-1  |     'x-ms-date': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-client-request-id': '542bf7f8-cff9-11ec-be7e-0242ac120002'
azure-search-query-classification-api-1  | No body was attached to the request
azure-search-query-classification-api-1  | 2022-05-10 00:37:11,034 [INFO] Response status: 206
azure-search-query-classification-api-1  | Response headers:
azure-search-query-classification-api-1  |     'Content-Length': '318002'
azure-search-query-classification-api-1  |     'Content-Type': 'application/json'
azure-search-query-classification-api-1  |     'Content-Range': 'REDACTED'
azure-search-query-classification-api-1  |     'Last-Modified': 'Wed, 13 Apr 2022 07:57:32 GMT'
azure-search-query-classification-api-1  |     'Accept-Ranges': 'REDACTED'
azure-search-query-classification-api-1  |     'ETag': '"0x8DA1D2343C0A57A"'
azure-search-query-classification-api-1  |     'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
azure-search-query-classification-api-1  |     'x-ms-request-id': 'eade4213-c01e-007d-7106-640321000000'
azure-search-query-classification-api-1  |     'x-ms-client-request-id': '542bf7f8-cff9-11ec-be7e-0242ac120002'
azure-search-query-classification-api-1  |     'x-ms-version': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-creation-time': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-blob-content-md5': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-lease-status': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-lease-state': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-blob-type': 'REDACTED'
azure-search-query-classification-api-1  |     'Content-Disposition': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-server-encrypted': 'REDACTED'
azure-search-query-classification-api-1  |     'Date': 'Tue, 10 May 2022 00:37:10 GMT'
azure-search-query-classification-api-1  | 2022-05-10 00:37:11,141 [INFO] Downloaded sample data
azure-search-query-classification-api-1  | 2022-05-10 00:37:12,956 [INFO] Index created successfully 201
azure-search-query-classification-api-1  | 2022-05-10 00:37:12,960 [INFO] data_path:/usr/local/src/myscripts/data/articles
azure-search-query-classification-api-1  | 2022-05-10 00:37:16,251 [INFO] Creating new terms map from /usr/local/src/myscripts
azure-search-query-classification-api-1  | 2022-05-10 00:37:16,254 [INFO] data_path:/usr/local/src/myscripts/data/articles
azure-search-query-classification-api-1  | 2022-05-10 00:37:16,870 [INFO] full_data: 2000
azure-search-query-classification-api-1  | 2022-05-10 00:37:16,892 [INFO] Load pretrained SentenceTransformer: msmarco-distilbert-dot-v5
azure-search-query-classification-api-1  | 2022-05-10 00:37:35,438 [INFO] Use pytorch device: cpu
azure-search-query-classification-api-1  | 2022-05-10 00:37:35,440 [INFO] [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/msmarco-distilbert-dot-v5.npy'
``

README Instructions for MacOS

Some issues encountered following the README.md instructions on a relatively clean install of MacOS:

Running app locally (without docker) section, steps 3 & 4. Documentation says to run pip install -r docker-api\code\requirements.txt, but the backslashes get removed, so it results in a file not found error. If you replace the backslashes with forward slashes, it works: pip install -r docker-api/code/requirements.txt

By default, there does not seem to be an alias for python with the version that comes installed on MacOS - instead, had to change the command in step 5 to python3 local-run.py

README Instructions for .env Configuration

It may be helpful to provide more explicit instruction as to where to get the values needed for the .env file:

index_name - should clarify that this should not be the name of an existing index as it'll encounter an error - instead, use a new index name and it'll be created and populated.

MacOS - Setup (w/ Docker): Failed to run docker images

Following the docker steps and using different slash when building the images - e.g. docker build docker-web/. -t ..... But hit file not found errors when running the api image. Pull one error as an example:

[Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/msmarco-distilbert-dot-v5.npy'

Full logs

(base) vivihung@Viviens-MacBook-Pro azure-search-query-classification % docker compose up
[+] Running 3/3
 ⠿ Network azure-search-query-classification_default  Created                                                                                                                0.1s
 ⠿ Container azure-search-query-classification-api-1  Created                                                                                                                0.1s
 ⠿ Container azure-search-query-classification-web-1  Created                                                                                                                0.2s
Attaching to azure-search-query-classification-api-1, azure-search-query-classification-web-1
azure-search-query-classification-api-1  | 2022-04-19 23:31:24,216 [INFO] Request URL: 'https://csciblob.blob.core.windows.net/product-data/output.json?sv=REDACTED&st=REDACTED&se=REDACTED&sr=REDACTED&sp=REDACTED&sig=REDACTED'
azure-search-query-classification-api-1  | Request method: 'GET'
azure-search-query-classification-api-1  | Request headers:
azure-search-query-classification-api-1  |     'x-ms-range': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-version': 'REDACTED'
azure-search-query-classification-api-1  |     'Accept': 'application/xml'
azure-search-query-classification-api-1  |     'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.10 (Linux-5.10.104-linuxkit-x86_64-with-glibc2.28)'
azure-search-query-classification-api-1  |     'x-ms-date': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-client-request-id': 'd396009a-c038-11ec-976d-0242ac120002'
azure-search-query-classification-api-1  | No body was attached to the request
azure-search-query-classification-api-1  | 2022-04-19 23:31:24,468 [INFO] Response status: 206
azure-search-query-classification-api-1  | Response headers:
azure-search-query-classification-api-1  |     'Content-Length': '318002'
azure-search-query-classification-api-1  |     'Content-Type': 'application/json'
azure-search-query-classification-api-1  |     'Content-Range': 'REDACTED'
azure-search-query-classification-api-1  |     'Last-Modified': 'Wed, 13 Apr 2022 07:57:32 GMT'
azure-search-query-classification-api-1  |     'Accept-Ranges': 'REDACTED'
azure-search-query-classification-api-1  |     'ETag': '"0x8DA1D2343C0A57A"'
azure-search-query-classification-api-1  |     'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
azure-search-query-classification-api-1  |     'x-ms-request-id': '42ec70fa-401e-0011-0245-54e8b6000000'
azure-search-query-classification-api-1  |     'x-ms-client-request-id': 'd396009a-c038-11ec-976d-0242ac120002'
azure-search-query-classification-api-1  |     'x-ms-version': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-creation-time': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-blob-content-md5': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-lease-status': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-lease-state': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-blob-type': 'REDACTED'
azure-search-query-classification-api-1  |     'Content-Disposition': 'REDACTED'
azure-search-query-classification-api-1  |     'x-ms-server-encrypted': 'REDACTED'
azure-search-query-classification-api-1  |     'Date': 'Tue, 19 Apr 2022 23:31:24 GMT'
azure-search-query-classification-api-1  | 2022-04-19 23:31:24,570 [INFO] Downloaded sample data
azure-search-query-classification-api-1  | 2022-04-19 23:31:26,026 [INFO] Index created successfully 204
azure-search-query-classification-api-1  | 2022-04-19 23:31:26,027 [INFO] data_path:/usr/local/src/myscripts/data/articles
azure-search-query-classification-api-1  | 2022-04-19 23:31:27,519 [INFO] Creating new terms map from /usr/local/src/myscripts
azure-search-query-classification-api-1  | 2022-04-19 23:31:27,520 [INFO] data_path:/usr/local/src/myscripts/data/articles
azure-search-query-classification-api-1  | 2022-04-19 23:31:27,712 [INFO] full_data: 2000
azure-search-query-classification-api-1  | 2022-04-19 23:31:27,720 [INFO] Load pretrained SentenceTransformer: msmarco-distilbert-dot-v5
azure-search-query-classification-api-1  | 2022-04-19 23:31:33,679 [INFO] Use pytorch device: cpu
azure-search-query-classification-api-1  | 2022-04-19 23:31:33,680 [INFO] [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/msmarco-distilbert-dot-v5.npy'
azure-search-query-classification-api-1  | 2022-04-19 23:31:33,680 [INFO] Creating new embeddings with model msmarco-distilbert-dot-v5
Batches: 100%|██████████| 9/9 [00:00<00:00,  9.15it/s]
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/msmarco-distilbert-dot-v5.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 994.161, 'ms)')
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/msmarco-distilbert-dot-v5.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 994.161, 'ms)')
azure-search-query-classification-api-1  | 2022-04-19 23:31:34,680 [INFO] Load pretrained SentenceTransformer: all-mpnet-base-v2
azure-search-query-classification-api-1  | 2022-04-19 23:31:41,322 [INFO] Use pytorch device: cpu
azure-search-query-classification-api-1  | 2022-04-19 23:31:41,322 [INFO] [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/all-mpnet-base-v2.npy'
azure-search-query-classification-api-1  | 2022-04-19 23:31:41,322 [INFO] Creating new embeddings with model all-mpnet-base-v2
Batches: 100%|██████████| 9/9 [00:01<00:00,  6.90it/s]
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/all-mpnet-base-v2.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 1309.463, 'ms)')
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/all-mpnet-base-v2.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 1309.463, 'ms)')
azure-search-query-classification-api-1  | 2022-04-19 23:31:42,635 [INFO] Load pretrained SentenceTransformer: nq-distilbert-base-v1
azure-search-query-classification-api-1  | 2022-04-19 23:31:48,049 [INFO] Use pytorch device: cpu
azure-search-query-classification-api-1  | 2022-04-19 23:31:48,050 [INFO] [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/nq-distilbert-base-v1.npy'
azure-search-query-classification-api-1  | 2022-04-19 23:31:48,050 [INFO] Creating new embeddings with model nq-distilbert-base-v1
Batches: 100%|██████████| 9/9 [00:00<00:00, 12.35it/s]
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/nq-distilbert-base-v1.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 733.0640000000001, 'ms)')
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/nq-distilbert-base-v1.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 733.0640000000001, 'ms)')
azure-search-query-classification-api-1  | 2022-04-19 23:31:48,786 [INFO] Load pretrained SentenceTransformer: all-MiniLM-L6-v2
azure-search-query-classification-api-1  | 2022-04-19 23:31:54,432 [INFO] Use pytorch device: cpu
azure-search-query-classification-api-1  | 2022-04-19 23:31:54,432 [INFO] [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/all-MiniLM-L6-v2.npy'
azure-search-query-classification-api-1  | 2022-04-19 23:31:54,433 [INFO] Creating new embeddings with model all-MiniLM-L6-v2
Batches: 100%|██████████| 9/9 [00:00<00:00, 37.56it/s]
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/all-MiniLM-L6-v2.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 244.118, 'ms)')
azure-search-query-classification-api-1  | --- Logging error ---
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 161, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = np.load(os.path.join(path,"embeddings", f"{model['name']}.npy"))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/numpy/lib/npyio.py", line 417, in load
azure-search-query-classification-api-1  |     fid = stack.enter_context(open(os_fspath(file), "rb"))
azure-search-query-classification-api-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/src/myscripts/embeddings/all-MiniLM-L6-v2.npy'
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | During handling of the above exception, another exception occurred:
azure-search-query-classification-api-1  | 
azure-search-query-classification-api-1  | Traceback (most recent call last):
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
azure-search-query-classification-api-1  |     msg = self.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
azure-search-query-classification-api-1  |     return fmt.format(record)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
azure-search-query-classification-api-1  |     record.message = record.getMessage()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
azure-search-query-classification-api-1  |     msg = msg % self.args
azure-search-query-classification-api-1  | TypeError: not all arguments converted during string formatting
azure-search-query-classification-api-1  | Call stack:
azure-search-query-classification-api-1  |   File "/usr/local/bin/uvicorn", line 8, in <module>
azure-search-query-classification-api-1  |     sys.exit(main())
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
azure-search-query-classification-api-1  |     return self.main(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
azure-search-query-classification-api-1  |     rv = self.invoke(ctx)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
azure-search-query-classification-api-1  |     return ctx.invoke(self.callback, **ctx.params)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
azure-search-query-classification-api-1  |     return __callback(*args, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 425, in main
azure-search-query-classification-api-1  |     run(app, **kwargs)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/main.py", line 447, in run
azure-search-query-classification-api-1  |     server.run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
azure-search-query-classification-api-1  |     return asyncio.run(self.serve(sockets=sockets))
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
azure-search-query-classification-api-1  |     return loop.run_until_complete(main)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
azure-search-query-classification-api-1  |     self.run_forever()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
azure-search-query-classification-api-1  |     self._run_once()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
azure-search-query-classification-api-1  |     handle._run()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
azure-search-query-classification-api-1  |     self._context.run(self._callback, *self._args)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
azure-search-query-classification-api-1  |     config.load()
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
azure-search-query-classification-api-1  |     self.loaded_app = import_from_string(self.app)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
azure-search-query-classification-api-1  |     module = importlib.import_module(module_str)
azure-search-query-classification-api-1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
azure-search-query-classification-api-1  |     return _bootstrap._gcd_import(name[level:], package, level)
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
azure-search-query-classification-api-1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./api.py", line 34, in <module>
azure-search-query-classification-api-1  |     all_models, terms = utils.load_models(
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 189, in load_models
azure-search-query-classification-api-1  |     t_emb = get_model_embeddings(mod, terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 166, in get_model_embeddings
azure-search-query-classification-api-1  |     terms_embedding = compute_terms_embeddings(model,terms)
azure-search-query-classification-api-1  |   File "/usr/local/src/myscripts/./utilities/utils.py", line 83, in compute_terms_embeddings
azure-search-query-classification-api-1  |     logging.info("terms", '(encoded in', difference_in_ms, 'ms)')
azure-search-query-classification-api-1  | Message: 'terms'
azure-search-query-classification-api-1  | Arguments: ('(encoded in', 244.118, 'ms)')
azure-search-query-classification-api-1  | 2022-04-19 23:31:54,687 [INFO] Creating new embeddings with model curie-search-query-msft
azure-search-query-classification-api-1  | 2022-04-19 23:31:54,690 [INFO] message='Request to OpenAI API' method=post path=https://api.openai.com/v1/engines/curie-search-query-msft/embeddings
container for service "api" is unhealthy

Error running app in browser

Once running (app locally w/o docker) on MacOS, it did not launch the browser as indicated by the README. When I navigate to http://127.0.01:8000, I get the following error:

{"detail":[{"loc":["query","query"],"msg":"field required","type":"value_error.missing"},{"loc":["query","model_name"],"msg":"field required","type":"value_error.missing"}]}

Error in console: INFO: 127.0.0.1:53591 - "GET / HTTP/1.1" 422 Unprocessable Entity

README Instructions - Not clear what to do after launching

Once I finally got everything working (Running App Locally w/ Docker), I launched the web ui (manually on MacOS) but it wasn't clear what to do next. I changed the search query to mens blue t-shirt and submitted, but the semantic search results didn't seem to take the semantic values into consideration - top results were all just type shirt, no gender, and colors were random (and mostly not blue). It wasn't clear that I needed to enable various boosts/filters (which I finally discovered by clicking around). Some step-by-step instructions of things to try and what to observe would be really helpful here.

Easy to put in wrong value for YOUR_AZURE_COGNITIVE_SEARCH_SERVICE

When specifying environment variable for YOUR_AZURE_COGNITIVE_SEARCH_SERVICE, the description says Azure cognitive service name, e.g. https://XXX.search.windows.net, use just XXX. However, it was too easy to just see the first part of the sentence and ignore the last part of "use just XXX"

Probably would be more clear to just say it's "The name given to the Azure Cognitive service resource"?

README Instructions for Docker

In the Running App Locally section, for those who are not familiar with Docker, it's not clear what to do with steps 3 & 4. I'm assuming I need to set up some sort of Docker service somewhere, but some guidance here would be helpful - even if it's a pointer to documentation and/or tutorial (although inline help might be better).

Error changing boost values in configuration

Ran into the following error when attempting to update the boost value for 'color' in the configuration:

TypeError: update_scoring_profile() takes from 0 to 1 positional arguments but 6 were given

Traceback:

File "/usr/local/lib/python3.9/site-packages/streamlit/script_runner.py", line 376, in _run_script
    self._session_state.call_callbacks()
File "/usr/local/lib/python3.9/site-packages/streamlit/state/session_state.py", line 484, in call_callbacks
    self._new_widget_state.call_callback(wid)
File "/usr/local/lib/python3.9/site-packages/streamlit/state/session_state.py", line 228, in call_callback
    callback(*args, **kwargs)

Need to specify Python minimum version (and any other version requirements)

I had issues on step 3 of https://github.com/microsoft/azure-search-query-classification#running-the-app-locally-without-docker.

I believe that there may have been a Python version issue (Python version command said that I was using 3.7.8, but I got a deprecation notice for 2.7 on the following line). Assuming that it's using Python 2.7, the command fails. If this is the case, we should set a minimum version requirement for Python.

C:\Code\azure-search-query-classification [jennmar/suggestions +1 ~2 -0 !]> python --version
Python 3.7.8
C:\Code\azure-search-query-classification [jennmar/suggestions +1 ~2 -0 !]> pip install -r docker-api\code\requirements.txt
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Looking in links: https://download.pytorch.org/whl/cpu/torch_stable.html
ERROR: Could not find a version that satisfies the requirement torch==1.10.1 (from -r docker-api\code\requirements.txt (line 2)) (from versions: none)
ERROR: No matching distribution found for torch==1.10.1 (from -r docker-api\code\requirements.txt (line 2))
C:\Code\azure-search-query-classification [jennmar/suggestions +1 ~2 -0 !]>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.