aws-samples / zero-administration-inference-with-aws-lambda-for-hugging-face Goto Github PK

Hi,
This is a great project. It worked for sentiment analysis example. However, my need is question answering use case.

I created myquestionanswer.py as below
import json
from transformers import pipeline

import json
from transformers import pipeline

summarizer = pipeline("question-answering")

def handler(event, context):
response = {
"statusCode": 200,
"body": summarizer(event['article'])[0]
}
return response

i.e. Only change I made is the string in pipeline parameter. Now it is 'question-answering'

What is the json input format to be given at Lambda test ? I tried the following. Both failed:

1. { "context": "My name is Rama. Sita is his wife", "question": " what is your name?"}
2. {"context": questions": [ "What is the name?", "Who is his wife?"] }

I saw other huggingface examples. These aren't applicable since they directly feed into model and are not helpful.

Thanks in advance.

I am following the instructions to deploy the model on AWS and get the following error when running cdk bootstrap:

 File "app.py", line 8, in <module>
    from aws_cdk import (
  File "/Users/alioskooei/opt/anaconda3/envs/nlp/lib/python3.7/site-packages/aws_cdk/__init__.py", line 22552, in <module>
    from . import aws_acmpca
  File "/Users/alioskooei/opt/anaconda3/envs/nlp/lib/python3.7/site-packages/aws_cdk/aws_acmpca/__init__.py", line 79, in <module>
    from ._jsii import *
  File "/Users/alioskooei/opt/anaconda3/envs/nlp/lib/python3.7/site-packages/aws_cdk/aws_acmpca/_jsii/__init__.py", line 11, in <module>
    import aws_cdk.core._jsii
  File "/Users/alioskooei/opt/anaconda3/envs/nlp/lib/python3.7/site-packages/aws_cdk/core/__init__.py", line 6643, in <module>
    class ConstructNode(metaclass=jsii.JSIIMeta, jsii_type="@aws-cdk/core.ConstructNode"):
  File "/Users/alioskooei/opt/anaconda3/envs/nlp/lib/python3.7/site-packages/aws_cdk/core/__init__.py", line 6694, in ConstructNode
    runtime_info: typing.Optional[aws_cdk.cx_api.RuntimeInfo] = None,
AttributeError: module 'aws_cdk' has no attribute 'cx_api'

Unfortunately, I could not find any tips online as to why I am seeing this error. I have installed the requirements according to the instructions and am using the following package versions:

CDK 2.12.0 (build c9786db
Node v16.3.0
Python 3.7.11
aws-cli/2.4.16
npm 7.15.1

I would appreciate any tips on how to resolve this issue. Thank you.

Thanks for your response on Q&A question in other issue.
With regard to multiple inferences, is there any precaution to take?

I was hoping that I just just call the model repeatedly in loop.

	import json
	from transformers import pipeline
	import requests
	question_answerer = pipeline("question-answering")
	
    def handler(event, context):
	    questionsetList['questionlist']
	    answerlist = []
	    for question in questionsetList:
		    answer = question_answerer({'question':question,'context':event['context']})
		    answerlist.push(answer)
            return jsonify({"Result": answerlist})

I got the following error on lambda test event.
START RequestId: b06fd2cb-54df-4807-91c8-34ea7cfb614f Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
/usr/local/lib/python3.6/dist-packages/joblib/_multiprocessing_helpers.py:45: UserWarning: [Errno 38] Function not implemented. joblib will operate in serial mode
warnings.warn('%s. joblib will operate in serial mode' % (e,))
questions before splitting by ? mark

1. Why are you troubled?~ 2.Who is the person to blame? ~3. How long are you frustrated about this?
   Traceback (most recent call last):
   File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
   "main", mod_spec)
   File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
   exec(code, run_globals)
   File "/function/awslambdaric/main.py", line 20, in
   main(sys.argv)
   File "/function/awslambdaric/main.py", line 16, in main
   bootstrap.run(app_root, handler, lambda_runtime_api_addr)
   File "/function/awslambdaric/bootstrap.py", line 415, in run
   log_sink,
   File "/function/awslambdaric/bootstrap.py", line 171, in handle_event_request
   log_error(error_result, log_sink)
   File "/function/awslambdaric/bootstrap.py", line 122, in log_error
   log_sink.log_error(error_message_lines)
   File "/function/awslambdaric/bootstrap.py", line 306, in log_error
   sys.stdout.write(error_message)
   File "/function/awslambdaric/bootstrap.py", line 283, in write
   self.stream.write(msg)
   UnicodeEncodeError: 'ascii' codec can't encode characters in position 79-80: ordinal not in range(128)
   END RequestId: b06fd2cb-54df-4807-91c8-34ea7cfb614f
   REPORT RequestId: b06fd2cb-54df-4807-91c8-34ea7cfb614f Duration: 22056.43 ms Billed Duration: 22057 ms Memory Size: 8096 MB Max Memory Used: 962 MB
   RequestId: b06fd2cb-54df-4807-91c8-34ea7cfb614f Error: Runtime exited with error: exit status 1
   Runtime.ExitError

It appeared like I can not call the model in a loop. In other implementations without pipeline I had used model in a loop.

Please suggest if there is any specific precaution like clean up required before calling for second question.

Thanks in advance.

The tutorial creates costs under Amazon Elastic Compute Cloud NatGateway.

There were two unassigned Elastic IP's on my account and I think it also builds NAT Gateway, which isn't part of free tier. Is there any way to run the tutorial without these costs?