When using CPU as a device, after trying various prompts, gradio returns with the foll

<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="15

<a href="https://github.com/gradio-app/gradio/issues/3185" data-hovercard

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Something went wrong Unexpected token '<', "<html> <h"... is not valid JSON,about tloen/alpaca-lora

Comments (11)

DanqingZ commented on July 24, 2024 35

gradio-app/gradio#3185
change
demo.launch(share=True)
to
demo.queue().launch(share=True)
Enabling the queue is required for inference times > 60 seconds: https://gradio.app/key-features/#queuing

from alpaca-lora.

norsigmacom commented on July 24, 2024

could it be a server timeout ? how long did the task take ?

from alpaca-lora.

ThatCoffeeGuy commented on July 24, 2024

Reasonable time, about 4-5 seconds, I doubt it'd be a timeout.

from alpaca-lora.

Anuvathan commented on July 24, 2024

Change run type type as GPU and try.

from alpaca-lora.

ThatCoffeeGuy commented on July 24, 2024

Change run type type as GPU and try.

I don't have enough VRAM for the larger models, hence why trying with CPU.

from alpaca-lora.

Anuvathan commented on July 24, 2024

demo = gr.Interface(
fn=process_document,
inputs="image",
outputs="json",
# live = True,
title=" OCR",
enable_queue=True,
cache_examples=False,
debug=True,
)
demo.launch(share=True)

This is my gradio API function code. Previously I had the same issue. But after I change to GPU in colab It's working.

and it's not for this repository. It's for other different type.

from alpaca-lora.

zychyz95822 commented on July 24, 2024

gradio-app/gradio#3185 change demo.launch(share=True) to demo.queue().launch(share=True) Enabling the queue is required for inference times > 60 seconds: https://gradio.app/key-features/#queuing

fix my same problem

from alpaca-lora.

18620146591 commented on July 24, 2024

I have changed to demo.queue().launch(share=True), but still cannot fix my problem

from alpaca-lora.

chanheehi commented on July 24, 2024

gradio-app/gradio#3185 추론 시간 > 60초에 대기열 활성화가 필요함 으로 변경 : https://gradio.app/key-features/#queuing demo.launch(share=True) demo.queue().launch(share=True)

thanks for your comments

from alpaca-lora.

swapnil3597 commented on July 24, 2024

Hi @DanqingZ, I'm running gradio Application with FastAPI, how to handle it in such cases?

Following is the configuration of my gradio application:

from fastapi import FastAPI
import gradio as gr

app = FastAPI()
app = gr.mount_gradio_app(app, demo, path="/my_custom_route")

Getting following error:

Connection errored out.

How to handle requests which take more than 60s in case of using gradio with FastAPI?

from alpaca-lora.

arditobryan commented on July 24, 2024

Hi @DanqingZ, I'm running gradio Application with FastAPI, how to handle it in such cases?

Following is the configuration of my gradio application:
from fastapi import FastAPI
import gradio as gr

app = FastAPI()
app = gr.mount_gradio_app(app, demo, path="/my_custom_route")
Getting following error:
Connection errored out.
How to handle requests which take more than 60s in case of using gradio with FastAPI?

I am experiencing the same issue. I tried passing in the gradio instance as queue:
app = gr.mount_gradio_app(app, demo.queue(), path="/my_custom_route")
but is not working

from alpaca-lora.

Recommend Projects