[Feature Request]: Vercel Functions for Hobby can now run up to 60 seconds about chatgpt-next-web HOT 5 OPEN

About api/openai/[...path]:
If using edge, as long as the first byte arrives within 25 seconds, the streaming can continue indefinitely.
If using nodejs, regardless of when the first byte arrives, the total transmission time will be cut off if it exceeds the max-duration.
So the conclusion is to keep api/openai/[...path] as it is——set it as an edge function.

Hk-Gosuto#258 (comment)

from chatgpt-next-web.

@maxduke Could you please reopen the issue? I think there should be a way to avoid the problem of Edge Functions timing out if they don't receive an API response within 25 seconds. I found the following two links on GitHub, and both approaches set up a keep alive heartbeat in the router:

vercel/ai#487 (comment)

As the docs explain, Edge Functions don't have a maximum streaming time once they started streaming. So a possible workaround for this issue could be to start streaming empty strings with a 2 second interval so that we keep the connection alive while the LLM API call loads.

https://github.com/orgs/vercel/discussions/3553#discussioncomment-7131497

Specifically, apart from sending a response within 25 seconds (which I currently do) you actually have to keep sending messages periodically (unclear at what frequency, but 15 seconds resolves the issue) to keep the stream alive.

But I'm a programming newbie and can't really handle the code😂😂😂 I hope some skilled expert can implement the heartbeat to solve the 25s timeout issue. Thanks a lot!

reopened. Thanks a lot for the information. Let’s see.

from chatgpt-next-web.

The timeout is currently set to 60 seconds in nextchat, you might want to check if something else is causing the 25-second prompt to run out, Or do you have any error screenshots? Let's examine the causes of early timeouts

from chatgpt-next-web.

The timeout is currently set to 60 seconds in nextchat, you might want to check if something else is causing the 25-second prompt to run out, Or do you have any error screenshots? Let's examine the causes of early timeouts

So I think maybe Edge runtime is involved.
According to https://vercel.com/docs/functions/configuring-functions/duration.

You can't configure a maximum duration for functions using the Edge runtime. They can run indefinitely provided they send an initial response within 25 seconds.

from chatgpt-next-web.

@maxduke
Could you please reopen the issue? I think there should be a way to avoid the problem of Edge Functions timing out if they don't receive an API response within 25 seconds. I found the following two links on GitHub, and both approaches set up a keep alive heartbeat in the router:

vercel/ai#487 (comment)

As the docs explain, Edge Functions don't have a maximum streaming time once they started streaming. So a possible workaround for this issue could be to start streaming empty strings with a 2 second interval so that we keep the connection alive while the LLM API call loads.

https://github.com/orgs/vercel/discussions/3553#discussioncomment-7131497

Specifically, apart from sending a response within 25 seconds (which I currently do) you actually have to keep sending messages periodically (unclear at what frequency, but 15 seconds resolves the issue) to keep the stream alive.

But I'm a programming newbie and can't really handle the code😂😂😂
I hope some skilled expert can implement the heartbeat to solve the 25s timeout issue. Thanks a lot!

from chatgpt-next-web.