Code Monkey home page Code Monkey logo

Comments (3)

IanFan avatar IanFan commented on August 12, 2024 1

It makes sense, thank you for saving my day!

from cleverbird.

btfranklin avatar btfranklin commented on August 12, 2024

Hi @IanFan . You should be able to accomplish this in whatever code you're using to process the stream. For example, I have a project right now that switches the state of the UI depending on whether a stream is currently being received, and I do it with this snippet from the ViewModel:

    private func startStreamingResponse() {
        self.isCurrentlyStreamingResponse = true

        streamingTask = Task {
            defer {
                Task { @MainActor in
                    self.isCurrentlyStreamingResponse = false
                }
            }

            do {
                let stream = try await self.chatThread.complete()

                // Iterate through the stream, processing chunks as they arrive
                for try await chunk in stream {
                    Task { @MainActor in
                        let streamingMessage: ChatMessage
                        if self.messages.last?.role != .assistant {
                            streamingMessage = ChatMessage(role: .assistant, content: "")
                        } else {
                            streamingMessage = self.messages.popLast() ?? ChatMessage(role: .assistant, content: "")
                        }
                        let newContent = streamingMessage.content + chunk
                        self.messages.append(ChatMessage(role: .assistant, content: newContent, id: streamingMessage.id))
                    }
                }

            } catch {
                if Task.isCancelled {
                    self.chatThread.cancelStreaming()
                } else {
                    print("Error while processing stream: \(error)")
                }
            }
        }
    }

I also have a simple test application that I use to test the API as I'm developing it, and it has this snippet which has a clear spot that identifies when the stream has completed:

    private func testStreamingAPI() async {
        let openAIAPIConnection = OpenAIAPIConnection(apiKey: Secrets.OPENAI_API_KEY)
        streamableChatThread = ChatThread(connection: openAIAPIConnection).withStreaming()
            .addSystemMessage("You are an expert ornithologist.")
            .addUserMessage("What is the most common bird in North America?")

        do {
            print("request token count: \(try streamableChatThread!.tokenCount())")

            let stream = try await streamableChatThread!.complete()
            Task { @MainActor in
                apiResult = ""
                jsonResult = "{}"
            }

            // Iterate through the stream, processing chunks as they arrive
            for try await chunk in stream {
                Task { @MainActor in
                    apiResult += chunk
                }
            }
            print("Finished with stream!")
        } catch {
            print("Error while processing stream: \(error)")
        }

        Task { @MainActor in
            let lastMessage = streamableChatThread!.getNonSystemMessages().last!
            jsonResult = "id: \(lastMessage.id)\n\(lastMessage.prettyPrintedJSONString)"
        }
    }

Do these approaches satisfy your use case?

from cleverbird.

btfranklin avatar btfranklin commented on August 12, 2024

Fantastic! I'm going to go ahead and close this issue since it seems to be resolved, but please let me know if you run into anything else.

from cleverbird.

Related Issues (11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.