Code Monkey home page Code Monkey logo

Comments (3)

tluyben avatar tluyben commented on August 19, 2024

In https://github.com/cogentapps/chat-with-gpt/blob/1be33f5d0b0cdd8b35e7cab1a5d6bdffb6cec2ba/app/src/tokenizer.ts#L49there is code that should limit the messages, but maybe there is a bug or something? Trying to reproduce this.

from chat-with-gpt.

Frajder avatar Frajder commented on August 19, 2024

@tluyben Thank you for devoting your time to this issue. To reproduce this you just have to have a long enough conversation. Let's see how long mine was.

┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ user_msg_count=$(cat long_conversation.htm | grep '<div class="message by-user' | wc -l) && \                                                                          
assistant_msg_count=$(cat long_conversation.htm | grep '<div class="message by-assistant' | wc -l) && \
total_characters=$(python3 -c "import re; print(len(''.join(re.findall(r'<p>(.*?)<\/p>', open('long_conversation.htm', 'r').read(), re.DOTALL))))" | awk '{print $1}') && \
echo "Total number of user messages: $user_msg_count | Total number of assistant messages: $assistant_msg_count | Total characters: $total_characters"
Total number of user messages: 37 | Total number of assistant messages: 37 | Total characters: 28065

┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ # ok maybe that was not the most reliable way...

┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ scp w:~/dockers/chatgpt-ui/data/* .
chat.sqlite
object-store.sqlite
                                       
┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ user_msg_count=$(sqlite3 chat.sqlite "select count(id) from messages where chat_id = '8bb13ebe-8e68-400d-b920-bf1002f6cd04' and data like '%\"role\":\"user\"%';") && \
assistant_msg_count=$(sqlite3 chat.sqlite "select count(id) from messages where chat_id = '8bb13ebe-8e68-400d-b920-bf1002f6cd04' and data like '%\"role\":\"assistant\"%';") && \
total_characters=$(sqlite3 chat.sqlite "SELECT json_extract(data, '$.content') as content FROM messages WHERE chat_id = '8bb13ebe-8e68-400d-b920-bf1002f6cd04';" | awk '{ sum += length } END { print sum }')

┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ echo "Total number of user messages: $user_msg_count | Total number of assistant messages: $assistant_msg_count | Total characters: $total_characters"
Total number of user messages: 37 | Total number of assistant messages: 37 | Total characters: 25259

So, in my case, there were a total of 37 messages sent and received, for a combined total of 74. The total number of characters was somewhere between 25,000 and 30,000.

They said GPT-4 is good for coding. Let's give it a shot, see here: Problem-with-long-conversations_63-GPT4_attempt.md 😂😂😂

from chat-with-gpt.

cogentapps avatar cogentapps commented on August 19, 2024

Thanks for the detailed report!

This should be fixed by b6881c6.

from chat-with-gpt.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.