julesgm / parlai_searchengine Goto Github PK

View Code? Open in Web Editor NEW

132.0 132.0 50.0 404 KB

A search engine for ParlAI's BlenderBot project (and probably other ones as well)

License: Creative Commons Attribution 4.0 International

Python 69.94% Jupyter Notebook 30.06%

parlai_searchengine's People

Contributors

Stargazers

Watchers

Forkers

justinbuzzni bannng trendingtechnology xiuxiu hitchingsh mikful texhno rayshark vikaskapur alan-ai-learner scy6500 afilimonov tonylu0728 mohammedgomaa sublimotion ruthvik-17 jsedoc afiqmuzaffar alanwangvt xuchang-eva ppkyle yuanmeng1120 digitalcompanion peepleinc sujikarv daje0601 a-ichikura techthiyanes urvi-awasthi rosssong kuchidareo nguyenthekhoig7 jind11 kenlock specialistexperts darth-carrotpie mayankpm lohithsurya

parlai_searchengine's Issues

How do you make a web version?

How can you make a website localhost chat?

Fixes

I still need lxml and chardet to run so it would be great if you could add it to the requirements

When I do the query:

curl -X POST "http://0.0.0:8090" -d "q=Wandavision&n=5"

it crashes the search_server.py and to fix it I added the line so if the title is non existant it does not crash

if output_dict["title"]:
    output_dict["title"] = output_dict["title"].replace("\n", "").replace("\r", "")

Note, this error also crashes blenderbot2 and you can reproduce it by typing the following sentence into blenderbot2 while using your search_server.py

I like the TV show Wandavision

Note, I have made and tested this fix for both of the cases above and both now work properly. Can you make this to your github?

PS: Thank you so much for writing this server! It's been a life saver.

working fine cmd & not working webpage 501

As above, in cmd, the url address is output, but the error "code 501, message Unsupported method ('GET')" occurs. Why are you doing this?

How do I call your server from the MacOS terminal linux command line?

When trying to setup blenderbot2 with your search it does not seem to fetch the queries. On my Mac in the terminal window here's what I did:

$ git clone https://github.com/RodolfoFigueroa/ParlAI.git ./ParlAI
$ git clone https://github.com/pytorch/fairseq ./fairseq
$ pip install -r ./ParlAI/requirements.txt
$ cd ParlAI/
$ python ./setup.py develop
$ cd ../fairseq
$ pip install --editable ./
$ cd ..
$ git clone https://github.com/JulesGM/ParlAI_SearchEngine
$ pip install -r ./ParlAI_SearchEngine/requirements.txt
$ python ./ParlAI_SearchEngine/search_server.py serve --host 0.0.0.0:8080&
$ parlai interactive --model-file zoo:blenderbot2/blenderbot2_400M/model --search-server http://0.0.0.0:8080

10:39:39 | Current ParlAI commit: ad57e5281de76ed42a378bdd2bcdef2fafc54ab8
Enter [DONE] if you want to end the episode, [EXIT] to quit.
10:39:39 | creating task(s): interactive
Enter Your Message: Who went into space this week?
blenderbot2/lib/python3.7/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ../aten/src/ATen/native/BinaryOps.cpp:467.)
return torch.floor_divide(self, other)

[BlenderBot2Fid]: I'm not sure, but I'm pretty sure it was Elon Musk. POTENTIALLY_UNSAFE_
Enter Your Message: who won the superbowl this year
[BlenderBot2Fid]: I don't think Elon Musk has ever been to space, but it would be cool if he did.
Enter Your Message: [BlenderBot2Fid]: That's a good question. I don't know who won the Super Bowl this year, though.
Enter Your Message:

and when I try to curl I get the following error:

$ curl -X GET "http://0.0.0.0:8080?q=baseball"
127.0.0.1 - - [21/Jul/2021 10:32:34] code 501, message Unsupported method ('GET')
127.0.0.1 - - [21/Jul/2021 10:32:34] "GET /?q=baseball HTTP/1.1" 501 -

<title>Error response</title>

Error response

Error code: 501

Message: Unsupported method ('GET').

Error code explanation: HTTPStatus.NOT_IMPLEMENTED - Server does not support this operation.

any help would be appreciated. Thank you.

Failed To Connect To Port

While chatting with blenderbot with the $HOST search_server it stopped taking inputs after a while and when I checked with the command

!curl -X POST $HOST -d "q=biggest gpt model&n=1"

it threw the error

curl: (7) Failed to connect to 0.0.0.0 port 1111: Connection refused

Using ParlAi

Just wondering is there anyway I could make a class to get a response to simply gall the other procedures with ParlAI to make it easier to use? Also is there a way to increase the number of websites and ares of text it parses so that we can expand the knowledge?

Adding custom files with information to the search engine

I've tried the search engine using your colab.ipynb and It worked perfect after several trials.
It seemed that colab works better at some hours, whereas it fails when resources demand is high.

I am still wondering in which way can I add to your ParlAI_SearchEngine a custom directory with text files so the relevant documents can be found also in that directory. This way one can add custom information to the web search.
Any idea?

Accept pull request with Bing added, 10X speed up option, cleaner web page text returned to ParlAI, command line args?

@JulesGM I just did a big commit on my fork and would like to submit it as a pull request if you think you'd be willing to accept it. I was wondering if you could take a look at it first and also I'm open to making suggested changes before submitting including stylistic ones like how I coded the command line args

Commit: hitchingsh@14f0fba

ParlAI_SearchEngine license

Hi. Thanks for making this project public.

I want to fork this repository and add files, but the license is not specified. Could you please specify the license? Or is it set to standard copyright?

[Fix] Making port an integer

Thanks for making this repo!

I have been using it and noticed that I needed to make a small fix to make it work.

Need to convert port to integer otherwise it will throw an error.

def _parse_host(host: str) -> Tuple[str, str]:
    splitted = host.split(":")
    hostname = splitted[0]
    port = splitted[1] if len(splitted) > 1 else _DEFAULT_PORT
    port = int(port) # port is integer
    return hostname, port

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.