I try the example code with the attribute parse_context, but i get the error that tell me that this attribute was not known. Did I miss something ?

curl_string  = 'curl "https://www.invesco.co.uk/uk/products/perpetual-income-and-growth-investment-trust-plc" -H "authority: www.invesco.co.uk" -H "cache-control: max-age=0" -H "upgrade-insecure-requests: 1" -H "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" -H "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3" -H "accept-encoding: gzip, deflate, br" -H "accept-language: en-US,en;q=0.9" -H "cookie: VISITOR=returning; NEW_VISITOR=new; InvescoAcceptedTerms=true; InvescoAcceptedCookies=true; InvescoAcceptedCookieNotice=^\^"2019-07-19T02:35:05.258-05:00^\^"; nivid=iXYdKKgc2SX-bgohul3; _ga=GA1.3.2111614667.1563521709; visitor_id481331=117043127; visitor_id481331-hash=4fb6817866eba661fda08e792283ea7febdf047dbd6c66ece3c9362ef12510f34976c846ce3c80c14ec4e4f84861aae4538705fe; LPVID=UxMGU4YjI0YTJlYjM3NDUy; InvescoUserCookieName=investor; JSESSIONID=FB36DBB7399523ECE3F5A36C0914B22B; nisid=1; _gid=GA1.3.1472917646.1563778854; LPSID-32683207=tGw2wBB3TCOU3UTW6ZgjQw; _pk_ses.7.b8b7=*; _pk_id.7.b8b7=f78f951728c734ee.1563521711.3.1563779098.1563778856." --compressed'

import uncurl
request_context = uncurl.parse_context(curl_string)

In [16]: request_context.cookies
Out[16]: 
OrderedDict([('VISITOR', 'returning'),
             ('NEW_VISITOR', 'new'),
             ('InvescoAcceptedCookies', 'true'),
             ('InvescoAcceptedTerms', 'true')])

in above snippet lot of cookies are not being captured by parse_context method of this module.
if you see above curl string can find lot of other cookies as well like InvescoAcceptedCookieNotice and other. can use this website https://curl.trillworks.com to see all cookies

I had the same issue as #33 and could fix it by running

pip install git+https://github.com/spulec/uncurl.git

According to https://pypi.org/project/uncurl/#history the pypi version is outdated.

Does it support to use command line in windows system?
I tried type uncurl in CMD and git bash, neither succeeds.

G'day there, I am using this python package in the hope that it does something identical to curl.trillworks.com.

Mostly works hunky-dory, but I note that the website mentioned above returns a headers dictionary in the correct headers order, whereas the uncurl.parse() function gives me an unordered dictionary, in alphabetical order. It it possible to change this to an ordered dictionary?

Input:

uncurl.parse("curl 'https://www.amazon.com/' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/77.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1'")

Output:
'requests.get("https://www.amazon.com/",\n headers={\n "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",\n "Accept-Language": "en-US,en;q=0.5",\n "Connection": "keep-alive",\n "DNT": "1",\n "Upgrade-Insecure-Requests": "1",\n "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/77.0"\n },\n cookies={},\n)'

Desired output:

headers = { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/77.0', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.5', 'DNT': '1', 'Connection': 'keep-alive', 'Upgrade-Insecure-Requests': '1', }

Cheers and thanks for your work

This happens when the argument of -b starts with two dashes. Example:

$ echo "curl url -b '--'" | uncurl
usage: uncurl [-h] [-d DATA] [-b DATA_BINARY] [-X X] [-H HEADER]
              [--compressed] [--insecure]
              command url
uncurl: error: argument -b/--data-binary: expected one argument

when i run curl -T thefile.txt ...., I get unrecognized arguments -T

It appears that the tool was developed specifically for use in combination with "Copy as cURL" in Chrome Developer Tool. And it is a great tool for that :) I used it on many occasions.

This week I was planning/trying to use it for a different kind of problem. I wrote some cURL commands that access a webservice API, saved these as small shell scripts named 1.sh, 2.sh, etc., and verified that these worked. I then wanted to use these as templates. Let my Python script read the .sh files, parse them with uncurl, and make small adaptions before sending the request with the requests library.

But so far my experience is that uncurl is unable to handle most of the shell scripts I wrote. Eg. arguments like --cert and --key are not supported. Also uncurl cares more about the order of arguments than curl does.

It would be helpful if the documentation contained a "warning" that it should work for all (or most) of cURL commands that Chrome Dev Tool will produce, but can not be expected to work with any curl command. And a very rough overview of what is supported and what is not.

If you have any interest in this, I could write my suggestion for a short paragraph about this, and post as a comment to this issue? (I have not yet learned how to do pull requests and actually contribute to repos)

Sincerely :)

When there's binary in the curl request, sometimes the quotation marks screw up shlex and it won't parse properly.

For example

curl -i -s -k -X $'POST'
-H $'Host: ayk-web.mo.konami.net' -H $'User-Agent: UnityPlayer/2020.3.18f1 (UnityWebRequest/1.0, libcurl/7.75.0-DEV)' -H $'Accept: /' -H $'Accept-Encoding: gzip, deflate' -H $'Content-Type: application/octet-stream' -H $'X_acts: User.home' -H $'Atoken: 23528e425359819ac4252c83e73cbc934095818015c1ff83843c9f62ff1d' -H $'X-Unity-Version: 2020.3.18f1' -H $'Content-Length: 194'
--data $'`\x00^\x00\xd6\x1eq\xe1\xbc[$\xf5\xba\x05O\xd7\xf1'\xad=o\xcf\x8a5\xe4[\x0a2r\xc9\xe4\x1c\xdc/WL\x1b\xe1\x13_AjI?\x9d\xe4\xa7\x0f\x8dbSTE\xd2j\xe2{\x8d\xb4\xc7\xf7/_C\xa6B\x07=c\xa4\xd0\x12\xd7\xdf\x01\x98b\xa1\x0f]\xbaK\xb7a\x19o\x84\x1c\xfe\x95\x96\xfc\xb8\x08\xa0\xf4\xb9\xbb\xf1\x0d{"acts":[{"act":"User.home","id":13}],"v":"1.0.2","ua":"Android/7.1.2/SM-G973N","h":558161692}'
$'https://ayk-web.mo.konami.net/ayk/api/User.home'

Traceback (most recent call last):
File "<pyshell#46>", line 1, in
uncurl.parse(curl)
File "D:\python\lib\site-packages\uncurl\api.py", line 74, in parse
parsed_context = parse_context(curl_command)
File "D:\python\lib\site-packages\uncurl\api.py", line 30, in parse_context
tokens = shlex.split(curl_command)
File "D:\python\lib\shlex.py", line 305, in split
return list(lex)
File "D:\python\lib\shlex.py", line 295, in next
token = self.get_token()
File "D:\python\lib\shlex.py", line 105, in get_token
raw = self.read_token()
File "D:\python\lib\shlex.py", line 187, in read_token
raise ValueError("No closing quotation")
ValueError: No closing quotation

some commands where the binary is a bit nicer, will parse just fine. But others will not.

Is there any bandaid solution to this?

When submitting a request to a website with an expired certificate, a --insecure flag is passed in the request, This flag is not properly parsed, as is evident in this example:

uncurl.parse(curl "http://" --insecure)

This returns the following:

requests.get("http://",
    headers={},
    cookies={},
{}verify=False
)

It should instead return:

requests.get("http://",
    headers={},
    cookies={},
    verify=False
)

main.py [-h] [-d DATA] [-b DATA_BINARY] [-X X]
[-H HEADER] [--compressed] [--insecure]
command url
main.py: error: unrecognized arguments: --user https://example.com/jenkins/job/F5Routing/job/ISAM-B2E-
Dev/job/sts/build?delay=0sec

curl --location --request GET 'localhost:8000/cs/api/user/info/?user_id=c0f8f66d-bd4e-4305-a536-6de3b2da926c'
--header 'Authorization: Token 92c7c270bd43e79a6546d8b80de1c1ce4e593724'
--header 'Cookie: csrftoken=tmbcb5KPaZFBU4PavGa4Z5hTrrbcL1eSlgrxGVqXsHV8njXVuIRZdxJK1NVmLvgP'

This is a valid curl. But when I passed it to uncurl.parse_context(), it threw an error:
"unrecognized arguments: --location --request ........."

how can convert string data to dict

Chrome will let you export a network call as curl by right clicking it -> copy -> copy as curl.

Chrome also likes to export long cookies like such:

curl 'https://someurl.com' \
  -H 'Connection: keep-alive' \
  -H 'Cache-Control: max-age=0' \
  -H 'sec-ch-ua: "Google Chrome"; v="83"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'DNT: 1' \
  -H 'Upgrade-Insecure-Requests: 1' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4093.3 Safari/537.36' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'Sec-Fetch-Site: none' \
  -H 'Sec-Fetch-Mode: navigate' \
  -H 'Sec-Fetch-User: ?1' \
  -H 'Sec-Fetch-Dest: document' \
  -H 'Accept-Language: en-US,en;q=0.9,it;q=0.8,la;q=0.7' \
  -H $'Cookie: somereallyreallylongcookie=true;' \
  --compressed

This gets spit out as:

requests.get("https://someurl.com",
    headers={
        "$Cookie": "somereallyreallylongcookie=true;",
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "Accept-Language": "en-US,en;q=0.9,it;q=0.8,la;q=0.7",
        "Cache-Control": "max-age=0",
        "Connection": "keep-alive",
        "DNT": "1",
        "Sec-Fetch-Dest": "document",
        "Sec-Fetch-Mode": "navigate",
        "Sec-Fetch-Site": "none",
        "Sec-Fetch-User": "?1",
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4093.3 Safari/537.36",
        "sec-ch-ua": "\"Google Chrome\"; v=\"83\"",
        "sec-ch-ua-mobile": "?0"
    },
    cookies={},
)

by uncurl. Notice the $Cookie instead of Cookie.

When I try to install uncurl in windows, I encounter some error I don't why happen this,
Any help for this?

File "<string>", line 1, in <module>
  File "C:\Users\jackao\AppData\Local\Temp\pip-install-t_vtyo2u\xerox\setup.py", line 6, in <module>
    import xerox
  File "C:\Users\jackao\AppData\Local\Temp\pip-install-t_vtyo2u\xerox\xerox\__init__.py", line 1, in <module>
    from .core import *
  File "C:\Users\jackao\AppData\Local\Temp\pip-install-t_vtyo2u\xerox\xerox\core.py", line 13, in <module>
    from .win import *
  File "C:\Users\jackao\AppData\Local\Temp\pip-install-t_vtyo2u\xerox\xerox\win.py", line 12, in <module>
    raise Pywin32NotFound
xerox.base.Pywin32NotFound

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in C:\Users\jackao\AppData\Local\Temp\pip-install-t_vtyo2u\xerox\

Tried both copy as curl-bash and copy as curl-cmd (working on windows machine here, ananconda 3)

Here is the call:

uncurl "curl 'https://www.bestbuy.com/cart/api/v1/addToCart'
-H 'authority: www.bestbuy.com'
-H 'accept: application/json'
-H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36'
-H 'content-type: application/json; charset=UTF-8'
-H 'origin: https://www.bestbuy.com'
-H 'sec-fetch-site: same-origin'
-H 'sec-fetch-mode: cors'
-H 'sec-fetch-dest: empty'
-H 'referer: https://www.bestbuy.com/site/evga-ko-ultra-gaming-nvidia-geforce-rtx-2060-6gb-gddr6-pci-express-3-0-graphics-card-black-gray/6403801.p?skuId=6403801'
-H 'accept-language: en-US,en;q=0.9'
-H 'cookie: redacted'
--data-binary '{"items":[{"skuId":"6403801"}]}'
--compressed"

File "", line 13
--data-binary '{"items":[{"skuId":"6403801"}]}' \

^
SyntaxError: invalid syntax

在django中使用，会报错，提示：usage: manage.py [-h] [-d DATA] [-b DATA_BINARY] [-X X] [-H HEADER]
[--compressed] [-k] [--user USER] [-i] [-s]
command url
manage.py: error: unrecognized arguments:

When an empty string (curl_command = '') is passed to uncurl.parse_context(curl_command) a python script ends abnormally showing this error:

usage: [-h] [-d DATA] [-b DATA_BINARY] [-X X] [-H HEADER] [--compressed] [-k] [--user USER] [-i] [-s] command url
: error: the following arguments are required: command, url

The abnormal termination could be fixed with something like

    try:
        parsed_args = parser.parse_args(tokens)
    except:
        pass

in

What do you think about supporting other "backends", like urlllib2?

@spulec Thanks for this helpful tool! I'm having some trouble with the xerox setup and unfortunately its latest release is four years old see. Would it be possible to switch to pyperclip?

It would be nice to accept HAR or raw HTTP requests. Maybe we would need to rename the library though...

/.venv/bin/uncurl "curl --request POST
--url http://localhost:8080/customer
--header 'Content-Type: application/json'
--data '{
"customer": {
"customer_id": "1",
"customer_type": "regular"
}
}'"
usage: uncurl [-h] [-d DATA] [-b DATA_BINARY] [-X X] [-H HEADER] [--compressed] [-k] [--user USER] [-i] [-s] command url
uncurl: error: unrecognized arguments: --request --url http://localhost:8080/customer

when I run

uncurl "curl -H 'PID: 20000079' -H 'MT: 4' -H 'DivideVersion: 1.0' -H 'SupPhone: Redmi Note 3' -H 'SupFirm: 5.0.2' -H 'IMEI: wx_app' -H 'IMSI: wx_app' -H 'SessionId: ' -H 'CUID: wx_app' -H 'ProtocolVersion: 1.0' -H 'Sign: 7876480679c3cfe9ec0f82da290f0e0e' -H 'Accept: /' -H 'BodyEncryptType: 0' -H 'User-Agent: Mozilla/5.0 (Linux; Android 6.0.1; OPPO R9s Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/67.0.3396.87 Mobile Safari/537.36 hap/1.0/oppo com.nearme.instant.platform/2.1.0beta1 com.felink.quickapp.reader/1.0.3 ({"packageName":"com.oppo.market","type":"other","extra":{}})' -H 'Content-Type: text/plain; charset=utf-8' -H 'Host: pandahomeios.ifjing.com' --data-binary '{"CateID":"508","PageIndex":1,"PageSize":30}' --compressed 'http://pandahomeios.ifjing.com/action.ashx/otheraction/9028'"

I got

requests.post("http://pandahomeios.ifjing.com/action.ashx/otheraction/9028",
data='{CateID:508,PageIndex:1,PageSize:30}',
headers={
"Accept": "/",
"BodyEncryptType": "0",
"CUID": "wx_app",
"Content-Type": "text/plain; charset=utf-8",
"DivideVersion": "1.0",
"Host": "pandahomeios.ifjing.com",
"IMEI": "wx_app",
"IMSI": "wx_app",
"MT": "4",
"PID": "20000079",
"ProtocolVersion": "1.0",
"SessionId": "",
"Sign": "7876480679c3cfe9ec0f82da290f0e0e",
"SupFirm": "5.0.2",
"SupPhone": "Redmi Note 3",
"User-Agent": "Mozilla/5.0 (Linux; Android 6.0.1; OPPO R9s Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/67.0.3396.87 Mobile Safari/537.36 hap/1.0/oppo com.nearme.instant.platform/2.1.0beta1 com.felink.quickapp.reader/1.0.3 ({packageName:com.oppo.market,type:other,extra:{}})"
},
cookies={},
)

I run it in jupyter notebook and I didn't get what I want.

then I try convert that curl command here https://curl.trillworks.com/

I got the following, and I run it in jupyter notebook I got what I want.

import requests

headers = {
'PID': '20000079',
'MT': '4',
'DivideVersion': '1.0',
'SupPhone': 'Redmi Note 3',
'SupFirm': '5.0.2',
'IMEI': 'wx_app',
'IMSI': 'wx_app',
'SessionId': '',
'CUID': 'wx_app',
'ProtocolVersion': '1.0',
'Sign': '7876480679c3cfe9ec0f82da290f0e0e',
'Accept': '/',
'BodyEncryptType': '0',
'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0.1; OPPO R9s Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/67.0.3396.87 Mobile Safari/537.36 hap/1.0/oppo com.nearme.instant.platform/2.1.0beta1 com.felink.quickapp.reader/1.0.3 ({"packageName":"com.oppo.market","type":"other","extra":{}})',
'Content-Type': 'text/plain; charset=utf-8',
'Host': 'pandahomeios.ifjing.com',
}

data = '{"CateID":"508","PageIndex":1,"PageSize":30}'

response = requests.post('http://pandahomeios.ifjing.com/action.ashx/otheraction/9028', headers=headers, data=data)

after comparing them I found

curl command: --data-binary '{"CateID":"508","PageIndex":1,"PageSize":30}'
uncurl turns it into: data='{CateID:508,PageIndex:1,PageSize:30}'
that website turns it into: data = '{"CateID":"508","PageIndex":1,"PageSize":30}'

is that a bug?

I was trying to convert a curl request with a key --unix-socket, but I am getting the below error.

import uncurl
print (uncurl.parse("curl --unix-socket /var/run/docker.sock http:/v1.30/images/json"))

Error:
usage: uncurl_trial.py [-h] [-d DATA] [-b DATA_BINARY] [-H HEADER]
[--compressed]
command url
uncurl_trial.py: error: unrecognized arguments: --unix-socket http:/v1.30/images/json

AttributeError: module 'uncurl' has no attribute 'parse'

Version 0.0.8 does not include parse_context and it seems it has been in master for quite a while. Would it be possible to publish the latest version? I really need to make a proper request call from a curl command. Thanks!

The example from the doc does not work. Any idea why?

In [12]: import uncurl

In [13]: uncurl "curl 'https://pypi.python.org/pypi/uncurl' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla
    ...: /5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36' -H 'Accept: text/html,application/xh
    ...: tml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'Cookie: foo=bar;' -H 'Connection: keep-alive' --compressed"
  File "<ipython-input-13-018c847865b1>", line 1
    uncurl "curl 'https://pypi.python.org/pypi/uncurl' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'Cookie: foo=bar;' -H 'Connection: keep-alive' --compressed"
           ^
SyntaxError: invalid syntax

uncurl==0.0.10
Python 3.9.1