Code Monkey home page Code Monkey logo

curlconverter.github.io's Introduction

Transpile curl commands into C, C#, ColdFusion, Clojure, Dart, Elixir, Go, HTTPie, Java, JavaScript, Julia, Kotlin, Lua, MATLAB, Objective-C, OCaml, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Swift, Wget, Ansible, HAR, HTTP or JSON.

Try it on curlconverter.com or as a drop-in curl replacement:

$ curlconverter --data "hello=world" example.com
import requests

data = {
    'hello': 'world',
}

response = requests.post('http://example.com', data=data)

Features:

  • Implements a lot of curl's argument parsing logic
    • Knows about all 255 curl arguments but most are ignored
    • Supports shortening -O -v -X POST to -OvXPOST
    • --data @filename generates code that reads that file and @- reads stdin
  • Understands Bash syntax
    • ANSI-C quoted strings
    • stdin redirects and heredocs
    • Generated code reads environment variables and runs subcommands
    • Ignores comments
    • Reports syntax errors
  • Converts JSON data to native objects
  • Warns about issues with the conversion

Limitations:

  • Only HTTP is supported
  • Code generators for other languages are less thorough than the Python generator
  • curl doesn't follow redirects or decompress gzip-compressed responses by default, but the generated code will do whatever the default is for that runtime, to keep it shorter. For example Python's Requests library follows redirects by default, so unless you explicitly set the redirect policy with -L/--location/--no-location, the generated code will not handle redirects the same way as the curl command
  • Shell variables can arbitrarily change how the command would be parsed at runtime. The command curl $VAR can do anything, depending on what's in $VAR. curlconverter assumes that environment variables don't contain characters that would affect parsing
  • Only simple subcommands such as curl $(echo example.com) work, more complicated subcommands (such as nested commands or subcommands that redirect the output) won't generate valid code
  • The Bash parser doesn't support all Bash syntax
  • and much more

Install

Install the command line tool with

npm install --global curlconverter

Install the JavaScript library for use in your own projects with

npm install curlconverter

curlconverter requires Node 12+.

Usage

Usage from the command line

curlconverter acts as a drop-in replacement for curl. Take any curl command, change "curl" to "curlconverter" and it will print code instead of making the request

$ curlconverter example.com
import requests

response = requests.get('http://example.com')

To read the curl command from stdin, pass -

$ echo 'curl example.com' | curlconverter -
import requests

response = requests.get('http://example.com')

Choose the output language by passing --language <language>. The options are

  • ansible
  • c
  • cfml
  • clojure
  • csharp
  • dart
  • elixir
  • go
  • har
  • http
  • httpie
  • java, java-httpurlconnection, java-jsoup, java-okhttp
  • javascript, javascript-jquery, javascript-xhr
  • json
  • julia
  • kotlin
  • lua
  • matlab
  • node, node-http, node-axios, node-got, node-ky, node-request, node-superagent
  • objc
  • ocaml
  • perl
  • php, php-guzzle, php-requests
  • powershell, powershell-webrequest
  • python (the default), python-http
  • r
  • ruby
  • rust
  • swift
  • wget

--verbose enables printing of conversion warnings and error tracebacks.

Usage as a library

The JavaScript API is a bunch of functions that can take either a string of Bash code or an array of already-parsed arguments (like process.argv) and return a string with the resulting program:

import * as curlconverter from 'curlconverter';

curlconverter.toPython('curl example.com');
curlconverter.toPython(['curl', 'example.com']);
// "import requests\n\nresponse = requests.get('http://example.com')\n"

Note: add "type": "module", to your package.json for the import statement above to work. curlconverter must be imported as an ES module with import this way and not with require() because it uses top-level await.

There's a corresponding set of functions that also return an array of warnings if there are any issues with the conversion:

curlconverter.toPythonWarn('curl ftp://example.com');
curlconverter.toPythonWarn(['curl', 'ftp://example.com']);
// [
//   "import requests\n\nresponse = requests.get('ftp://example.com')\n",
//   [ [ 'bad-scheme', 'Protocol "ftp" not supported' ] ]
// ]

If you want to host curlconverter yourself and use it in the browser, it needs two WASM files to work, tree-sitter.wasm and tree-sitter-bash.wasm, which it will request from the root directory of your web server. If you are hosting a static website and using Webpack, you need to copy these files from the node_modules/ directory to your server's root directory in order to serve them. You can look at the webpack.config.js for curlconverter.com to see how this is done. You will also need to set {module: {experiments: {topLevelAwait: true}}} in your webpack.config.js.

Usage in VS Code

There's a VS Code extension that adds a "Paste cURL as <language>" option to the right-click menu: https://marketplace.visualstudio.com/items?itemName=curlconverter.curlconverter. It doesn't support the same languages, curl arguments or Bash syntax as the current version because it has to use an old version of curlconverter.

Contributing

See CONTRIBUTING.md

License

MIT © Nick Carneiro

curlconverter.github.io's People

Contributors

cf512 avatar jgroom33 avatar lemonlemmings avatar nickcarneiro avatar verhovsky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

curlconverter.github.io's Issues

Error: Cannot find module 'yargs'

Hey,

I'm trying to run the app locally, following the steps in the readme.

npm install - finished successfully.

But the next step npm run-script build, fails with error -

user@mymachine:/curltorequests$ npm run-script build

> [email protected] build /curltorequests
> ./build.sh

Error: Cannot find module 'yargs' from '/curltorequests/node_modules/curlconverter'
    at /curltorequests/node_modules/browser-resolve/node_modules/resolve/lib/async.js:46:17
    at process (/curltorequests/node_modules/browser-resolve/node_modules/resolve/lib/async.js:173:43)
    at ondir (/curltorequests/node_modules/browser-resolve/node_modules/resolve/lib/async.js:188:17)
    at load (/curltorequests/node_modules/browser-resolve/node_modules/resolve/lib/async.js:69:43)
    at onex (/curltorequests/node_modules/browser-resolve/node_modules/resolve/lib/async.js:92:31)
    at /curltorequests/node_modules/browser-resolve/node_modules/resolve/lib/async.js:22:47
    at FSReqWrap.oncomplete (fs.js:152:21)

Thanks

Safari re-requests preloaded Wasm files

For parsing Bash, curlconverter requires 2 wasm files /tree-sitter.wasm and /tree-sitter-bash.wasm. In index.html we have the bundle of all the JavaScript code

<script src="/main.js"></script>

and in that code, some of the first lines it executes is

await Parser.init(); // requests /tree-sitter.wasm
const Bash = await Parser.Language.load("/tree-sitter-bash.wasm");

https://github.com/curlconverter/curlconverter/blob/e4b6fb74d508a946d1311dc63cf0ec853d9a6c1d/src/shell/webParser.ts#L7-L8

So to load the website, you

  1. get the HTML file which tells you to download the JavaScript file
  2. download the JavaScript file, start executing it and it tells you to download tree-sitter.wasm
  3. download tree-sitter.wasm execute it and then keep executing the javascript, which next tells you to download tree-sitter-bash.wasm
  4. download the tree-sitter-bash.wasm

and you've finally loaded the website. To avoid the extra round trips from having to parse and execute JavaScript to know that you need to load more Wasm files

Screenshot 2024-01-30 at 02 01 56

I added these lines to the HTML to tell it to fetch the Wasm files as soon as it parses the HTML

<link rel="preload" href="/tree-sitter.wasm" type="application/wasm" as="fetch" crossorigin>
<link rel="preload" href="/tree-sitter-bash.wasm" type="application/wasm" as="fetch" crossorigin>

and this works on Chrome and Firefox

Screenshot 2024-01-30 at 02 03 19

but on Safari it loads the wasm files twice, once from the preload and once from the JavaScript

Screenshot 2024-01-30 at 02 14 56

and the console says

The resource http://localhost:8000/tree-sitter.wasm was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing.
The resource http://localhost:8000/tree-sitter-bash.wasm was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing.

I tried changing crossorigin to ="", ="anonymous" and ="use-credentials". I tried removing the type="application/wasm", removing the crossorigin= and removing both type= and crossorigin=.

One difference I see is in the Network tab under "Type" it says they're preloaded as xhr but then loaded by the JavaScript as fetch. The only difference in the sent request is the prefetch sends a Origin: http://localhost:8000 header if you have crossorigin="".

npm install errors

➜  curlconverter.github.io git:(index-html-update) ✗ npm -v
9.7.2
➜  curlconverter.github.io git:(index-html-update) ✗ npm install
npm WARN deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
npm WARN deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
npm WARN deprecated [email protected]: This package is broken and no longer maintained. 'mkdirp' itself supports promises now, please switch to that.
npm WARN deprecated [email protected]: this library is no longer supported
npm ERR! code 1
npm ERR! git dep preparation failed
npm ERR! command /usr/local/Cellar/node/20.3.1_1/bin/node /usr/local/lib/node_modules/npm/bin/npm-cli.js install --force --cache=/Users/connor/.npm --prefer-offline=false --prefer-online=false --offline=false --no-progress --no-save --no-audit --include=dev --include=peer --include=optional --no-package-lock-only --no-dry-run
npm ERR! > [email protected] prepare
npm ERR! > husky install && npm run compile && tree-sitter build-wasm node_modules/@curlconverter/tree-sitter-bash && mv tree-sitter-bash.wasm dist/
npm ERR!
npm ERR! husky - git command not found, skipping install
npm ERR!
npm ERR! > [email protected] compile
npm ERR! > tsc
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! fatal: not a git repository (or any of the parent directories): .git
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! You must have either emcc or docker on your PATH to run this command
npm ERR! npm ERR! code 1
npm ERR! npm ERR! path /Users/connor/.npm/_cacache/tmp/git-clonew0coPs
npm ERR! npm ERR! command failed
npm ERR! npm ERR! command sh -c husky install && npm run compile && tree-sitter build-wasm node_modules/@curlconverter/tree-sitter-bash && mv tree-sitter-bash.wasm dist/
npm ERR!
npm ERR! npm ERR! A complete log of this run can be found in: /Users/connor/.npm/_logs/2023-07-01T20_35_25_486Z-debug-0.log

npm ERR! A complete log of this run can be found in: /Users/connor/.npm/_logs/2023-07-01T20_35_01_053Z-debug-0.log

Release

Bump release to publish the ansible uri output

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.