Code Monkey home page Code Monkey logo

cake.curl's Introduction

Cake.Curl

NuGet NuGet Downloads AppVeyor Travis CI Tests Coverage

Cake.Curl is a cross-platform add-in for Cake that allows to transfer files to and from remote URLs using curl.

Cross-platform

Cake.Curl targets the .NET Standard 2.0 and the .NET Framework 4.6. As such, it will run on Linux, macOS and Windows.

Prerequisites

In order to use Cake.Curl, you will need to have a copy of the curl executable for your OS. It doesn't have to be in a specific location; as long as it's included in your PATH environment variable, Cake will find it.

Usage

The purpose of this add-in is to expose the functionality of curl to the Cake DSL by being a very thin wrapper around its command line interface; this means that you can use Cake.Curl in the same way as you would normally use curl, only with a different interface.

Here are a few examples of how some common usage scenarios would look like in a Cake script.

First of all, you need to import Cake.Curl in your build script by using the add-in directive:

#addin Cake.Curl

Downloading Files

Downloading a text file from a remote HTTP server onto the working directory:

Task("Download")
    .Does(() =>
{
    CurlDownloadFile(new Uri("http://host/file.txt"));
});

Downloading a sequence of text files numbered between 1 and 10 from a remote HTTP server onto the working directory:

Task("Download")
    .Does(() =>
{
    CurlDownloadFile(new Uri("http://host/file[1-10].txt"));
});

Downloading a text file from a remote HTTP server onto the working directory and giving it a different name:

Task("Download")
    .Does(() =>
{
    CurlDownloadFile(
        new Uri("http://host/file.txt"),
        new CurlDownloadSettings
        {
            OutputPaths = new FilePath[] { "renamed.txt" }
        });
});

Downloading multiple files concurrently from different servers onto the working directory:

Task("Download")
    .Does(() =>
{
    CurlDownloadFiles(new[]
    {
        new Uri("ftp://host/file.txt"),
        new Uri("ftp://anotherhost/anotherfile.txt"),
        new Uri("http://yetanotherhost/yetanotherfile.txt")
    }
});

Downloading multiple files into specific paths:

Task("Download")
    .Does(() =>
{
    CurlDownloadFiles(
        new[]
        {
            new Uri("ftp://host/file.txt"),
            new Uri("http://anotherhost/anotherfile.txt"),
        }
        new CurlDownloadSettings
        {
            OutputPaths = new FilePath[]
            {
                "some/path/file.txt",
                "some/other/path/anotherfile.txt"
            }
        });
});

Uploading Files

Uploading a local text file to a remote HTTP server:

Task("Upload")
    .Does(() =>
{
    CurlUploadFile("some/file.txt", new Uri("http://host/path"));
});

Uploading a local text file to a remote FTPS server using credentials:

Task("Upload")
    .Does(() =>
{
    CurlUploadFile(
        "some/file.txt",
        new Uri("ftps://host/path"),
        new CurlSettings
        {
            Username = "username",
            Password = "password"
        });
});

Custom HTTP Headers

Transferring a file using a custom HTTP header in the request:

Task("Upload")
    .Does(() =>
{
    CurlUploadFile(
        "some/file.txt",
        new Uri("http://host/path"),
        new CurlSettings
        {
            Headers = new Dictionary<string, string>
            {
                ["X-SomeHeader"] = "SomeValue"
            }
        });
});

Additional Resources

You can find more information about how to use Cake.Curl in the official documentation for these projects:

You can also see Cake.Curl in action in the following videos:

cake.curl's People

Contributors

deqenq avatar ecampidoglio avatar gep13 avatar johanbenschop avatar jzeferino avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cake.curl's Issues

error when loading the curl.exe

Hi

I got a problem when loading the module, the addin is correctly installed but I this error message :
Could not locate executable

the curl.exe in isntalled into tools/cake/lib/curl.exe but I think he don't find it. I put a env var into the build.ps1 but I probably make a misteake there:

build.ps1
$CURL = Join-Path $TOOLS_DIR "curl/bin/curl.exe"

build.cake
#tool nuget:?package=NUnit.ConsoleRunner #tool nuget:?package=NUnit.Extension.NUnitV2Driver #tool nuget:?package=NUnit.Extension.NUnitV2ResultWriter #tool "nuget:?package=GitVersion.CommandLine&version=3.6.5" #tool "nuget:?package=NuGet.CommandLine&version=4.5.1" #addin "Cake.Compression&version=0.1.4" #addin "nuget:?package=Cake.Powershell&version=0.3.5" #addin "nuget:?package=Cake.Curl&version=1.0.0"

CurlUploadFile(filename, new Uri("myArtifactory"));
any Idea? :)

Thank,

Hk

Add support for specifying the HTTP method

Generally speaking, it's better to use specialized arguments to create the right kind of request rather than manually specifying the HTTP method.

This guideline is clearly stated in curl's man page:

Normally you don't need this option. All sorts of GET, HEAD, POST and PUT requests are rather invoked by using dedicated command line options.

Nonetheless, there may be situations where a remote endpoint supports a different method than the one chosen by a given curl option.

For example, when you upload a file to an HTTP endpoint using the -T parameter, curl will by default issue a PUT request. Now, that won't work if the endpoint only supports POST. In those situations, you must be able to specify the HTTP method using the -X option:

curl -X POST -T file.txt http://host/path

It would be nice if CurlSettings supported the -X --request option directly.

Recommended changes resulting from automated audit

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

  • You are currently referencing Cake.Core 0.30.0. Please upgrade to 0.33.0
  • The nuget package for your addin should use the cake-contrib icon. Specifically, your addin's .csproj should have a line like this: <PackageIconUrl>https://cdn.jsdelivr.net/gh/cake-contrib/graphics/png/cake-contrib-medium.png</PackageIconUrl>.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.

This issue was created by a tool: Cake.AddinDiscoverer version 3.12.1

cake-contrib user on NuGet

@ecampidoglio what are your thoughts on doing this?

This was discussed here:

http://cakebuild.net/blog/2016/08/cake-contribution-organization

We are currently going through a process of increasing the visibility of addins, and also trying to ensure their long term maintainability.

To that end, we are asking addin creators to add the cake-contrib user on NuGet as a co-owner (this can be done through the NuGet website by clicking on Manage Owners on the package page).

https://www.nuget.org/profiles/cake-contrib

Would you be interested in doing this for Cake.Curl? If you have any questions about this, please let me know. There was some initial concern that the Cake Team were trying to "take over" packages, and that couldn't be further from the truth, and if you have this concern, or others, I would like to address them.

Thanks!

Recommended changes resulting from automated audit

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

  • You are currently referencing Cake.Core 0.26.0. Please upgrade to 0.28.0
  • The Cake.Core reference should be private. Specifically, your addin's .csproj should have a line similar to this: <PackageReference Include="Cake.Core" Version="0.28.0" PrivateAssets="All" />
  • Your addin should target netstandard2.0. Please note that there is no need to multi-target, netstandard2.0 is sufficient.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.org and in the project on github.

Test class should use its corresponding fixture class

@ecampidoglio I noticed that test Should_Not_Set_The_Location_Option_As_Argument in file CurlDownloadMultipleFilesTests.cs uses CurlDownloadFileFixture instead of CurlDownloadMultipleFilesFixture. The same mistake is in test Should_Not_Set_The_Fail_Option_As_Argument.

The same mistake is in CurlUploadFileTests.cs (here should be used CurlUploadFileFixture).

If this is bug, I will fix it.

Moving to the cake-contrib organization

In order to guarantee the future maintenance of Cake.Curl (and to align it with the other Cake addins), I'm moving this repo to the cake-contrib organization. I'll still be the primary maintainer of the project, but I won't be the only maintainer. ๐Ÿ˜‰

While the transfer process is well documented and pretty straightforward, there are a few things that must be taken care of for it to happen:

  • Transfer the repo to @gep13
  • Create an AppVeyor project under the cake-contrib account
  • Create a Travis CI project under the cake-contrib account
  • Update the URLs of the repository in the project file
  • Update the URL of the build status badges in the README file
  • Update the URLs in this file
  • Let @gep13 know the best email address to use for adding you to a Slack Team

I'll add new items as they come up.

Recommended changes resulting from automated audit

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

  • You are currently referencing Cake.Core 0.30.0. Please upgrade to 0.33.0
  • The nuget package for your addin should use the cake-contrib icon. Specifically, your addin's .csproj should have a line like this: <PackageIconUrl>https://cdn.jsdelivr.net/gh/cake-contrib/graphics/png/cake-contrib-medium.png</PackageIconUrl>.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.

This issue was created by a tool: Cake.AddinDiscoverer version 3.12.1

Trigger deployments from the build script

A deployment is triggered when the commit being built has an annotated tag.

The problem is that, right now, it's AppVeyor that detects the presence of a tag and if so, triggers a deployment:

if ($env:AppVeyor_Repo_Tag -eq $true) {
    .\build.ps1 -Target Deploy -PackageFilePath $artifacts["NuGet"].path
}

It would be better to do this from the build script itself, instead of relying on a particular CI server to trigger the deployments.

Curl executable

I guess there is no way around of installing curl to path ?

Add support for retry options

Curl has built-in functionality to automatically retry a failed request. This behavior is controlled primarily with three options:

By default, curl will increment the delay between attempts by doubling the last waiting time, starting from 1 second until it reaches the value specified in --retry-max-time or 10 minutes. Although this behavior should cover most scenarios, it's possible to override it by setting a fixed delay right from the start with the --retry-delay option

It would be nice if Cake.Curl exposed these options through the CurlSettings class.

Would it be possible to use the Cake Contrib Icon for your NuGet Package?

Thanks again for creating this Cake Addin, we really appreciate the effort that you have put in to creating it.

We, the Cake Team, recently announced a new Cake Contrib Icon, details of which can be found here:

http://cakebuild.net/blog/2017/06/new-cake-contrib-icon

Would you consider changing the nuspec file for your NuGet Package to use this new Cake Contrib Icon? If so, the recommended URL to use is:

https://cdn.rawgit.com/cake-contrib/graphics/a5cf0f881c390650144b2243ae551d5b9f836196/png/cake-contrib-medium.png

Details of the above URL can be found in the repository here:

https://github.com/cake-contrib/graphics

Please let me know if you have any questions.

Support the --create-dirs option

It would be nice if Cake.Curl exposed the --create-dirs option when downloading a file from a remote host.

From curl's documentation:

When used in conjunction with the -o, --output option, curl will create the necessary local directory hierarchy as needed. [...] If the --output file name uses no dir or if the dirs it mentions already exist, no dir will be created.

Since this option only applies to downloading a file, it would make sense to add it to the CurlDownloadSettings type.

Inconsistent cake version

In file build.ps1 there is defined older version of cake $CAKE_VERSION = "0.23.0" while in proj files and build.sh there is version 0.30.0. It would be nice to align these versions.
As of today the latest Cake version is 0.32.1, so maybe it could be the opportunity to update and align all to the latest version?

Recommended changes resulting from automated audit

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

  • You are currently referencing Cake.Core 0.30.0. Please upgrade to 0.33.0
  • The nuget package for your addin should use the cake-contrib icon. Specifically, your addin's .csproj should have a line like this: <PackageIconUrl>https://cdn.jsdelivr.net/gh/cake-contrib/graphics/png/cake-contrib-medium.png</PackageIconUrl>.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.

This issue was created by a tool: Cake.AddinDiscoverer version 3.12.1

Generic way to add missing options?

Is there any way to specify curl options that are not listed in the CurlSettings class? I need to add --ftp-create-dirs. If not should there be a generic way to manually specify arguments?

Recommended changes resulting from automated audit

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

  • You are currently referencing Cake.Core 0.30.0. Please upgrade to 0.33.0
  • The nuget package for your addin should use the cake-contrib icon. Specifically, your addin's .csproj should have a line like this: <PackageIconUrl>https://cdn.jsdelivr.net/gh/cake-contrib/graphics/png/cake-contrib-medium.png</PackageIconUrl>.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.

This issue was created by a tool: Cake.AddinDiscoverer version 3.12.1

Add support for reporting HTTP errors

When HTTP servers fail to fulfill a request, they will often return an HTML page with the error information to the client. In situations like these, curl will simply send the content of the error page to stdout and exit with a zero exit code. While this may be fine at the command line, it can be quite problematic in scripts, since it may lead to false positives. ๐Ÿ˜ฑ

Fortunately, we can tell curl to consider HTTP errors (4xx and 5xx) for what they are and exit with a non-zero status code by passing it the -f --fail option.

Cake.Curl should support that in the CurlSettings class.

Add support for headers in the settings

I need to supply a header called PRIVATE-TOKEN with my download request from some API. The curl command would look like the following:

curl --header "PRIVATE-TOKEN: kejf9343fdf33424" "https://api.example.com/download-me.zip"

Support for headers would be greatly appreciated.

Add support to follow HTTP redirects

Sometimes an HTTP server may respond with a redirect status code (3xx) indicating that the requested resource has moved to a new URI; curl can automatically redo the request to the new URI if passed the -L --location option. It would be nice if Cake.Curl exposed that in the CurlSettings class.

Link against Cake 0.22.0 or later

The addin is still referencing Cake 0.17.0, which generates this error when using with the latest version of Cake (0.23.0, at this time):

Error: The assembly 'Cake.Curl, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
is referencing an older version of Cake.Core (0.17.0).
This assembly need to reference at least Cake.Core version 0.22.0.
Another option is to downgrade Cake to an earlier version.
It's not recommended, but you can explicitly opt-out of assembly verification
by configuring the Skip Verification setting to true
(i.e. command line parameter "--settings_skipverification=true",
environment variable "CAKE_SETTINGS_SKIPVERIFICATION=true",
read more about configuration at https://cakebuild.net/docs/fundamentals/configuration)

Cake.Curl needs to link against the newer version of Cake. Note that this means dropping support for net45 (see cake-build/cake@4fbb422).

Reference to Cake.Core 0.26.0 or later

The actual version of this addin needs to be updated because of this error:

Error: The assembly 'Cake.Curl, Version=2.1.0.0, Culture=neutral, PublicKeyToken=null'
is referencing an older version of Cake.Core (0.22.0).
This assembly must reference at least Cake.Core version 0.26.0.

Could you please update it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.