xinsnake / go-http-digest-auth-client Goto Github PK

View Code? Open in Web Editor NEW

41.0 2.0 66.0 30 KB

Golang Http Digest Authentication Client

License: BSD 2-Clause "Simplified" License

Go 100.00%

digest-authentication golang golang-library

go-http-digest-auth-client's Introduction

go-http-digest-auth-client

Golang Http Digest Authentication Client

This client implements RFC7616 HTTP Digest Access Authentication and by now the basic features should work.

Usage

// import
import dac "github.com/xinsnake/go-http-digest-auth-client"

// create a new digest authentication request
dr := dac.NewRequest(username, password, method, uri, payload)
response1, err := dr.Execute()

// check error, get response

// reuse the existing digest authentication request so no extra request is needed
dr.UpdateRequest(username, password, method, uri, payload)
response2, err := dr.Execute()

// check error, get response

Or you can use it with http.Request

t := dac.NewTransport(username, password)
req, err := http.NewRequest(method, uri, payload)

if err != nil {
    log.Fatalln(err)
}

resp, err := t.RoundTrip(req)
if err != nil {
    log.Fatalln(err)
}
defer resp.Body.Close()

fmt.Println(resp)

go-http-digest-auth-client's People

Contributors

Stargazers

Watchers

Forkers

rjp jls5177 tatsumi33 datsuns rturowicz vtanh paul-hoehne 123haynes akshaykarle dis2 iselind happyshittake dbenders nmulcahey stefa975 nickysemenza ionbazan td7650 gumpyoung shuralle agis reinventer sp0n-7 lukascoppens bmess nitper justinfx orphaner judgegregg ahaooahaz ybriffa toyray jakecoffman dhananjayanxage snawoot iftikharqureshi wizardishungry palinda skytodmoon bon-ami mzack9999 kindevil as564335sa geliyahappy leonidkrn xiaomayi-van yesoreyeram wvvvwv advoenkin ajelic-cookieai commanderredyt markwinds

go-http-digest-auth-client's Issues

[Feature] ability to set custom timeout

Hello,

I'm using the library but the hardcoded timeout of 30" is too small for my use-case (I'm downloading very large files).

What do you think about making the value configurable?It could be added to DigestRequest and DigestTransport so that it's picked up by the paths that execute the requests eventually.

If that sounds good, I could prepare a patch. Thanks!

P.S. This is probably related to #5.

Release tags are in the incorrect format

The last 2 releases were tagged without the v in front: 0.5.0 and 0.6.0. As a result, these 2 releases are not discoverable via go modules. See https://pkg.go.dev/github.com/xinsnake/go-http-digest-auth-client?tab=versions for the list of discoverable releases.

The 0.5.0 and 0.6.0 releases should be retagged to v0.5.0 and v0.6.0 respectively.

net/http: request canceled (Client.Timeout exceeded while awaiting headers)

I'm trying to call an external api in loop using go concurrency at one time nearly 16k but I'm getting error

net/http: request canceled (Client.Timeout exceeded while awaiting headers)

here is my impletenrtin

`
var wg sync.WaitGroup
wg.Add(16367)

// fmt.Println("Running for loop…")

for results.Next() {
	var tag StationData
	// for each row, scan the result into our tag composite object
	err = results.Scan(&tag.ID, &tag.Station)
	if err != nil {
		panic(err.Error()) // proper error handling instead of panic in your app
	}
	id := tag.ID
	stationname := tag.Station
	go func(id int) {
		defer wg.Done()
		time.Sleep(time.Millisecond * 500)
		t := dac.NewTransport("username", "password")
		req, err := http.NewRequest("GET", "urlsomttthh", nil)

		if err != nil {
			log.Fatalln(err)
		}

		response, err := t.RoundTrip(req)
		if err != nil {
			fmt.Printf("somthing missing from data %s\n", err)
		} else {
			data, _ := ioutil.ReadAll(response.Body)
			fmt.Println(string(data))
		}
	}(id)

	// datasprin := "somthing" + tag.Station
	// and then print out the tag's Name attribute
	// log.Printf(datasprin)

}

wg.Wait()
// fmt.Println("Finished for loop")
fmt.Printf("total time %s", time.Now().Sub(start))

TCP Connections not reused

I think the TCP connections are not reused with the current code:

package main

import (
	dac "github.com/xinsnake/go-http-digest-auth-client"
	"io"
	"io/ioutil"
)

const ENDPOINT = "https://foobar.com"
const USERNAME = "Foo"
const PASSWORD = "Bar"
const DBPATH = "Baz"

func main() {
	// create a new digest authentication request
	dr := dac.NewRequest(USERNAME, PASSWORD, "GET", ENDPOINT+"v1/documents/?uri="+DBPATH, "")
	resp, _ := dr.Execute()
	io.Copy(ioutil.Discard, resp.Body)
	resp.Body.Close()
	dr.UpdateRequest(USERNAME, PASSWORD, "GET", ENDPOINT+"v1/documents/?uri="+DBPATH, "")
	resp, _ = dr.Execute()
	io.Copy(ioutil.Discard, resp.Body)
	resp.Body.Close()
	dr = dac.NewRequest(USERNAME, PASSWORD, "GET", ENDPOINT+"v1/documents/?uri="+DBPATH, "")
	resp, _ = dr.Execute()
	io.Copy(ioutil.Discard, resp.Body)
	resp.Body.Close()
}

I patched the client according to: https://blog.golang.com/http-tracing

diff --git a/digest_auth_client.go b/digest_auth_client.go
index a2e69c4..9f0f7ba 100644
--- a/digest_auth_client.go
+++ b/digest_auth_client.go
@@ -4,7 +4,9 @@ import (
 	"bytes"
 	"crypto/tls"
 	"fmt"
+	"log"
 	"net/http"
+	"net/http/httptrace"
 	"time"
 )
 
@@ -106,6 +108,12 @@ func (dr *DigestRequest) Execute() (resp *http.Response, err error) {
 		return nil, err
 	}
 	req.Header = dr.Header
+	trace := &httptrace.ClientTrace{
+		GotConn: func(connInfo httptrace.GotConnInfo) {
+			log.Printf("Got Conn: %+v\n", connInfo)
+		},
+	}
+	req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
 
 	client := dr.getHTTPClient()
 
@@ -169,6 +177,13 @@ func (dr *DigestRequest) executeRequest(authString string) (resp *http.Response,
 	req.Header = dr.Header
 	req.Header.Add("Authorization", authString)
 
+	trace := &httptrace.ClientTrace{
+		GotConn: func(connInfo httptrace.GotConnInfo) {
+			log.Printf("Got Conn: %+v\n", connInfo)
+		},
+	}
+	req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
+
 	client := dr.getHTTPClient()
 	return client.Do(req)
 }

And here's what I got:

Go/src/test via 🐹 v1.14.4 took 6s
[N] ➜ go run main.go
2020/07/03 15:08:58 Got Conn: {Conn:0xc000080380 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:08:58 Got Conn: {Conn:0xc000600e00 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:09:00 Got Conn: {Conn:0xc000601180 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:09:02 Got Conn: {Conn:0xc000601880 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:09:02 Got Conn: {Conn:0xc000580380 Reused:false WasIdle:false IdleTime:0s}

I believe the problem is twofold:

First we create a transport for each request, which will cancel the benefit of caching requests:

https://golang.org/src/net/http/transport.go?h=http.Transport

// By default, Transport caches connections for future re-use.
// This may leave many open connections when accessing many hosts.
// This behavior can be managed using Transport's CloseIdleConnections method
// and the MaxIdleConnsPerHost and DisableKeepAlives fields.
//
// Transports should be reused instead of created as needed.
// Transports are safe for concurrent use by multiple goroutines.

By applying the following patch:

diff --git a/digest_auth_client.go b/digest_auth_client.go
index a2e69c4..13ff3c3 100644
--- a/digest_auth_client.go
+++ b/digest_auth_client.go
@@ -54,9 +56,9 @@ func (dr *DigestRequest) getHTTPClient() *http.Client {
 
 	return &http.Client{
 		Timeout: 30 * time.Second,
-		Transport: &http.Transport{
-			TLSClientConfig: &tlsConfig,
-		},
+		//Transport: &http.Transport{
+		//TLSClientConfig: &tlsConfig,
+		//},
 	}
 }

we get the following:

Go/src/test via 🐹 v1.14.4 took 7s 
[N] ➜ go run main.go                                       
2020/07/03 15:13:06 Got Conn: {Conn:0xc000105880 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:13:06 Got Conn: {Conn:0xc000104700 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:13:06 Got Conn: {Conn:0xc000104700 Reused:true WasIdle:true IdleTime:221.992µs}
2020/07/03 15:13:06 Got Conn: {Conn:0xc000104700 Reused:true WasIdle:true IdleTime:115.194µs}
2020/07/03 15:13:06 Got Conn: {Conn:0xc000053500 Reused:false WasIdle:false IdleTime:0s}

Second, we don't discard the body on the first request (authentication), which keeps the connection in a "waiting state":
If we discard it:

diff --git a/digest_auth_client.go b/digest_auth_client.go
index a2e69c4..d97ad9a 100644
--- a/digest_auth_client.go
+++ b/digest_auth_client.go
@@ -4,7 +4,11 @@ import (
 	"bytes"
 	"crypto/tls"
 	"fmt"
+	"io"
+	"io/ioutil"
 	"net/http"
 	"time"
 )
 
@@ -129,6 +139,7 @@ func (dr *DigestRequest) executeNewDigest(resp *http.Response) (resp2 *http.Resp
 	)
 
 	// body not required for authentication, closing
+	io.Copy(ioutil.Discard, resp.Body)
 	resp.Body.Close()
 
 	if waString = resp.Header.Get("WWW-Authenticate"); waString == "" {

Then we get:

Go/src/test via 🐹 v1.14.4
[N] ➜ go run main.go
2020/07/03 15:17:59 Got Conn: {Conn:0xc000080380 Reused:false WasIdle:false IdleTime:0s}
2020/07/03 15:17:59 Got Conn: {Conn:0xc000080380 Reused:true WasIdle:true IdleTime:91.744µs}
2020/07/03 15:18:01 Got Conn: {Conn:0xc000080380 Reused:true WasIdle:true IdleTime:117.822µs}
2020/07/03 15:18:01 Got Conn: {Conn:0xc000080380 Reused:true WasIdle:true IdleTime:79.115µs}
2020/07/03 15:18:01 Got Conn: {Conn:0xc000080380 Reused:true WasIdle:true IdleTime:193.787µs}

The problem with hanging connections is that under a heavy load, the client can run out of TCP connections, since they're all in a TIME-WAIT state.

I think problem 2 is easy to solve (simply discard the resp.Body)
I have no idea how to solve problem 1 while still keeping a configurable client, though.

Sorry for that long text, and thank you very much for your work on this useful library :)

Provide a http.RoundTripper implementation

It would be nice if your package can provide a http.RoundTripper implementation similar to https://godoc.org/github.com/bobziuchkovski/digest

This would allow your package to be used with a http.Client which can then be used with the ctxhttp package.

Add Headers in func NewRequest

In func NewRequest, could create parameter with headers.

I had a problem with an api, I had to send Content-Type. I made the following change:

func (dr *DigestRequest) UpdateRequest(username, password, method, uri, body string) *DigestRequest {
	dr.Body = body
	dr.Method = method
	dr.Password = password
	dr.URI = uri
	dr.Username = username
	dr.Header = make(map[string][]string)
	
        //HERE
       dr.Header["Content-Type"] = []string{"application/json"}

	return dr
}

parameter for timeout

Hi @xinsnake,
thanks for this neat library! I'm using it to grab some pictures from an IP Camera which requires the digest based authentication.

The only wish I have is to be able to set a shorter timeout. Currently the timeout is hard coded to 30 seconds.

Maybe an implementation using the context package would make sense. Context.WithTimeout could be an adequate solution.

regex failed

test string: "Digest realm="e29072b1b78a1e4d7bfdec11", domain="::", qop="auth", nonce="nonce", opaque="", algorithm="MD5", stale="FALSE"", regex error when param has empty string

cann't set header for digest auth

need add code in digest_auth_client.go line: 94:

// 修复header问题
if req.Header != nil {
dr.Header = req.Header
}

response body not close

response body not close , whitch created in func (dr *DigestRequest) Execute() (resp *http.Response, err error)

user only can get and close the response body created in func (dr *DigestRequest) executeRequest(authString string) (*http.Response, error), but the response body created in func (dr *DigestRequest) Execute() (resp *http.Response, err error) is not close anywhere.

i think can add resp.Body.Close() after line 102 in file digest_auth_client.go, is it right

401 Unauthorized

Hello,

I am trying to authenticate on a server that uses digest authentication. After installing your plugin, i still receive 401.

uri := "http://localhost:18082/json_rpc"
fmt.Println("-----")
dr := dac.NewRequest("mihai", "mihai", "POST", uri, "")
response1, err := dr.Execute()
fmt.Println(response1)
fmt.Println("-----")

This is the server response:

HTTP/1.1 401 Unauthorized
Server: Epee-based
Content-Length: 98
Content-Type: text/html
Last-Modified: Tue, 02 Jan 2018 14:20:12 GMT
Accept-Ranges: bytes
WWW-authenticate:Digest qop="auth",algorithm=MD5,realm="monero-rpc",nonce="PfLdSGoqjOOSMw5bmJOv/g==",stale=false
WWW-authenticate:Digest qop="auth",algorithm=MD5-sess,realm="monero-rpc",nonce="PfLdSGoqjOOSMw5bmJOv/g==",stale=false

PS: I have also tried with http.Request

What am i doing wrong?
Thanks

Leaking goroutines and filedescriptors

Using pprof i noticed alot of hanging goroutines and in prometheus i could see that filedescriptors was accumulating. Traced the leak down to be from this package, switched to github.com/rkl-/digest instead and the leak is now gone. Thought i would let you know though.

Time for a new release?

There's been around 10 new commits since the last release. Do you think it's possible to tag a new release?

HTTP Headers not recognized.

I'm trying to do an HTTP POST using this library with the second approach but http headers are not recognized.

transport := dig.NewTransport(username, password)
req, err := http.NewRequest("POST", fullUri, someJsonData)
req.Header.Set("Content-Type", "application/json")
resp, _ := transport.RoundTrip(req)

Tried to do a POST on to https://requestb.in and Content-Type header is indeed not received.
Tried to use another library but this exact same code worked.

I haven't looked into the pull requests but this PR #6 might have already solved this problem.