apple / swift-http-types Goto Github PK

let baseURL = URL(string: "https://foo.com")!
let url = URL(string: "/bar", relativeTo: baseURL)!
let httpRequest = HTTPRequest(url: url)

will hit precondition in HTTPRequest+URL.swift:103:

precondition(schemeRange.location != kCFNotFound, "Schemeless URL is not supported")

That seems not supported yet?

tvOS 13.0.0
watchOS 6.0.0
macOS 10.15.0

There are many use cases where you may want to store some metadata on the request or even response (for example storing a request ID or a trace ID for distributed systems). Currently there's no way to do this, without adding an additional header which may not be desirable, or a pseudo header which feels like it's working around the problem rather than solving it.

Would a dictionary property or similar be something the team would consider?

Since it's handled as CoW through isKnownUniquelyReferenced, I would not expect to need a lock.

Hello,

This issue is a feature request for a new API in the HTTPTypesFoundation module:

extension HTTPRequest {
    public func url(baseURL: URL) -> URL?
}

I met a need for this API while using http://github.com/apple/swift-openapi-generator

When writing a ClientMiddleware that processes http requests performed through URLSession, I need an absolute Foundation.URL in order to use various services such as HTTPCookieStorage.

This package defines a HTTPRequest.url property that looks very promising.

But the HTTPRequest I get from OpenAPIRuntime is not absolute. It is relative to a base URL, and its url property is nil:

struct MyMiddleware: ClientMiddleware {    
    func intercept(
        _ request: HTTPRequest,
        body requestBody: HTTPBody?,
        baseURL: URL,
        operationID: String,
        next: (HTTPRequest, HTTPBody?, URL) async throws -> (HTTPResponse, HTTPBody?)
    ) async throws -> (HTTPResponse, HTTPBody?)
    {
        print(request.url) // nil
    }
}

Since my requests are performed through URLSession, I was pretty sure to find some code that turns a (request, baseURL) pair into an absolute URL, suitable for URLRequest. Indeed, it is there.

This code is not public, so I defined my own version, inspired from the above implementation:

extension HTTPRequest {
    /// Returns an absolute URL.
    ///
    /// Inspired from <https://github.com/apple/swift-openapi-urlsession/blob/0.3.0/Sources/OpenAPIURLSession/URLSessionTransport.swift#L185-L208>
    func url(baseURL: URL) -> URL? {
        guard
            var baseUrlComponents = URLComponents(string: baseURL.absoluteString),
            let requestUrlComponents = URLComponents(string: path ?? "")
        else {
            return nil
        }

        let path = requestUrlComponents.percentEncodedPath
        baseUrlComponents.percentEncodedPath += path
        baseUrlComponents.percentEncodedQuery = requestUrlComponents.percentEncodedQuery
        return baseUrlComponents.url
    }
}

Since the implementation is not quite trivial, I believe it is a good candidate for inclusion in HTTPTypesFoundation.

What do you think?

Steps To Reproduce:
Perform any action and capture the HTTP Request.(iOS mobile apps with swift code)
Observe that in response X-XSS-Protection header was used.
HTTP/2
200
OK
Server :jag
Etag: 3655723661770096640-4617501349922598127
Content-Type:text/html;charset=utf-8
Content-Length:1
X-Xss-Protection: 1; mode=block

Recommendation:
Disable use of deprecated headers such as X-XSS-Protection.

References
X-XSS-Protection - HTTP | MDN

Impact
Even though this feature can protect users of older web browsers that don't yet support Content-Security-Policy, in some cases, XSS protection can create XSS vulnerabilities in otherwise safe websites.

In current implementation (0.2.1), after you set a combined field value via the plain subscript method, you will get the combined value from raw subscript method too:

var fields = HTTPFields()
fields[.acceptLanguage] = "en-US, zh-Hans-CN"
print(fields[.acceptLanguage]!) // en-US, zh-Hans-CN
print(fields[raw: .acceptLanguage]) // ["en-US, zh-Hans-CN"]

But I expect the raw subscript method will return ["en-US", "zh-Hans-CN"] in the above example. Is current behavior expected? Or it is a bug?

Hello, it looks like this package is achieving mainly http2-like structures. In my case I use query string and I am having hard time to use this package along.

Am I using it wrong? How can I easily add a param to the query part of a URL of an HTTPRequest?

Thanks in advance for answer.

These types should all support Codable

This is the list of standards where the names came from, and ones that are currently missing:

• RFC 2068: Hypertext Transfer Protocol -- HTTP/1.1
• RFC 6265: HTTP State Management Mechanism
• RFC 6266: Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP)
• RFC 6454: The Web Origin Concept
• RFC 6455: The WebSocket Protocol
• RFC 6797: HTTP Strict Transport Security (HSTS)
• RFC 7838: HTTP Alternative Services
  Alt-Used
• RFC 9110: HTTP Semantics
  Allow
  Content-Language
  Content-Location
  Content-Range
  From
  If-Match
  If-Unmodified-Since
  Max-Forwards
  Retry-After
  TE
• RFC 9111: HTTP Caching
• RFC 9218: Extensible Prioritization Scheme for HTTP
• Content Security Policy Level 3
  Content-Security-Policy-Report-Only
• Fetch
  Cross-Origin-Resource-Policy
  Sec-Purpose
• Upgrade Insecure Requests

Let's evaluate which of these are useful to include (and which ones we can drop).

Vapor supports some nice hard-typed APIs that let users interact with some header fields in a convenient manner.
For example, consider the following:

import Vapor /// `Vapor` exports `NIOHTTP1`

var headers = NIOHTTP1.HTTPHeaders()
headers.contentType = HTTPMediaType.json 

This feature requests asks if such a feature can be supported by swift-http-types itself, instead of us having to implement it in each of HTTP-server packages such as Hummingbird (v2 is WIP and uses swift-http-types) or Vapor (in v5, the work has not yet begun but it's clear we'll be using swift-http-types).

My understanding is that a package like this would want to be very conservative about adding a feature like this, but I still think such convenient APIs are very valuable.

As i see it, one of the followings should be the case about this feature request, so I'm interested to know which one:

• In scope, can be supported.
• Not a main feature, maybe it can be in an e.g. HTTPTypesExtras target, or package.
• Completely out of scope, will not be supported.

This is how vapor implements such a feature for the content-type header.
Note that content-type is just an example. There could be many more header field names supported with an API like this.

I noticed that this library uses CoreFoundation directly. IIRC that is not portable; CF is available on Darwin platforms and Linux, but is deliberately omitted from other platforms (e.g. Windows).

CoreFoundation is not meant to be public, and the Windows port has generally been far more aggressive about preventing leakage of internal interfaces and encouraging cross-compilation.

See discussion on the Swift forums announcement thread. Basically, Foundation doesn't (yet) provide the APIs this library needs. There may be ways around it, but that discussion is probably better to have here than on the forums.

HTTPRequest always adds a / after the host, sometimes leading to invalid URLs.

let request = HTTPRequest(url:  URL(string: "https://example.com?page=some")!)
XCTAssertEqual(request.url!.absoluteString, "https://example.com?page=some")
// XCTAssertEqual failed: ("https://example.com/?page=some") is not equal to ("https://example.com?page=some")

I'm not sure if "https://example.com/?page=some" is an invalid URL, but I have experienced some servers returning errors for requests with "/?".
There is also another unexpected behavior:

let request = HTTPRequest(url:  URL(string: "https://example.com")!)
XCTAssertEqual(request.url!.absoluteString, "https://example.com")
// XCTAssertEqual failed: ("https://example.com/") is not equal to ("https://example.com")

I would not expect my URL to be altered in any way by HTTPRequest.

I don't know how to set a query string like URLQueryItem.
e.g. ?zipcode=2790031 in http://zipcloud.ibsnet.co.jp/api/search?zipcode=2790031

Based on swift-http-types's Package.swift this package support Swift 5.7.0+. However if I try to build it using Swift 5.7.0 it fails. Swift 5.7.1 works however.

See Swift 5.7.0 logs:

➜  swift-web-push git:(ff-fix-concurrency-problem) ✗ docker run -v $(pwd):/src -w /src -it swift:5.7.0-jammy /bin/bash
Unable to find image 'swift:5.7.0-jammy' locally
5.7.0-jammy: Pulling from library/swift
56a2caa6b2c6: Pull complete 
5f5781394e74: Pull complete 
4c758f5def4e: Pull complete 
cec232b6c8e8: Pull complete 
Digest: sha256:473744e71c112bb20fe897a3e0d985a6cb8aed98b6a9ae1461df5cf36214cb11
Status: Downloaded newer image for swift:5.7.0-jammy
root@8486157fd10c:/src# swift build
[1/1] Planning buildCompiling plugin MakeVapidKeyPlugin...
Compiling plugin Swift-DocC...
Compiling plugin Swift-DocC Preview...
Building for debugging...
/src/.build/checkouts/swift-http-types/Sources/HTTPTypes/HTTPFields.swift:165:68: error: cannot convert value of type 'String' to expected argument type 'String.Element' (aka 'Character')
                    self[fields: name] = newValue.split(separator: "; ", omittingEmptySubsequences: false).map {
                                                                   ^
/src/.build/checkouts/swift-http-types/Sources/HTTPTypes/HTTPFields.swift:165:68: error: cannot convert value of type 'String' to expected argument type 'String.Element' (aka 'Character')
                    self[fields: name] = newValue.split(separator: "; ", omittingEmptySubsequences: false).map {
                                                                   ^
[33/310] Compiling x_x509.c

HTTPFields currently have 3 subscripts: subscript(_: HTTPField.Name) -> String?, subscript(raw: HTTPField.Name) -> [String], and subscript(fields: HTTPField.Name) -> [HTTPField] .

• The first one subscript(_: HTTPField.Name) -> String? is equivalent to Foundation URLRequest/HTTPURLResponse value(forHTTPHeaderField:).
• The second one subscript(raw: HTTPField.Name) -> [String] is equivalent to NIO HTTPHeaders.subscript(_:).
• The third one has no equivalence in previous APIs since HTTPField is a new abstraction.
• There is no equivalence to NIO HTTPHeaders.subscript(canonicalForm:).

During past discussions, it's been brought up that raw is a weird name, and "fields" is more "raw" than raw. Let's figure out if there is a better set of names for these subscripts.

Currently order matters when comparing if two HTTPFields instances are equal.
I'm by no means an expert in HTTP standards but I found this surprising as the interface is very similar to that of a Dictionary.

My use case is I want to verify a HTTPRequest is as expected in tests but currently the test fails due to the headers having different ordering.

Should order matter here?

    static func testHeadersSameOrder() -> Bool {
        var headers1: HTTPFields = [:]
        headers1[.contentType] =  "application/json"
        headers1[.acceptLanguage] = "fr-CA"
        
        var headers2: HTTPFields = [:]
        headers2[.contentType] =  "application/json"
        headers2[.acceptLanguage] = "fr-CA"
        
        // returns true
        return headers1 == headers2
    }
    
    static func testHeadersDifferentOrder() -> Bool {
        var headers1: HTTPFields = [:]
        headers1[.acceptLanguage] = "fr-CA"
        headers1[.contentType] =  "application/json"
        
        var headers2: HTTPFields = [:]
        headers2[.contentType] =  "application/json"
        headers2[.acceptLanguage] = "fr-CA"
        
        // returns false
        return headers1 == headers2
    }

I'm working a PR for Swift AWS Lambda Events to adopt Swift HTTP Types.

Would it be acceptable to add Codable conformance directly to HTTPResponse.Status and HTTPRequest.Method to better support this / would you welcome a PR for this?

HTTPRequest doesn't have init(method:url).

https://github.com/apple/swift-http-types#create-a-request-from-a-foundation-url

var request = HTTPRequest(method: .get, url: URL(string: "https://www.example.com/")!)
request.method = .post
request.path = "/upload"

The URLSession convenience methods are currently only available on Apple platforms. Why is that?