criteo / lolhttp Goto Github PK

View Code? Open in Web Editor NEW

91.0 19.0 18.0 1.73 MB

An HTTP Server and Client library for Scala.

License: Apache License 2.0

Scala 99.96% HTML 0.04%

scala http http2 https web

lolhttp's Introduction

lolhttp

An HTTP Server and Client library for Scala.

About the library

A server is built from a plain function accepting an HTTP request and eventually giving back an HTTP response. The computation is wrapped into an effect and can handle pure & impure and synchronous & asynchronous scenarios. Requests and responses are just HTTP metadata along with a lazy content body based on fs2, making it easy to handle streaming scenarios if needed. For additional convenience, the library provides content encoders and decoders for the common scala types. Clients and Servers share the same API and the same concepts, making it easy to compose them. SSL is supported on both sides.

Hello World

// Let's start an HTTP server
Server.listen(8888) {
  case GET at "/hello" =>
    Ok("Hello World!")
  case _ =>
    NotFound
}

// Let's connect with an HTTP client
Client.runSync(Get("http://localhost:8888/hello")) { res =>
  res.readAs[String].map { contentBody =>
    println(s"Received: $contentBody")
  }
}

Usage

The library is cross-built for Scala 2.11 and Scala 2.12.

The core module to use is "com.criteo.lolhttp" %% "lolhttp" % "0.13.1".

There are also 2 optional companion libraries:

"com.criteo.lolhttp" %% "loljson" % "0.13.1", provides integration with the circe JSON library.
"com.criteo.lolhttp" %% "lolhtml" % "0.13.1", provides minimal HTML templating.

Documentation

The API documentation is the main reference.

If you never used cats effects before, you should start by having a quick look at it. Service Responses must be wrapped into an IO (although there is an implicit conversion translating Response into a pure IO if needed). Also consuming the underlying content stream data is an effect. For asynchronous scenarios you have to deal with asynchronous effects, unless you prefer working with Future and wrapping it at the end using IO.fromFuture.

If you need to access the underlying content stream, you should first have a look a the fs2 documentation to understand the basics.

For those who prefer documentation by example, you can also follow these hands-on introductions:

License

This project is licensed under the Apache 2.0 license.

Copyright

lolhttp's People

Contributors

Stargazers

Watchers

Forkers

ggrossetie boniface malzzz guillaumebort 0x1997 alexeyeryshev lordshinjo jerive haveo dufrannea mchataigner scala-steward antonlin1 icodein bubblesly canut isabella232 dclaisse

lolhttp's Issues

UrlMatcher overly lenient with empty params

I found this unintuitive, though it may or may not actually be correct (haven't read the corresponding RFCs) and I apologize in advance if it is.

A UrlMatcher pattern containing a query parameter that has no value (/a?b=) matches urls that do not have the parameter present at all.

So, the following works, though it seems to me that it shouldn't.

import io.http._

val url"/?section=&update=${date}" = "/?section=contact&update=now"
// date: String = now

Client.runSync does not follow redirect url

I found that the Client.runSync does not follow redirect url. I have created a testcase to reproduce the problem.

package samples

import java.io.{BufferedReader, InputStreamReader}
import java.net.{HttpURLConnection, URL}
import java.nio.charset.Charset
import java.util.stream.Collectors

import lol.http._
import org.scalatest.FunSuite

import scala.concurrent.ExecutionContext.Implicits.global

class RedirectSuite extends FunSuite {

test("Redirect"){

val SA = Server.listen(8888){
  case GET at url"/service-a" =>
    println("Service A, redirecting to Service B")
    Redirect("http://localhost:9999/service-b")

  case _ =>
    NotFound("Service A not found")
}

val SB = Server.listen(9999){
  case GET at url"/service-b" =>
    println("Service B.. Got It!")
    Ok("Got It!")

  case r =>
    println(s"Route not found ${r.url}")
    NotFound(s"Route not found ${r.url}")
}

// Client.runSync is broken
// Client.runSync(Get("http://localhost:8888/service-a")){resp =>
// resp.readAs[String].map(body => println("Body " + body))
// }

val (status, body) = get("http://localhost:8888/service-a")
println(s"status ${status}, body ${body}")

SA.stop()
SB.stop()

}

def get(url: String, timeout: Int = 30, enc: String = "UTF-8"): (Int, String) = {
val con = createConnection("GET", url, timeout)
val responseCode = con.getResponseCode

//Reading response
val charset = Charset.forName(enc)
val in = new BufferedReader(new InputStreamReader(con.getInputStream, charset))

try {
  val content = in.lines().collect(Collectors.joining("\n"))
  (responseCode, content)
}finally{
  in.close()
}

}

private def createConnection(method: String, url: String, timeout:Int): HttpURLConnection = {
val obj = new URL(url)
val con = obj.openConnection().asInstanceOf[HttpURLConnection]
con.setRequestMethod(method)
con.setConnectTimeout(timeout * 1000)
con.setAllowUserInteraction(false)
con.setRequestProperty("Accept", "/")
con.setRequestProperty("User-Agent", "")
con
}

}

Old version of Cats Effect

The build.sbt file seems to mention v0.4 of cats-effect although in practice it seems to pull in v0.8. The latest version is 1.0.0-RC. It would be great to see this library update to the stable 1.x branch, which offers more features.

Explicitly pulling in cats-effect at the latest version seems to work, although there is no guarantee of binary compatibility so I may stumble across a feature that causes an exception at runtime.

Cross-building failure due to SSL.class

[error] missing or invalid dependency detected while loading class file 'SSL.class'.
[error] Could not access term io in package <root>,

Unexpected EOF error

Hi,
I encounter an unexpected EOF error when requesting from lol.http.Client in lolhttp version 0.11.0.

Below the scenario, the code to reproduce the issue (Forked from ReverseProxy example) and the logs .

Scenario :

curl http://localhost:8888/1MB.zip OK
Wait 30 seconds
curl http://localhost:8888/1MB.zip KO
curl http://localhost:8888/1MB.zip OK

import lol.http._

import scala.concurrent.ExecutionContext.Implicits.global
import org.slf4j.LoggerFactory

object ReverseProxy {
  val log = LoggerFactory.getLogger(this.getClass)

  def main(args: Array[String]): Unit = {

    val downloadClient = Client("ipv4.download.thinkbroadband.com", 80, "http")

    Server.listen(8888) {
      case GET at url"/" =>
        Redirect("/wiki/Criteo")

      case request =>
        downloadClient {
          request.addHeaders(Headers.Host -> h"ipv4.download.thinkbroadband.com")
        }
    }

    println("Proxying  on http://localhost:8888...")
  }
}

logs

2018-12-04 14:12:12 | 15497 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Stage | Starting up.
2018-12-04 14:12:12 | 15500 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Head | Stage ServerStatusStage sending inbound command: Connected
2018-12-04 14:12:13 | 16394 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Head | Stage ByteBufferHead sending inbound command: Connected
2018-12-04 14:12:48 | 51387 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Stage | Shutting down.
2018-12-04 14:12:48 | 51388 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Tail | Stage ServerStatusStage sending outbound command: Disconnect
2018-12-04 14:12:48 | 51390 | local | WARN | loltest | org.http4s.blaze.pipeline.Stage | ServerStatusStage received unhandled inbound command: Disconnected
2018-12-04 14:12:48 | 51390 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Head | Stage ServerStatusStage sending inbound command: Disconnected
2018-12-04 14:12:48 | 51390 | local | WARN | loltest | org.http4s.blaze.pipeline.Stage | HTTP/1.1_Stage received unhandled inbound command: Disconnected
2018-12-04 14:13:06 | 69724 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Stage | Starting up.
2018-12-04 14:13:06 | 69724 | local | DEBUG | loltest | org.http4s.blaze.pipeline.Head | Stage ServerStatusStage sending inbound command: Connected
2018-12-04 14:13:06 | 69731 | local | DEBUG | loltest | org.http4s.blaze.channel.nio2.ByteBufferHead | closeWithError(EOF)
EOF

[0.7.2] Cross-building failure due to SSL.class

A recurrent issue as in #27

Fail to decode https response

Using lolhttp 0.4.0 and trying to reach https://jsonplaceholder.typicode.com/users leads to SSL error :

mai 23, 2017 1:08:57 PM lol.http.internal.io.netty.channel.DefaultChannelPipeline onUnhandledInboundException
AVERTISSEMENT: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
lol.http.internal.io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: handshake_failure
	at lol.http.internal.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
	...
Caused by: javax.net.ssl.SSLException: Received fatal alert: handshake_failure
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
	...

The future returned an exception of type: lol.http.Error, with message: Connection closed.
ScalaTestFailureLocation: org.sample.HelloLol at (HelloLol.scala:26)
org.scalatest.exceptions.TestFailedException: The future returned an exception of type: lol.http.Error, with message: Connection closed.
	at org.scalatest.concurrent.Futures$FutureConcept.tryTryAgain$1(Futures.scala:531)
	...
Caused by: lol.http.Error: Connection closed
	at lol.http.Error$.<init>(Errors.scala:30)
	...

Here is the code :

class HelloLol extends FunSpec with Matchers with ScalaFutures with BeforeAndAfterAll {
  implicit val defaultPatience = PatienceConfig(timeout = Span(5, Seconds), interval = Span(500, Millis))
  val typicodeClient = Client("jsonplaceholder.typicode.com", 443, "https", ssl = SSL.trustAll)

  override def afterAll(): Unit = {
    typicodeClient.stop()
  }

  describe("LOL") {
    it("should perform requests") {
      val res = typicodeClient.run(Get("/users")) { response =>
        response.readAs[Json].map(root.each.username.string.getAll)
      }
      res.futureValue shouldBe List()
    }
  }
}

Upgrade fs2 to 0.10?

I was wondering if there are plans to upgrade fs2 from 0.9.5 to 0.10.0? If so, then I have a fork that's refactored to use cats.effect.IO instead of fs2.Task, etc. I can submit a pull request if desired.

[info] HtmlTests:
[info] - HTML encoding
[info] - HTML auto escaping
[info] - HTML safe include
[info] - HTML option rendering
[info] - HTML seq rendering
[info] Run completed in 226 milliseconds.
[info] Total number of tests run: 5
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 5, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] UrlsTests:
[info] - Path parts
[info] - Path tail
[info] - QueryString
[info] ResponseTests:
[info] - Future.map on a Response
[info] - Future.flatMap on a Response
[info] JsonTests:
[info] - JSON encoding
[info] - JSON decoding
[info] - JSON over HTTP
[info] - JSON over Server Sent Events
[info] Run completed in 19 seconds, 833 milliseconds.
[info] Total number of tests run: 4
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] ContentTests:
[info] - Text encoding
[info] - Text decoding
[info] - UrlEncoded
[info] - InputStream
[info] - File
[info] - Classpath resources
[info] ServerSentEventsTests:
[info] - Valid string events stream
[info] - Not an events stream
[info] - Invalid events stream 
[info] ConnectionUpgradeTests:
[info] - Upgrade connection
[info] - Server push directly
[info] - Read content twice
[info] - Upgrade twice
[info] - Server refuse to upgrade
[info] SSLTests:
[info] - SSL over self signed certificate
[info] - insecure connection rejected
[info] ServerTests:
[info] - Hello World
[info] - Headers
[info] - Methods
[info] - Redirects
[info] - Upload
[info] - No Content-Length
[info] ClientTests:
[info] - Client
[info] - Large content
[info] - Connection close
[info] - Connection leak
[info] - Single connection
[info] - Timeouts
[info] - Connection errors
[info] Run completed in 15 seconds, 491 milliseconds.
[info] Total number of tests run: 34
[info] Suites: completed 8, aborted 0
[info] Tests: succeeded 34, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.