jto / validation Goto Github PK

validation api extracted from play

Scala 99.60% HTML 0.23% Shell 0.18%

scala json validation xml csv json4s play-framework

validation's Introduction

The unified data validation library

Overview

The unified validation API aims to provide a comprehensive toolkit to validate data from any format against user defined rules, and transform them to other types.

Basically, assuming you have this:

import play.api.libs.json._
import jto.validation._

case class Person(name: String, age: Int, lovesChocolate: Boolean)

val json = Json.parse("""{
  "name": "Julien",
  "age": 28,
  "lovesChocolate": true
}""")

implicit val personRule = {
  import jto.validation.playjson.Rules._
  Rule.gen[JsValue, Person]
}

It can do this:

scala> personRule.validate(json)
res0: jto.validation.VA[Person] = Valid(Person(Julien,28,true))

BUT IT'S NOT LIMITED TO JSON

It's also a unification of play's Form Validation API, and its Json validation API.

Being based on the same concepts as play's Json validation API, it should feel very similar to any developer already working with it. The unified validation API is, rather than a totally new design, a simple generalization of those concepts.

Design

The unified validation API is designed around a core defined in package jto.validation, and "extensions". Each extension provides primitives to validate and serialize data from / to a particular format (Json, form encoded request body, etc.). See the extensions documentation for more information.

To learn more about data validation, please consult Validation and transformation with Rule, for data serialization read Serialization with Write. If you just want to figure all this out by yourself, please see the Cookbook.

Using the validation api in your project

Add the following dependencies your build.sbt as needed:

resolvers += Resolver.sonatypeRepo("releases")

val validationVersion = "2.1.0"

libraryDependencies ++= Seq(
  "io.github.jto" %% "validation-core"      % validationVersion,
  "io.github.jto" %% "validation-playjson"  % validationVersion,
  "io.github.jto" %% "validation-jsonast"   % validationVersion,
  "io.github.jto" %% "validation-form"      % validationVersion,
  "io.github.jto" %% "validation-delimited" % validationVersion,
  "io.github.jto" %% "validation-xml"       % validationVersion
  // "io.github.jto" %%% "validation-jsjson"    % validationVersion
)

Play dependencies

Validation	Play
2.1.x	2.6.x
2.0.x	2.5.x
1.1.x	2.4.x
1.0.2	2.3.x

Documentation

Documentation is here

Contributors

Julien Tournay - http://jto.github.io
Olivier Blanvillain - https://github.com/OlivierBlanvillain
Nick - https://github.com/stanch
Ian Hummel - https://github.com/themodernlife
Arthur Gautier - https://github.com/baloo
Jacques B - https://github.com/Timshel
Alexandre Tamborrino - https://github.com/atamborrino

validation's People

Contributors

Stargazers

Watchers

validation's Issues

Laziness when using Format.gen

Right now, if you want Format.gen to works with JSON (for example), you need the following:

implicit val entityFormat = {
  import play.api.libs.json._
  import play.api.data.mapping.json.Rules._
  import play.api.data.mapping.json.Writes._
  Format.gen[JsValue, JsObject, Entity]
}

I was wondering if we could merge Rules and Writes implicits into a single Formats object? It's not much about saving one line of code but more about "I just want to use Format macros and I shouldn't bother with exactly what to import in order to make it works, the lib should handle that for me if possible".

implicit val entityFormat = {
  import play.api.libs.json._
  import play.api.data.mapping.json.Formats._
  Format.gen[JsValue, JsObject, Entity]
}

Variance

Could you make RuleLike[I, O] and WriteLike[I, O] contravariant in I, and covariant in O?

Implicit syntax is automatically added

Please separate the implicit conversions defined at the package level or in companion objects so that they are not added to the implicit scope automatically. They can cause ambiguous implicit resolutions if the developer already imports, for instance cats.syntax.cartesian._.

add Write/Rule.apply[I, O](implicit a: Write/Rule[I, O]) = a

class ValidationError included in the jar

I’ve got this unpleasant message from ProGuard:

Duplicate zip entry [play-datacommons_2.10-2.2.2.jar:play/api/data/validation/ValidationError.class]

Upon closer inspection, validation (this project) also includes that class in the jar!
I’m going to investigate this, but perhaps it’s due to the type alias in play.api.data.mapping?

Why did you guys use Success/Failure rather than specific names like ValidationSuccess?

You have probably ditched JsSuccess as it is in Play because this library is not json-specific anymore, but Success and Failure conflicts with scala.util.Try stuff.

I find myself always needing to alias them to ValidationSuccess and ValidationFailure so they don't conflict with scala default Try. Am I missing something?

Shapeless rules

Perhaps would be nice to add this to the shapeless branch: https://github.com/mandubian/shapeless-rules. Or this, which has a slightly more general operation: https://github.com/scala-needs/play-functional-extras.

Make it available as an ivy dependency

So people can use it and enjoy it right now!

Single member case class writes

When my case class has only one property, how do I define the writer?
I am probably missing something obvious, but it seems like an issue because it makes using the API consistently more difficult.

case class A(x: Int)
implicit val aW = To[JsObject] { __ =>
  import play.api.data.mapping.json.Writes._
  ((__ \ "x").write[Int])(unlift(A.unapply _))
}

error: type mismatch;
 found   : Option[Int]
 required: Option[play.api.data.mapping.WriteLike[Int,play.api.libs.json.JsObject]]

`Rule.compose` really should be named `Rule.andThen`

See how it is named in scalaz or cats.

That’s really confusing because compose usually means combination in the reverse side… (see in scalaz or cats).

ValidationErrors class?

I’m using something like this to report multiple validation errors:

case class ValidationErrors(paths: Seq[(Path, Seq[ValidationError])])
  extends Exception(paths.toString())

def throwValidationErrors[O](va: VA[O]) = va recoverTotal {
  case play.api.data.mapping.Failure(paths) ⇒ throw ValidationErrors(paths)
}

Maybe you could include a similar API in the project?

Is it possible to get a build for Scala 2.10.4?

It looks like Sonatype only has 2.11 builds available, would it be possible for 2.10.4 builds to be published?

O: Monoid → Monoid[Write[I, O]]

This is useful if we want to combine several Writes with |+|. Implementation:

implicit def `Write is a Monoid`[I, O: Monoid] = new Monoid[Write[I, O]] {
  def append(a1: Write[I, O], a2: Write[I, O]) = Write[I, O] { x ⇒
    a1.writes(x) |+| a2.writes(x)
  }
  def identity = Write[I, O] { _ ⇒
    implicitly[Monoid[O]].identity
  }
}

Unfortunately invariance in I makes it a bit awkward to add Write[Animal, JsObject] and Write[Squirrel, JsObject]. A method to map on I would help. Otherwise here’s a quick hack:

implicit class RichWrite[I, O](write: Write[I, O]) {
  def subtype[I1 <: I] = Write[I1, O](write.writes)
}

animalWrite.subtype[Squirrel] |+| squirrelWrite

Update to Play 2.5

When are you planning to create a release for play 2.5?

Thanks

Format type order

I was just wondering... currently, the Format type order is the following:

trait Format[IR, IW, O] extends RuleLike[IR, O] with WriteLike[O, IW]

I don't quite find it fully readable. If we read Rule and Write, it looks like "IR, O, O, IW", which is like "IR, O, IW". It means, "I can go from IR to O and from O to IW". Nice. So why not having:

trait Format[IR, O, IW] extends RuleLike[IR, O] with WriteLike[O, IW]

// Wuuut?
implicit val userFormat = Format.gen[JsValue, BSONDocument, User]

// Ohhhh, I see!
implicit val userFormat = Format.get[JsValue, User, BSONDocument]

Replacing Rule/Write with Format seems to fail because of variance

I tried to do the following refactoring: moving from Rule.gen and Write.gen to Format.gen

import play.api.data.mapping._

case class Organization (
  id: String,
  name: String
)

object Organization {
  implicit val organizationRule = {
    import play.api.libs.json._
    import play.api.data.mapping.json.Rules._
    Rule.gen[JsValue, Organization]
  }

  implicit val organizationWrite = {
    import play.api.libs.json._
    import play.api.data.mapping.json.Writes._
    Write.gen[Organization, JsObject]
  }
}

import play.api.data.mapping._

case class Organization (
  id: String,
  name: String
)

object Organization {
  implicit val organizationFormat = {
    import play.api.libs.json._
    import play.api.data.mapping.json.Rules._
    import play.api.data.mapping.json.Writes._
    play.api.data.mapping.Format.gen[JsValue, JsObject, Organization]
  }
}

But now, I get the following error:

[error] src/main/scala/Connector.scala:55: could not find implicit value for parameter writer: play.api.data.mapping.Write[Organization,play.api.libs.json.JsValue]
[error]     def save(organization: Organization) = saveOrganization(organization)

The definition of the saveOrganization function:

def saveOrganization(organization: Organization)(implicit writer: Write[Organization, JsValue])

So, as we can see, the function actually need a Write[Organization, JsValue], but be defined a Format[JsValue, JsObject, Organization] which produce a WriteLike[Organization, JsObject]. Since Write[Organization, JsObject] works just fine, I would have expected this one to works too, but no. Maybe it's related to the fact that Write has those magical symbol + and - that I could never fully understand about variance and covariance but not Format.

What do you think?

Validate XML : error on elem disapeared with Node

I stumbled upon a puzzling error. With a From[Elem] in this code :

case class Commentaire(
  codeMessage: Option[String],
  libelleMessage: String,
  codePriorite: Option[String]
)

object Commentaire {
  implicit val messageRule: Rule[Elem, Commentaire] = From[Elem] { __ =>
    import jto.validation.xml.Rules._

    (
      (__ \ "DossEtude"  \ "codeMessage").read[Option[String]] ~
      (__ \ "DossEtude" \ "libelleMessage").read[String] ~
      (__ \ "DossEtude" \ "prioriteMessage").read[Option[String]]
    )(Commentaire.apply _)
  }

}

I got the following error :

[error] /Users/gbe/projects/nfc-nb/commons/server/src/main/scala/models/OMDEMessage.scala:92: No implicit view available from jto.validation.Path => jto.validation.RuleLike[scala.xml.Elem,Option[String]].
[error]       (__ \ "DossEtude" \ "codeMessage").read[Option[String]] ~

This error disepeared when I replaced Elem by Node :

object Commentaire {
  implicit val fmt = Json.format[Commentaire]

  implicit val messageRule: Rule[Node, Commentaire] = From[Node] { __ =>
    (
      (__ \ "DossEtude" \ "codeMessage").read[Option[String]] ~
      (__ \ "DossEtude" \ "libelleMessage").read[String] ~
      (__ \ "DossEtude" \ "prioriteMessage").read[Option[String]]
    )(Commentaire.apply _)
  }

}

Whether this library should completely support Elem or the user should be notified that the type Elem is not supported by this library.

Validating string as numbers

Please provide Rules implementing the parsing of strings as numbers following this pattern :

def convert[A](str: String): Option[A] where A = Int, Long, Double, etc. avoiding NumberFormatException.

The goal would be to be able to deserialise that type of data

{
   "description": "candy",
    "price": {
      "currency": "EUR",
      "value": "3.0000"
    }
}

defining the required type in the parsing Rule with calls as simple as :

case class Item(description: String, price: Double, currency: String)

val itemReader: Rule[JsValue, Item] = From[JsValue] { __ =>
  (
    (__ \ "description").read[String].map(PrivateDealId) ~
    (__ \ "name").read[String] ~
    (__ \ "price" \ "currency").read[String] ~
    (__ \ "price" \ "value").read[Double] // or read[Option[Double]]
  )(Item.apply _)                         // or read(stringToDoubleR)
}

Thanks.

References

Scala Cookbook - How to convert a String to an Int
Scala Cookbook - How to parse a number from a String

Sequences stripping out duplicates

I recently ran into a problem validating a form containing a list of items. If the user entered duplicate values in the list then the validations removed the duplicates.

I think I've tracked the problem down to the toMap call in the spm function here https://github.com/jto/validation/blob/master/validation-form/src/main/scala/Writes.scala#L42 which is removing duplicate keys from the map.

I have a solution for our project which involves our own version of the spm function.

Is my interpretation of this as a problem correct? If so I'd be happy to prepare a PR with our fix and some tests if that would help.

thanks
Joe

validation for circe

I did the mapping between validation and circe, but needed to put it in the circe package because some of the definitions are package private. Sharing it here, and happy to open a PR, if we can agree on where to put this, or how to handle it.

package io.circe.validation

import io.circe._
import jto.validation._

trait Rules extends DefaultRules[Json] {
  private def jsonAs[T](f: PartialFunction[Json, Validated[Seq[ValidationError], T]])(msg: String, args: Any*) =
    Rule.fromMapping[Json, T](f.orElse {
      case j ⇒ Invalid(Seq(ValidationError(msg, args: _*)))
    })

  implicit def stringR =
    jsonAs[String] {
      case Json.JString(v) ⇒ Valid(v)
    }("error.invalid", "String")

  implicit def booleanR =
    jsonAs[Boolean] {
      case Json.JBoolean(v) ⇒ Valid(v)
    }("error.invalid", "Boolean")

  // Note: Mappings of JsNumber to Number are validating that the JsNumber is indeed valid
  // in the target type. i.e: JsNumber(4.5) is not considered parseable as an Int.
  // That's a bit stricter than the "old" Read, which just cast to the target type, possibly loosing data.
  implicit def intR =
    jsonAs[Int] {
      case Json.JNumber(v) if v.toInt.nonEmpty ⇒ Valid(v.toInt.get)
    }("error.number", "Int")

  implicit def shortR =
    jsonAs[Short] {
      case Json.JNumber(v) if v.toShort.nonEmpty ⇒ Valid(v.toShort.get)
    }("error.number", "Short")

  implicit def longR =
    jsonAs[Long] {
      case Json.JNumber(v) if v.toLong.nonEmpty ⇒ Valid(v.toLong.get)
    }("error.number", "Long")

  implicit def jsNumberR =
    jsonAs[Json.JNumber] {
      case v @ Json.JNumber(_) ⇒ Valid(v)
    }("error.number", "Number")

  implicit def jsBooleanR =
    jsonAs[Json.JBoolean] {
      case v @ Json.JBoolean(_) ⇒ Valid(v)
    }("error.invalid", "Boolean")

  implicit def jsStringR =
    jsonAs[Json.JString] {
      case v @ Json.JString(_) ⇒ Valid(v)
    }("error.invalid", "String")

  implicit def jsObjectR =
    jsonAs[Json.JObject] {
      case v: Json.JObject ⇒ Valid(v)
    }("error.invalid", "Object")

  implicit def jsArrayR =
    jsonAs[Json.JArray] {
      case v @ Json.JArray(_) ⇒ Valid(v)
    }("error.invalid", "Array")

  implicit def doubleR =
    jsonAs[Double] {
      case Json.JNumber(v) ⇒ Valid(v.toDouble)
    }("error.number", "Double")

  implicit def bigDecimal =
    jsonAs[BigDecimal] {
      case Json.JNumber(v) if v.toBigDecimal.nonEmpty ⇒ Valid(v.toBigDecimal.get)
    }("error.number", "BigDecimal")

  import java.{ math ⇒ jm }
  implicit def javaBigDecimal =
    jsonAs[jm.BigDecimal] {
      case Json.JNumber(v) if v.toBigDecimal.nonEmpty ⇒ Valid(v.toBigDecimal.get.bigDecimal)
    }("error.number", "BigDecimal")

  implicit val jsNullR: Rule[Json, Json.JNull.type] = jsonAs[Json.JNull.type] {
    case Json.JNull ⇒ Valid(Json.JNull)
  }("error.invalid", "null")

  implicit def ooo[O](p: Path)(implicit pick: Path ⇒ RuleLike[Json, Json],
                               coerce: RuleLike[Json, O]): Rule[Json, Option[O]] =
    optionR(Rule.zero[O])(pick, coerce)(p)

  def optionR[J, O](r: ⇒ RuleLike[J, O], noneValues: RuleLike[Json, Json]*)(
      implicit pick: Path ⇒ RuleLike[Json, Json],
      coerce: RuleLike[Json, J]
  ): Path ⇒ Rule[Json, Option[O]] =
    super.opt[J, O](r, (jsNullR.map(n ⇒ n: Json) +: noneValues): _*)

  implicit final def mapR[O](implicit r: RuleLike[Json, O]): Rule[Json, Map[String, O]] =
    super.mapR[Json, O](r, jsObjectR.map { case j: Json.JObject ⇒ j.o.toList })

  implicit final def JsValue[O](implicit r: RuleLike[Json.JObject, O]): Rule[Json, O] =
    jsObjectR.andThen(r)

  implicit def pickInJson[II <: Json, O](
      p: Path
  )(implicit r: RuleLike[Json, O]): Rule[II, O] = {

    def search(path: Path, json: Json): Option[Json] = path.path match {
      case KeyPathNode(k) :: t ⇒
        json match {
          case j: Json.JObject ⇒
            j.o.toList.find(_._1 == k).flatMap(kv ⇒ search(Path(t), kv._2))
          case _ ⇒ None
        }
      case IdxPathNode(i) :: t ⇒
        json match {
          case Json.JArray(js) ⇒ js.lift(i).flatMap(j ⇒ search(Path(t), j))
          case _               ⇒ None
        }
      case Nil ⇒ Some(json)
    }

    Rule[II, Json] { json ⇒
      search(p, json) match {
        case None ⇒
          Invalid(Seq(Path → Seq(ValidationError("error.required"))))
        case Some(js) ⇒ Valid(js)
      }
    }.andThen(r)
  }

  private def pickInS[T](implicit r: RuleLike[Seq[Json], T]): Rule[Json, T] =
    jsArrayR.map { case Json.JArray(fs) ⇒ fs.asInstanceOf[Seq[Json]] }.andThen(r)
  implicit def pickSeq[O](implicit r: RuleLike[Json, O]) =
    pickInS(seqR[Json, O])
  implicit def pickSet[O](implicit r: RuleLike[Json, O]) =
    pickInS(setR[Json, O])
  implicit def pickList[O](implicit r: RuleLike[Json, O]) =
    pickInS(listR[Json, O])
  implicit def pickArray[O: scala.reflect.ClassTag](implicit r: RuleLike[Json, O]) =
    pickInS(arrayR[Json, O])
  implicit def pickTraversable[O](implicit r: RuleLike[Json, O]) =
    pickInS(traversableR[Json, O])
}

object Rules extends Rules

/cc @travisbrown

JSON

Reading playframework/playframework#1904 I understand that this is work in progress, but the current status of pulling play-json out makes this library unusable. validation does not include Json.parse (not to depend on Jackson), but it’s not possible to include full play-json to deal with that (because the AST gets duplicated) Do you have a plan on how to solve this?

Upgrade Cats version

Validation depends on Cats 0.6.0. The latest Cats version (at time of writing) is 0.9.0.

This is a major roadblock to using to using this library, as many other libraries depend on newer versions of Cats, causing incompatibilities among library dependencies.

Flattening nested case class in JSON Writes

I was able to flatten a nested case class in JSON using the form like ((__ \ "z").write[String] ~ aW)(unlift(B.unapply _)) (though it seems more intuitive to support __.write[A] ) but I can't figure out how to do it in the case that A is optional on B, like

case class A(x: Int, y: Int)
case class B(z: String, a: Option[A])

implicit val aW = To[JsObject] { __ =>
  import play.api.data.mapping.json.Writes._
  ((__ \ "x").write[Int] ~
    (__ \ "y").write[Int]
    )(unlift(A.unapply _))
}

implicit val bW = To[JsObject] { __ =>
  import play.api.data.mapping.json.Writes._
  ((__ \ "z").write[String] ~
    aW)(unlift(B.unapply _))
}

val b: B = B("baz", Some(A(11, 24)))
printf(Json.prettyPrint(To[B, JsObject](b)))

error: overloaded method value apply with alternatives:
  [B](f: B => (String, A))(implicit fu: play.api.libs.functional.ContravariantFunctor[[I]play.api.data.mapping.Write[I,play.api.libs.json.JsObject]])play.api.data.mapping.Write[B,play.api.libs.json.JsObject] <and>
  [B](f: (String, A) => B)(implicit fu: play.api.libs.functional.Functor[[I]play.api.data.mapping.Write[I,play.api.libs.json.JsObject]])play.api.data.mapping.Write[B,play.api.libs.json.JsObject]
 cannot be applied to (B => (String, Option[A]))
         ((__ \ "z").write[String] ~
                                   ^

jto / validation Goto Github PK

validation's Introduction

The unified data validation library

Overview

Design

Using the validation api in your project

Play dependencies

Documentation

Contributors

validation's People

Contributors

Stargazers

Watchers

Forkers

validation's Issues

References

Recommend Projects

Recommend Topics

Recommend Org