I am using circe-generic-extras to add a discriminator to a simple ADT but I notice that when I manually provide a Encoder for one of the ADT members, the discriminator does not encode as expected however the discriminator does encode as expected if I derive an Encoder instance.

Here is a simple example to illustrate the discrepancy:

import io.circe.Codec
import io.circe.generic.semiauto.deriveCodec
import io.circe.generic.extras.semiauto.deriveConfiguredCodec
import io.circe.Codec
import io.circe.generic.semiauto.deriveEncoder
import io.circe.generic.extras.semiauto.deriveConfiguredEncoder
import io.circe.generic.JsonCodec
import io.circe.generic.extras.Configuration
import io.circe._
import io.circe.syntax._

object sumADTFun extends App {

  @JsonCodec
  sealed trait InnerA

  sealed trait Wrapped

  object Wrapped {
    implicit val config: Configuration =
      Configuration.default
        .withDiscriminator("operation")

    implicit val forCodecs: Encoder[Wrapped] = deriveConfiguredEncoder
  }

  @JsonCodec
  case class B(value: String, num: Int) extends InnerA

  case class WrappedB(innerA: InnerA) extends Wrapped
  object WrappedB {
    // ** different values depending on semi-derived vs manual codecs **
    //implicit val semiDerivedEncoder: Encoder[WrappedB] = deriveEncoder

    // when using semi-derived:
    // {
    //   "innerA" : {
    //     "B" : {
    //       "value" : "foo",
    //       "num" : 42
    //     }
    //   },
    //   "operation" : "WrappedB"
    // }

    // implicit val manualCodecs: Codec[WrappedB] = Codec.from(
    //   Decoder[B].map(WrappedB.apply),
    //   Encoder[B].contramap((wb: WrappedB) => wb.innerA.asInstanceOf[B])
    // )
    implicit val forEncoder: Encoder[WrappedB] = Encoder[InnerA].contramap((wb: WrappedB) => wb.innerA)

    // when using manually specified codecs:
    // {
    //   "WrappedB" : {
    //     "value" : "foo",
    //     "num" : 42
    //  }
    // }
  }

  val b = WrappedB(B("foo", 42))

  println {
    (b: Wrapped).asJson
  }
}

Note the different behavior depending on which Encoder[WrappedB] is in scope.

Also, here is a scastie with the same code as above with the versions I am currently using. Is this a bug or am I just happening to do things in an unexpected way?

Thanks in advance.

See circe/circe#1582

One the configurable parameters for decoders is superfluous fields leniency. This is behavior is implemented in:

And it makes use of Decoder#validate to report when an error of this kind is found:

  /**
   * Build a new instance that fails if the condition does not hold for the
   * input.
   *
   * Note that this condition is checked before decoding with the current
   * decoder, and if it does not hold, decoding does not continue. This means
   * that if you chain calls to this method, errors will not be accumulated
   * (instead only the error of the last failing `validate` in the chain will be
   * returned).
   */
  final def validate(errors: HCursor => List[String]): Decoder[A] = new Decoder[A] {

Which, by definition stops decoding upon error.

Even if considered brittle, accumulating decoding is useful under certain situations so I think it would make sense to use Decoder#ensure instead so strictness doesn't hamper error accumulation.

Hey, please I came across something today that I am not sure how to get around.
Out of curiosity how will this be implemented?. The implicit val config: Configuration = Configuration.default.... works fine when there isn't an implicit deriveUnwrappedEncoder.

Currently when the code compiles I get

{
  "someFriends" : "two",
  "resUrl" : "One"
}

which is not what I want, I want

{
  "some_friends" : "two",
  "res_url : "One"
}

import io.circe._
import io.circe.generic.extras.semiauto._
import io.circe.generic.extras.defaults._
import io.circe.generic.extras._, io.circe.syntax._

final case class ResUrl(resUrl: String) extends AnyVal
object ResUrl {
implicit val config: Configuration = Configuration.default.copy(
  transformMemberNames = {
    case "resUrl" => "res_url"
    case other => other
  }
)
  implicit val resUrlEncoder: Encoder[ResUrl] = deriveUnwrappedEncoder[ResUrl]
}

final case class SomeFriends(someFriends: String) extends AnyVal
object SomeFriends {
implicit val config: Configuration = Configuration.default.copy(
  transformMemberNames = {
    case "someFriends" => "some_friends"
    case other => other
  }
  )
  implicit val resUrlEncoder: Encoder[SomeFriends] =
    deriveUnwrappedEncoder[SomeFriends]
}

final case class FooBar(someFriends: SomeFriends, resUrl: ResUrl)
object FooBar {
  implicit val foobarEncoder: Encoder[FooBar] = deriveEncoder[FooBar]
}

val a = FooBar(SomeFriends("two"), ResUrl("One"))

Apparently, this library is lagging behind circe (and also circe-generic)... which means that for the sake of Scala Steward, we need to split the versions b/w this library and circe. Is it intentional? 🤔

In the above code the strict decoder is establishing the fields it expects in a JSON object (to check no extra unexpected fields are present).

If a discriminator is specified e.g. "type", then we should be expecting a "type" field in the JSON, there is no need to pass it through the constructorNameTransformer function

I saw the newest circe dependency is upgraded to 0.14.6 but it hasn't been released to Maven central.

And can we upgrade circe to 0.14.7 as well? I saw there is an auto PR #350 but the ScalaJS build failed. I think we need to merge the PR to upgrade ScalaJS first: #348

Circe has released new versions of their libraries 0.14.6. circe-generic-extras should upgrade the version used to allow organizations that use strict dependency checking to upgrade circe.

There is already a PR up to make this change. #312

This issue is not intended to dictate the version of circe-generic-extras used to release this change. The maintainers can choose the version number of circe-generic-extras that meets their release strategy.

Scala: 2.13.10
Circe: 0.14.5

I have met strange problem with Circe decoding. I have created simple errors structure:

sealed trait GeneralError extends Throwable {
  def message: String
  def code: Int
}

object Errors {

  final case class FirstErrorImpl(message: String,  code: Int) extends GeneralError {
    override def getMessage: String = "error"
  }

  final case class SecondErrorImpl(message: String,  code: Int) extends GeneralError {
    override def getMessage: String = "error"
  }
}

And I have a program (using ZIO) which runs decoder with trait GeneralError and specific implementation FirstErrorImpl:

object App extends ZIOAppDefault {

  override def run: ZIO[Any with ZIOAppArgs with Scope, Any, Any] = {
    val defaultCirceConfig = Configuration.default.withSnakeCaseMemberNames
    implicit val circeConfig: Configuration = defaultCirceConfig

    implicit val decoderError: Decoder[GeneralError] = deriveConfiguredDecoder

    val json = parser.parse("""{"message": "error", "code": 65}""").toOption.get
    printLine(json)
    printLine(decoderError.decodeJson(json))
  }
}

Here I have an output: Left(DecodingFailure at : JSON decoding to CNil should never happen)
It seems, Circe cannot parse trait correctly and it always returns this error.

When I changed decoder to concrete implementation of my example error:
implicit val decoderError: Decoder[FirstErrorImpl] = deriveConfiguredDecoder

and run the program, I got correct parsed output:
Right(testing.Errors$FirstErrorImpl: error)

It looks like a problem wtih this decoder or maybe it should works like this? But if it is- why?
It's also a problem when I want e.g. to use decoder in sttp client or any other lib:

    basicRequest
      .get(uri"http://example.org")
      .response(asJsonEither[GeneralError, SomeResponse])
      .send(testingBackend)
      .map(_.body.left.map {
        case HttpError(body, statusCode) => FirstErrorImpl(body.message, statusCode.code)
        case DeserializationException(_, _) => SecondErrorImpl("serialization exception", 500, "dad")
      })
      .flatMap(ZIO.fromEither(_))

Because of parsing issue, I always get DeserializationException and code never enters case HttpError(body, statusCode) even if model is fine. So I always need to use implementation asJsonEither[FirstErrorImpl, SomeResponse] instead of some trait. It is not good solution imo.

Can you look at this?

Hi,

I am not sure whether it is a bug, but for sure it's quite a surprising behavior.

There is an option in the configuration called Configuration.default.withDefaults which allows for the derivation of non-case classes based on the fields they specify.

When combined with the following scalac options:-Xlint:strict-unsealed-patmat and -Wconf:cat=other-match-analysis:error it causes compilation to fail with "match may not be exhaustive" error.

This alone isn't yet that bad, but the error will propagate downstream through the chain of defined derivations and macros making it quite hard to debug for the end-users. I myself got it when I was trying to derive some tapir's type-classes that rely on circe codecs.

Here is a failed build on CI which illustrates that problem: https://github.com/softwaremill/livestub/runs/2586343239

Update: When trying to prepare a minimal example to reproduce I discovered that such a class needs also to specify an unapply method.

The minimal example to reproduce:

import io.circe.generic.extras.Configuration
import io.circe.generic.semiauto._

trait JsonSupport {
  implicit val config: Configuration = Configuration.default.withDefaults

  implicit val headerEncoder = deriveEncoder[Header]
}

class Header(val name: String, val value: String) {}

object Header {
  def unapply(h: Header): Option[(String, String)] = Some((h.name, h.value))

}

Originally reported in circe/circe#1756

When upgrading from Scala 2.12 to 2.13 am running into the following compiler error:

Usage of named or default arguments transformed this annotation
[error] constructor call into a block. The corresponding AnnotationInfo
[error] will contain references to local values and default getters instead
[error] of the actual argument trees
[error]   @JsonCodec(decodeOnly = true)
[error]    ^

What is the correct way to express this in Scala 2.13?

Note that I have set my compiler options to very strict (https://github.com/DavidGregory084/sbt-tpolecat/blob/master/src/main/scala/io/github/davidgregory084/TpolecatPlugin.scala) and all warnings (which this might be) are converted to errors.

There is 0.14.3 version of circe but the latest version of circe-generic-extras is still 0.14.2. I believe we are not the only ones that version these artifacts together 🙏

Hi! I just spent considerable time looking for a reason why a codec wasn't derived, and the root cause turned out to be the missing sealed keyword. As a way to avoid this happening for others, I would suggesting annotations like:

@implicitNotFound(
  """Could not found ConfiguredAsObjectCodec for type ${A}.
Some possible causes for this:
- ${A} isn't a case class or sealed trat
- some of ${A}'s members don't have codecs of their own"""
- missing implicit Configuration
)

to ConfiguredAsObjectCodec, EnumerationCodec, ReprAsObjectCodec, UnwrappedCodec and their encoder/decoder counterparts.

This also applies to circe-generic, although I'm using -extras more, so I'm starting here.

Let me know if this sounds like a good idea and I'll add these.

I imagine this is easier said than done but, well, now there's an issue for it.
Thanks!

#1 copy-pastes some code from circe-generic so that we can fix this bug without waiting for this change to be available in a Circe release. Once we have a Circe release that includes that change, we should revert 8124de1.

Please release a new version of this for Scala 3.0.0-M3 as I rely on it for several projects that are stuck waiting.

For now it is impossible to convenietly generate encoder that does not produce null values at the output.

Scenario:

@JsonCodec case class Params(p1: String, p2: Option[Int])
def process(a: Json): Unit = ???

val params = Params("abc", None)
process(json"""{"requestId": 123, "params": $params}""")

And I want the a not to contain null values.
I know I can:

1. preprocess a in the process() method - apply dropNullValues, use customized printer, etc
2. postprocess json where I use interpolation
3. customize codec by .mapJson(.dropNullValues)

First and second one seems like wrong place to solve this issue - its like workaround for misbehaving codec.

Third option requires lots of bolierplate code: Switching from annotation to manual derivation (creation of companion objects, etc), and then mapping derived encoder output.

I believe most correct solution would be to add io.circe.generic.extras.Configuration.dropNullValues. Then one could use it with derived codecs as well as with @ConfiguredJsonCodec.