RFC 004 currently says that the verification server is stateful and has a checkNoTestCasesHaveBeenMissed endpoint (https://github.com/palantir/conjure/blob/develop/docs/rfc/004-consistent-wire-format-test-cases.md#stateful-verification-server). However, based on our chats with the Conjure team when implementing tests this week, it appears that this information is out of date/this is no longer the case.

If that is so, the RFC should be updated so that it does not contain misleading information.

Right now, it just says TODO ask Mark.
https://github.com/palantir/conjure/blob/develop/docs/spec/wire.md#behaviour

I think the whole point of supporting unknown variants was that they round trip so that an old "proxy" can pass on variants it doesn't understand without loss of information.
@markelliot can you confirm the above so we can amend the wire spec?

Related: #61

It looks like #80 addressed the issue with type aliases not being allowed as types for path params, but external types with e.g a base-type of string are still broken.

Here's an example of a configuration that fails on versions >= 4.0.1:

types:
  imports:
    Long:
      base-type: string
      external:
        java: java.lang.Long
services:
  TestService:
    ...
    endpoints:
      testExternalPathParam:
        docs: path params with external types fail on >=4.0.1
        http: POST /foo/{bar}
        args:
          bar:
            type: Long
            param-type: path

fails to parse with the following error:

Exception in thread "main" java.lang.IllegalStateException: Path parameters must be primitives or aliases: "bar" is not allowed
        at com.google.common.base.Preconditions.checkState(Preconditions.java:530)
        at com.palantir.conjure.defs.validator.EndpointDefinitionValidator$NoComplexPathParamValidator.lambda$validate$3(EndpointDefinitionValidator.java:181)
...

There is no reason that conjure should enforce any constraints on import nameSpace. These names are just aliases to another file and have no impact on the generated IR.

I suggest that we remove this check: https://github.com/palantir/conjure/blob/develop/conjure-core/src/main/java/com/palantir/conjure/parser/types/names/Namespace.java#L40

Conjure's union types can end up being extremely inefficient in terms of bytes in the wire format, for example when encoding ASTs (where there is generally a lot of polymorphic branching). The double-encoding of the type for union types is the main first order cause, as for small-ish ASTs it will almost double the size:

Lit(Int(42)):
{"type":"lit","lit":{"type":"int","int":42}}
vs
{"lit":{"int":42}}

while not actually adding any new information. @markelliot believes the second form can be implemented for all currently-known targets.

Has this already been discussed? Is this worth the extra work to change?

@markelliot We discussed this a little bit ago, in the context of the Conjure encoding overhead (in terms of bytes used). As it will be significantly more expensive to make this change once this specification has been released (as it's a wire format change), I wanted to start a discussion now.

We have datetime, but no duration type.
We should add one that serializes using the ISO standard that Java uses (e.g. PT1s).
I've already come across a use case where people just use longs because nothing else is convenient, and we all know that's just a bug waiting to happen.

It would be useful to add the ability to specify markers on individual fields and enum values to conjure definitions.

The initial use case that this request is deprecating fields and enum values. In models or enums that are used widely, it may not be feasible to create full replacements (e.g. Person2 replaces Person). By marking a field deprecated, tooling could be used to ensure deprecated fields are removed following major releases, as an example.

Workarounds such as adding keywords in docs would work, but may clutter up useful documentation and wouldn't be portable.

Currently we allow fields names to be non-camel case for ease of migration. We however enforce that endpoint parameters must be camelCase.

We should be consistent and permissive and allow parameters to also be non-camel case

Currently, any project/language that wants to generate IR from YAML needs to use the Conjure CLI provided by this repo, which is packaged as JARs with an invocation script.

Invoking the CLI requires a JRE (which is heavyweight/unneeded in many environments like Docker) and it's also not easy to package because it currently uses a directory structure (JARs + script) rather than a single binary.

This currently makes the generation story in any non-Java environment unwieldy.

Although the language spec allows for the definition of multiple external types, the IR only has space for a single external type which conforms to the java naming convention.

When running the CLI with a directory as input, it failed with the following:

➜  bin ./conjure compile inner-in/ out.json 
Exception in thread "main" unacceptable character '' (0x0) special characters are not allowed
in "'reader'", position 0
	at com.fasterxml.jackson.dataformat.yaml.snakeyaml.reader.StreamReader.checkPrintable(StreamReader.java:93)
	at com.fasterxml.jackson.dataformat.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:192)
	at com.fasterxml.jackson.dataformat.yaml.snakeyaml.reader.StreamReader.<init>(StreamReader.java:60)
	at com.fasterxml.jackson.dataformat.yaml.YAMLParser.<init>(YAMLParser.java:154)
	at com.fasterxml.jackson.dataformat.yaml.YAMLFactory._createParser(YAMLFactory.java:420)
	at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:306)
	at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:14)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2691)
	at com.palantir.conjure.parser.ConjureParser$RecursiveParser.parseInternal(ConjureParser.java:105)
	at com.palantir.conjure.parser.ConjureParser$RecursiveParser.parse(ConjureParser.java:90)
	at com.palantir.conjure.parser.ConjureParser.parse(ConjureParser.java:60)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.palantir.conjure.defs.Conjure.parse(Conjure.java:35)
	at com.palantir.conjure.cli.ConjureCli.generate(ConjureCli.java:61)
	at com.palantir.conjure.cli.ConjureCli.main(ConjureCli.java:40)

At first, it was not at all clear what was going on. After further investigation, I determined that the issue was that the directory contained a .DS_Store file:

➜  bin ls -la inner-in 
total 24
drwxr-xr-x  4 nmiyake  nmiyake   128 Aug 17 10:26 .
drwxr-xr-x@ 9 nmiyake  nmiyake   288 Aug 17 10:26 ..
-rw-r--r--@ 1 nmiyake  nmiyake  6148 Aug 17 10:26 .DS_Store
-rw-r--r--@ 1 nmiyake  nmiyake   252 Aug 17 10:24 in.conjure.yml
➜  bin rm inner-in/.DS_Store 
➜  bin ./conjure compile ./inner-in out.json

2 issues here:

• The failure message should include the path of the file that fails to parse
• When processing a directory, the CLI should either:
  • Only include YAML files in the directory (.yml)
  • Ignore invisible files

The Conjure spec says that type names must be PascalCase: https://github.com/palantir/conjure/blob/develop/docs/spec/source_files.md#typename. However, it doesn't clearly articular the rules for 1-letter words/acronyms.

According to http://wiki.c2.com/?PascalCase:

WikiCase is more restrictive than PascalCase: WikiCase does not allow 1-letter words, but PascalCase does. ("ReadARecord" is PascalCase but not WikiCase.)

Based on that, ReadARecord should be a valid Conjure identifier, but based on testing the IR generator it is not.

I think it would be best to define exactly what the spec means when it says "PascalCase" (if the spec's definition of PascalCase doesn't allow 1-letter words that's fine, but we should state that, especially if this differs from definitions listed elsewhere as above)

I created a local directory with 2 Conjure files in it:

base.yml:

types:
  definitions:
    default-package: com.palantir.test
    objects:
      UserId:
        alias: string

other.yml:

types:
  definitions:
    default-package: com.palantir.other
    objects:
      Payload:
        union:
          user: UserId
          other: string

Running the Conjure CLI on the directory containing these 2 files failed with an error stating that UserId was not defined:

➜  ./bin/conjure compile input out.json
Exception in thread "main" java.lang.IllegalStateException: Unknown LocalReferenceType: TypeName{name=UserId}
	at com.palantir.conjure.defs.ConjureTypeParserVisitor$ByParsedRepresentationTypeNameResolver.resolveFromTypeName(ConjureTypeParserVisitor.java:79)
	at com.palantir.conjure.defs.ConjureTypeParserVisitor$ByParsedRepresentationTypeNameResolver.resolve(ConjureTypeParserVisitor.java:59)

When a directory is specified as input, I would expect the definitions to be combined/merged before the output is computed to allow references of this form -- without this functionality, it seems like specifying a directory is no better than running the CLI on each individual YAML file.

Concretely, this makes the use case of "manually importing" another Conjure definition much harder -- the definition has to be merged in by hand, which can be difficult if there are things like default-package defined.

BLUF: Conjure spec states that header and query parameters can only be primitives or optional of primitives. However, the IR generator allows list<string> query parameters (but not header parameters). The spec or generator should be updated so they are in agreement on behavior.

In the https://github.com/palantir/conjure/blob/develop/docs/spec/source_files.md#argumentdefinition section of the spec, the documentation for the type field states:

The type MUST be a primitive if the argument is a path parameter and primitive or optional of primitive if the argument is header or query parameter.

This seems to be enforced for header parameters, as specifying list<string> as a type fails:

services:
  TestService:
    name: Test Service
    package: com.palantir.another
    endpoints:
      headerParam:
        http: GET /headerParam
        args:
          foo:
            param-type: header
            param-id: FOO
            type: list<string>

Fails with:

Exception in thread "main" java.lang.IllegalStateException: Header parameters must be primitives, aliases or optional primitive: "foo" is not allowed

However, it is possible to generate query parameters that are collection types.

The input:

services:
  TestService:
    name: Test Service
    package: com.palantir.another
    endpoints:
      headerParam:
        http: GET /headerParam
        args:
          foo:
            param-type: query
            type: list<string>

Produces the IR:

{
  "version" : 1,
  "errors" : [ ],
  "types" : [ ],
  "services" : [ {
    "serviceName" : {
      "name" : "TestService",
      "package" : "com.palantir.another"
    },
    "endpoints" : [ {
      "endpointName" : "headerParam",
      "httpMethod" : "GET",
      "httpPath" : "/headerParam",
      "args" : [ {
        "argName" : "foo",
        "type" : {
          "type" : "list",
          "list" : {
            "itemType" : {
              "type" : "primitive",
              "primitive" : "STRING"
            }
          }
        },
        "paramType" : {
          "type" : "query",
          "query" : {
            "paramId" : "foo"
          }
        },
        "markers" : [ ]
      } ],
      "markers" : [ ]
    } ]
  } ]
}

This does not match what is outlined in the spec -- the spec should be updated or the implementation should be changed to match it.

from slack:

looks like there may be a bug in the recently added regression fix for #80 (ty for getting this in btw). it looks like the TypeDefinition is invalid if the alias is imported from a conjure-import even if it's valid. e.g.:
foo.yml:

types:
  conjure-imports:
    common: common.yml
services:
  FooService:
    name: Foo Service
    package: com.palantir.foo
    base-path: /foo/bar
    default-auth: header
    endpoints:
      endpoint:
        http: POST /{foo}
        args:
          foo: common.foo

common.yml:

  definitions:
      default-package: com.palantir.common
      objects:
        foo:
          alias: string

not the end of the world since we can just add it into the local conjure definition and it works

Similar to #61, #98:

types:
  imports:
    Long:
      base-type: string
      external:
        java: java.util.Long
services:
  ExampleService:
    name: Example
    package: com.palantir.foo
    base-path: /foo
    default-auth: header
    endpoints:
      getFoo:
        http: GET /foo
        args:
          stringHeader:
            param-id: ParamId
            param-type: header
            type: optional<Long>

Fails with

Exception in thread "main" java.lang.IllegalStateException: Header parameters must be primitives, aliases or optional primitive: "stringHeader" is not allowed
        at com.google.common.base.Preconditions.checkState(Preconditions.java:530)
        at com.palantir.conjure.defs.validator.EndpointDefinitionValidator$NoComplexHeaderParamValidator.lambda$validate$3(EndpointDefinitionValidator.java:209)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
        at com.palantir.conjure.defs.validator.EndpointDefinitionValidator$NoComplexHeaderParamValidator.validate(EndpointDefinitionValidator.java:194)
        at com.palantir.conjure.defs.validator.EndpointDefinitionValidator$NoComplexHeaderParamValidator.validate(EndpointDefinitionValidator.java:188)
        at com.palantir.conjure.defs.validator.EndpointDefinitionValidator.validate(EndpointDefinitionValidator.java:82)
        at com.palantir.conjure.defs.validator.EndpointDefinitionValidator.validateAll(EndpointDefinitionValidator.java:59)
        at com.palantir.conjure.defs.ConjureParserUtils.parseEndpoint(ConjureParserUtils.java:353)
        at com.palantir.conjure.defs.ConjureParserUtils.lambda$parseService$6(ConjureParserUtils.java:256)
        at java.util.Collections$SingletonMap.forEach(Collections.java:4910)
        at com.palantir.conjure.defs.ConjureParserUtils.parseService(ConjureParserUtils.java:255)
        at com.palantir.conjure.defs.ConjureParserUtils.lambda$parseConjureDef$3(ConjureParserUtils.java:211)
        at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
        at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1505)
        at com.palantir.conjure.defs.ConjureParserUtils.lambda$parseConjureDef$4(ConjureParserUtils.java:210)
        at java.util.ArrayList.forEach(ArrayList.java:1249)
        at com.palantir.conjure.defs.ConjureParserUtils.parseConjureDef(ConjureParserUtils.java:197)
        at com.palantir.conjure.defs.Conjure.parse(Conjure.java:34)
        at com.palantir.conjure.cli.ConjureCli.generate(ConjureCli.java:61)
        at com.palantir.conjure.cli.ConjureCli.main(ConjureCli.java:40)

but succeeds if optional<Long> is switched to just Long.

What happened?

I created a valid conjure definition that introduced a cycle of package dependencies:

types:
  definitions:
    objects:
      Type1:
        package: com.palantir.odd
        fields:
          type2: Type2
      Type2:
        package: com.palantir.even
        fields:
          type3: Type3
      Type3:
        package: com.palantir.odd
        fields: {}
services: {}

This is valid according to the spec, and the current conjure compiler successfully create an IR from this, however any straightforward bindings for a language like Go which forbids circular package dependencies would not be able to generate code that compiles for these definitions by using the packages as defined in conjure.

$ go build ./conjure/...
can't load package: import cycle not allowed
package internal-github/jhenke/bug-circular-conjure-imports/conjure/com/palantir/even
	imports internal-github/jhenke/bug-circular-conjure-imports/conjure/com/palantir/odd
	imports internal-github/jhenke/bug-circular-conjure-imports/conjure/com/palantir/even
import cycle not allowed
package internal-github/jhenke/bug-circular-conjure-imports/conjure/com/palantir/even
	imports internal-github/jhenke/bug-circular-conjure-imports/conjure/com/palantir/odd
	imports internal-github/jhenke/bug-circular-conjure-imports/conjure/com/palantir/even

What did you want to happen?

I want there to be a recommendation on how languages which do not support circular dependencies should handle this situation or a change in the conjure spec and compiler to forbid this behavior.

Due to an NPM security incident where ESLint was compromised

Example red build: https://circleci.com/gh/palantir/conjure/136

> Task :conjure-api:publishTypeScript FAILED
npm WARN notice Due to a recent security incident, all user tokens have been invalidated. Please see https://status.npmjs.org/incidents/dn7c1fgrr7ng for more details. To generate a new token, visit https://www.npmjs.com/settings/~/tokens or run "npm login".
npm ERR! publish Failed PUT 401
npm ERR! code E401
npm ERR! 404 You must be logged in to publish packages. : conjure-api
npm ERR! 404 
npm ERR! 404  'conjure-api' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/circleci/.npm/_logs/2018-07-18T16_49_37_836Z-debug.log

FAILURE: Build failed with an exception.

Need to get hold of a new $NPM_AUTH_TOKEN for the palantir npm user account.

This issue will track changes that we would like to make as part of a future major revision to the Conjure IR

#80 fixed the issue for path params but not header params.

Right now the following are all parsed equivalently:

MyType:
  fields:
  docs: Some documentation about the whole type
  package: com.palantir.example

MyType:
  fields: null
  docs: Some documentation about the whole type
  package: com.palantir.example

MyType:
  fields: {}
  docs: Some documentation about the whole type
  package: com.palantir.example

The source files spec says that fields should be a map. We coerce null to an empty collection when deserializing from the wire format, which is fine, but we should keep them distinct in the source files. Therefore, the first two examples here should not be treated as valid Conjure and the parser should reject them.

If a program already has the input YAML in-memory, requiring it to write the file to a temporary file to be used as input is cumbersome -- it would be nice if the CLI supported reading from STDIN.

Related to #65 (but lower priority)

The CLI properly generates an IR for the following input YAML:

types:
  definitions:
    default-package: api
    objects:
      Example:
        fields:
          data: string

However, when the object type name is changed to ExampleA:

types:
  definitions:
    default-package: api
    objects:
      ExampleA:
        fields:
          data: string

The CLI fails with the following output:

➜  bin ./conjure compile in.conjure.yml out.json
Exception in thread "main" java.lang.RuntimeException: com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not construct Map key of type com.palantir.conjure.parser.types.names.TypeName from String ("ExampleA"): not a valid representation: null
 at [Source: in.conjure.yml; line: 5, column: 7] (through reference chain: com.palantir.conjure.parser.Json["types"]->com.palantir.conjure.parser.types.Json["definitions"]->com.palantir.conjure.parser.types.Json["objects"])

The output message in the exception is "Can not construct Map key of type com.palantir.conjure.parser.types.names.TypeName from String ("ExampleA"): not a valid representation: null" -- this does not make it clear at all why this name failed.

The failure should be more along the lines of "ExampleA" is not a valid type name: type names be PascalCase and cannot contain 1-letter words or "ExampleA" is not a valid type name: type names must match regular expression ...

From what I can tell there aren't any restrictions currently, but it seems like we should probably have some reasonable limits. Namespace aliases are currently limited to ^[_a-zA-Z][_a-zA-Z0-9]*$ and that could make sense for type/field/alias names as well.

Returning maps with complex keys is a potential foot-gun and we should just prevent people from doing so

The spec indicates that doubles serialize as number | "NaN" - would infinities be treated as NaN then?

e.g. "2017-01-02T04:04:05.000000000+01:00[Europe/Berlin]"

Can we change Conjure to not include zone IDs? Is there a reason why they're the way they currently are?

The links in the examples and service example sections are all broken.

Even though all state encoding should go through the current type-able constructs, there are cases where a constant value becomes part of an API. This can happen when interacting with a tertiary API which accepts dynamic keys, which are fixed for the service. For example, if a service registers with a notification system, the channel it registers might be constant, but not encodable in any currently-existing way (imagine e.g. a channel name that takes / or :; this cannot be encoded in the current enum specification), but should still be documented and accessible.

Proposal:

In the definitions field, add a constants subkey (see the snippet at the end for a sample conjure specification). Constants have a type and a value. The type can potentially be any locally-valid conjure type. Until we have concrete requirements for more complex types, the initially supported set of values should be restricted to primitives. The value would be a valid yaml-encoding of the conjure value. As in, it should be possible to deserialize the value of the value key to generate the constant value (though this isn't the recommended implementation for code generators).

The values must be validated (i.e. a invalid constant value must be a compile error, not a runtime error) according to the Conjure spec.

Constants also have a package & wrapper field, which correspond to the package & compilation unit the constants will be attached to. The $package.$wrapper name must be different from any Conjure object (i.e. there will be no mixed constant & object output). These can both be set through file defaults as well (default-package & default-wrapper).

Trade-offs/Risks

• Tertiary service APIs can force the usage of constants. This enables the formal encoding and documentation, instead of e.g. adding the constant in the doc string.
• Encoding of non-primitive constants not necessarily a good idea. This should be held off until concrete uses are found.
• This can be misused instead of fields/enums. For example, services might build endpoints of a map<string, any> type, with a set of assorted constant keys. This is an anti-pattern, and should be strongly discouraged.
• This can't solve the problem in cases where the decision is dynamic. In the notification example, imagine the target resource channel can be computed using a lambda (e.g. if there's a notification channel per dataset). There is no way to encode this (nor should there be), limiting the usefulness of this extension.

Sample

types:
  definitions:
    constants:
      default-package: com.example.foo
      default-wrapper: Constants
      ChannelA:
        docs: "Notification channel for high-priority service notifications"
        type: string
        value: "my:channel"
      ReadOperation:
        docs: "Operation used to enforce read checks"
        type: string
        value: "my-service:read"

Per Dan's comment on conjure cli

I think the naming looks good here. Long term, I think it would be kinda cool if people could just brew install conjure and it would pull in this core YML -> IR cli, and a few stock generators. I'd imaging people could then run conjure typescript and it would discover the conjure-typescript executable from the $PATH, maybe even prompt people through some steps (where is your input yml file, where would you like to generate your typescript).

example usage:

conjure compile <input> <output>            // compiles input yml files into ir
conjure typescript...                                    // runs typescript generator
conjure java..                                                // runs java generentor

Invoking the CLI with no arguments results in a stack trace:

➜  bin ./conjure       
Exception in thread "main" java.lang.IllegalArgumentException: Usage: conjure compile <target> <output>
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:134)
	at com.palantir.conjure.cli.ConjureCli.parseCliConfiguration(ConjureCli.java:51)
	at com.palantir.conjure.cli.ConjureCli.main(ConjureCli.java:40)

This output is not very friendly from a usability perspective -- would be nicer if this output was just the error message ("Usage: conjure compile ") and didn't include the stack trace.

Please document how conjure transports requests and responses over http. Right now we have to use https://github.com/palantir/http-remoting#api-contract and Java code as a reference for implementation in other languages.

Hey Conjure team!

We are trying to move our services to Conjure to "normalize" our code, and I hit the situation recently. I could not find any work-around.

I want my service to return paginated results. The return objects should have the following structure:

public interface PaginatedResult<R> {
    R getResult();
    Optional<String> getPageToken();
}

For some reasons, I have the PaginatedResult object above already implemented in an external library. Ideally, I would like my conjure config to look like:

types:
  imports:
    PaginatedResult:
      external:
        java: com.palantir...........PaginatedResult
definitions:
    default-package: com.palantir.....
    objects:
      MyResponseObject:
        fields:
          name: string
          otherField: string
service:
  MyService:
    name: MyServiceName
    package: com.palantir.......
    default-auth: header
    endpoints:
      myEndpoint:
        http: POST /endpoint
        args:
          request:
            type: string
        returns: PaginatedResult<MyResponseObject>

The problem is that it seems like Generics for external types are not handled yet. Am I correct?

A work-around for this would be to be able for conjure created object to inherit an external type. This way, I could make MyResponseObject inherit a PaginatedResult object that would contain only the page token, and it would be fine (but I find it less elegant). But still, this can not be done neither if I am not mistaken.

The only working solution I found is by defining two object in my conjure.yml file. The original MyResponseObject plus a MyResponseObjectPaginated that is basically a wrapper around the first object to add the page token. But there are some drawbacks with that:

1. We want to implement this "pagination" mechanism in a bunch of services and it means that we will always have to create twice a much objects.
2. More importantly, in the actual code, I actually get my PaginatedResult object from a call to an external library (not the service itself). Which means that ATM, I have to: 1. call my library which returns my PaginatedObject 2. unwrap it 3. construct the MyResponseObjectPaginated from the unwrapped object 4. returns this object. This could be done by a single call to my external library if generics were handled.

I hope it was clear enough. My questions are:

• Do you have a way to handle that kind of situation? (maybe I missed some features)
• If not, are there some FR on the short-term roadmap that will solve this?

There are different content types specified in the Conjure Wire Spec for request and response. Is it intentional? Can we make it consistent?

HTTP Request:

"For Conjure endpoints that define a body argument, a Content-Type header MUST be added. If the body is of type binary, the content-type application/octet-stream MUST be used. Otherwise, clients MUST send Content-Type: application/json."

HTTP Response:

"if the de-aliased return type is binary, servers MUST send Content-Type: application/octet-stream, otherwise, servers MUST send Content-Type: application/json;charset=utf-8"

How much effort would it be to add QOS as one of the possible ErrorCodes here? Is it just adding a QOS to this list? https://github.com/palantir/conjure/blob/develop/conjure-api/src/main/conjure/conjure-api.yml

Identified in palantir/conjure-verification#45

The wire spec states that all endpoints defining a body parameter must set one of two content-type headers:

Content-Type header - For Conjure endpoints that define a body argument, a Content-Type header MUST be added. If the body is of type binary, the content-type application/octet-stream MUST be used. Otherwise, clients MUST send Content-Type: application/json.

However, a bit further up under "Body parameter" it suggests (even recommends) that clients should send an empty body in the case of an absent optional body parameter, which doesn't seem to be valid JSON:

[when] the de-aliased argument is type optional<T> and the value is not present: it is RECOMMENDED to send an empty request body, although clients MAY alternatively send the JSON value null

Since we probably have to support this use case, we should say that content-type MUST NOT be set when sending an absent optional body as an empty body.

gen code:

            throw new IllegalArgumentException(
                    "Some required fields have not been set: " + missingFields);

req:

            throw new IllegalArgumentException(
                    "Some required fields have not been set: " + StringUtils.join(missingFields, ","));

I'm not sure if I'm reading this right, but this looks incorrect to me:

First of all, this precludes the (as far as I can tell) allowed format of:

EnumName:
  values:
    - value: ENUM_VALUE
       docs: This is a doc for the enum value

Second of all, I'm pretty sure ConjureType isn't correct anyway, since the following would be invalid:

EnumName:
  values:
    - string

I would propose creating a new type definition in the docs (idk how this is internally represented in the parser) for enum values that represents this.

@bavardage
I have a type which is an alias of a string

      MyTokenType:
        alias: string

and then I use it as a header parameter

    endpoints:
      myEndpoint:
        http: POST /
        args:
          someToken:
            param-id: My-Token
            param-type: header
            type: optional<MyTokenType>
          request: MyEndpointRequest

Exception in thread "main" java.lang.IllegalStateException: Header parameters must be primitives, aliases or optional primitive: "temporaryCredentialsAuthToken" is not allowed

I am able to switch the alias type to be itself alias: optional<string> (and take non-optional MyTokenType as the header param type) but I'm not sure the generated code then works (I haven't yet confirmed with a failing test) - @JsonCreator annotation ends up taking an Optional<String>

If the second argument to the compile command is a directory, the CLI currently fails with the following output:

➜  bin ./conjure compile ./inner-in inner-out 
Exception in thread "main" java.io.FileNotFoundException: inner-out (Is a directory)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
	at com.fasterxml.jackson.core.JsonFactory.createGenerator(JsonFactory.java:1140)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:950)
	at com.palantir.conjure.cli.ConjureCli.generate(ConjureCli.java:62)
	at com.palantir.conjure.cli.ConjureCli.main(ConjureCli.java:40)

The output should be more human-readable -- something like "the specified output path must be a file, but was a directory". It's mostly possible to deduce this from the stack trace that's printed, but a human-readable message would be much more user-friendly.

Currently, it seems that conjure disallows aliases of strings as path parameters: Path parameters must be primitives or aliases: "foo" is not allowed

type:
  definitions:
    default-package: package.example
    objects:
      StringAlias:
        alias: string
services:
  ExampleService:
  ...
  endpoints:
    endpoint:
      http: POST /{foo}
      args:
        foo: StringAlias

Currently, the Conjure language allows users to specify default-package and package fields for objects, services and errors. We use these fields together with the name to guarantee name (type, service, error) uniqueness. The issue is that every non-java generator only uses the name and therefore have to assume that names are not unique.

I suggest that we remove the these fields from the language and IR to improve ergonomics of non-java APIs. To avoid breaking Java generators, we would add the group as a command line argument

Our syntax for markers is pretty verbose, which makes large APIs with lots of safe arguments (marked with an appropriate Safe type) very verbose and a little hard to read. I think we should strive to make things both readable and concise. I'd like to propose we explore ways to make markers more concise.

As an example, today:

getFoo:
  http: GET /{foo}
  args:
    foo:
      type: string
      markers:
        - Safe

I think it'd be good if we could make this more concise, and allow types to carry marker information, e.g.:

getFoo:
  http: GET /{foo}
  args:
    foo:
      type: @Safe string

If using arg shorthand, this can actually collapse to:

getFoo:
  http: GET /{foo}
  args:
    foo: @Safe string

More concretely, I'd propose that one can use @<Type> as a marker anywhere type is allowed, and that we'd parse this as a marker on the item. The @ syntax follows most language definitions for annotations, and we use markers as an indication of meta information much the same way most languages use annotations, so the correspondence seems friendly.

To add precision, something like:

MARKER := "@"TYPE
MARKED_TYPE := (MARKER )*TYPE

And replacing TYPE with MARKED_TYPE wherever markers are allowed.

solely because of import javax.annotation.Generated;

You could do the same thing that immutables does and detect which one is on classpath when generating.

We have several projects extensively using 64-bit integers which cannot be migrated to integer53 (safelong).

palantir/gradle-conjure#43

In the case where the generator CLI is being invoked by another program, it would be nice to have the option to have the output written to STDOUT rather than to a file -- without this ability, the invoking program has to choose a temporary file, have the generator write to that file, read the temporary file and then remove it.

HTTP headers are case insensitive, so Conjure should prevent people from defining two headers Foo-Bar and FOO-BAR.