I've a field all uppercase, called SKU.
If I set the field as public, all works well.
If i set the field private with getters and setters, the column isn't populated.
If needed I can try to reproduce it in your tests.

I'm using v. 2.8.8 but I've reproduced also with the 2.9.0

(by @exbe https://github.com/FasterXML/jackson-dataformat-yaml/issues/85)

SnakeYaml converts some string values into Boolean while resolving input (link)

 public static final Pattern BOOL = Pattern.compile("^(?:yes|Yes|YES|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF)$");

While it is useful feature, some yamls might be converted with wrong value. Think about country code "NO", which is Norway.

Is there a way to disable or have better control over this with YamlParser features?

PS
Workaround is simple, just surround value with quotes, e.g.

countries:
    - AU
    - "NO"
    - MOON

The default hard-coded behaviour does not work for all CSV parsers.
I've worked around the issue by adding support for using the configured escape character instead of doubled-up doublequotes ""
I've added a pull request for this feature: #41

In case you're curious, we're trying to generate CSVs for ingestion into a Vertica database and "" needs to be \"

(from FasterXML/jackson-dataformat-csv#146 by @robmoore)

We're working with a set of files that contains both empty strings and NULL values. We are using .withNullValue("NULL") as well as DeserializationFeature.ACCEPT_EMPTY_STRING_AS_NULL_OBJECT but are ending up with empty strings rather than null values for these cases.

In issue 112, @cowtowncoder suggested it might be possible to support an alternate notation in addition to the explicit null value indicated in withNullValue().

Or I need to join column header after writeValueAsString() ?

Below is similar problem:
http://stackoverflow.com/questions/35295997/set-localized-column-title-in-csv-file-using-jackson

When CSV is consisted of single column, with no column delimiter, the iterator only gets the first row, all the rests are ignored.
Could you look into this issue? Thanks!

The idea

Original Jackson has @JsonAnySetter annotation which allow you deserializae any unknow fields into Map. Why not add this possibility for unknow columns? The setter could be Collection<String> with rest of column values or for Jackson compatibility Map<String, Object> for nested types (I do not know whether you support nested types during deseriablization) or simple Map<String, String> with keys as column order number. (just as idea)

The implementation should write all column as they described in JsonPropertyOrder and all other put into annotation setter.

The code

I have tried to run the following:

package com.json.jackson;

import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;

import java.util.List;

public class JacksonCsvAnyGetter {
    public static void main (String[] args) throws Exception {
        String csv =    "myPackageName,lang,packageTag1,packageValue1\n"
                +       "myFileName,lang,FileTag1,FileValue1";


        CsvMapper mapper = new CsvMapper();
        CsvSchema schema = mapper.schemaFor(Dto.class); // schema from 'Dto' definition
        MappingIterator<Dto> it = mapper.readerFor(Dto.class).with(schema)
                .readValues(csv);
        while (it.hasNextValue()) {
            Dto value = it.nextValue();
            System.out.println(value);
        }
        List<Dto> all = it.readAll();
    }
}

package com.json.jackson;

import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;

import java.util.HashMap;
import java.util.Map;

@JsonPropertyOrder({"name", "lang"})
public class Dto {
    private String name;
    private String lang;
    private Map<String, Object> all = new HashMap<>();

    public String getName() {
        return name;
    }

    public void setName(final String name) {
        this.name = name;
    }

    public String getLang() {
        return lang;
    }

    public void setLang(final String lang) {
        this.lang = lang;
    }

    @JsonAnySetter
    public void set(String name, Object value) {
        all.put(name, value);
    }

    @Override
    public String toString() {
        return "Dto{" +
                "name='" + name + '\'' +
                ", lang='" + lang + '\'' +
                ", all=" + all +
                '}';
    }
}

pom.xml snippet

        <properties>
            <jackson.version>2.9.1</jackson.version>
        </properties>

        <dependency>
            <groupId>com.fasterxml.jackson</groupId>
            <artifactId>jackson-bom</artifactId>
            <version>${jackson.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-csv</artifactId>
            <version>${jackson.version}</version>
        </dependency>

The exception

Exception in thread "main" com.fasterxml.jackson.dataformat.csv.CsvMappingException: Too many entries: expected at most 2 (value #2 (11 chars) "packageTag1")
 at [Source: (StringReader); line: 1, column: 20]
	at com.fasterxml.jackson.dataformat.csv.CsvMappingException.from(CsvMappingException.java:23)
	at com.fasterxml.jackson.dataformat.csv.CsvParser._reportCsvMappingError(CsvParser.java:1223)
	at com.fasterxml.jackson.dataformat.csv.CsvParser._handleExtraColumn(CsvParser.java:978)
	at com.fasterxml.jackson.dataformat.csv.CsvParser._handleNextEntry(CsvParser.java:839)
	at com.fasterxml.jackson.dataformat.csv.CsvParser.nextFieldName(CsvParser.java:649)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
	at com.fasterxml.jackson.databind.MappingIterator.nextValue(MappingIterator.java:277)
	at com.json.jackson.JacksonCsvAnyGetter.main(JacksonCsvAnyGetter.java:20)

(moved from FasterXML/jackson-dataformat-csv#116 by @georgewfraser)

When writing arrays to CSV, if there is a null, columns get written out of order. Reproduced in 2.7.1 with this test: https://gist.github.com/georgewfraser/164647fbaca61aadf9ee

(from FasterXML/jackson-dataformat-csv#148 by @bhchandra )

This is related to: writeValue writes out header in each call. #71 which is closed.

The following was mentioned in the page.

"But I think what you may really want to use is the new method(s) added in 2.5: ObjectWriter.writeValues() (http://fasterxml.github.io/jackson-databind/javadoc/2.5/com/fasterxml/jackson/databind/ObjectWriter.html#writeValues%28java.io.OutputStream%29).
It returns a SequenceWriter which has write and writeAll methods to use.
That should properly handle headers; and if not, that would definitely be a bug".

I am not sure, please take a look at this code.

                csvMapper.configure(JsonGenerator.Feature.AUTO_CLOSE_TARGET, false);
                fileStream = Files.newOutputStream(this.path, CREATE, WRITE, APPEND);
                jsonGenerator = csvMapper.getFactory().createGenerator(fileStream);

                //Works Good. Doesn't write header for each call
                csvMapper.writer(schema).writeValue(jsonGenerator, data);   
  
                //Writes header for each call.        
                csvMapper.writer(schema).writeValues(fileStream).write(data);

"In the end use of ObjectWriter.writeValues() is the recommended method, and the proper resolution"

I want to use writeValues() since it is the recommended.
Please resolve if I am making some mistake in code or this is a bug?
using 2.8.7.

Looks like a very simple case of using MappingIterator with "untyped" type of String[] fails to properly detect end of input.

(from FasterXML/jackson-dataformat-csv#85 by @MattFriedmanAfilias)

When an object is encountered when writing CSV, the library throws an exception.

Problem: The model might be changed and an object field could be introduced. In this case, the developer will not necessarily update any tests to include the new field.

If that happens the error may only be detected at runtime. We want to detect such problems sooner, or avoid them altogether.

It should be possible to detect the error sooner, or to introduce a setting that would cause the exception to not be thrown and to provide instead some default behavior.

Such behavior might be to simply print "Object" or "n/a" when an object is encountered, rather than throwing an exception; a log message warning about this might be useful too.

The developer should be in control of this setting IMO. The library should make it possible for the developer to decide on the behavior rather than only throwing an exception for this scenario.

I have an issue with CSV reader, when escapeChar is set to \ and nullValue to \N.

In this case reading a CSV like the following:

id,value
1,\N

results in value="N" instead of value=null.

I tried to replace nullValue with "\\\\N", but it doesn't help.

Any ideas?

Thank you in advance!

When I use a Set (but I think a Collection) of objects with feature JsonGenerator.Feature.IGNORE_UNKNOWN extra line separators are added.
I paste the code to reproduce it:

    protected static class Person
    {
        private String name;
        private String surname;

        public Person()
        {
        }

        public Person( String name, String surname )
        {
            this.name = name;
            this.surname = surname;
        }

        public String getName()
        {
            return name;
        }

        public void setName( String name )
        {
            this.name = name;
        }

        public String getSurname()
        {
            return surname;
        }

        public void setSurname( String surname )
        {
            this.surname = surname;
        }
    }

    protected static class MyClass
    {
        private Set<Person> people;
        private String address;
        private String phoneNumber;

        public MyClass()
        {
        }

        public Set<Person> getPeople()
        {
            return people;
        }

        public void setPeople( Set<Person> people )
        {
            this.people = people;
        }

        public String getAddress()
        {
            return address;
        }

        public void setAddress( String address )
        {
            this.address = address;
        }

        public String getPhoneNumber()
        {
            return phoneNumber;
        }

        public void setPhoneNumber( String phoneNumber )
        {
            this.phoneNumber = phoneNumber;
        }
    }

    public void testEmbeddedObjects() throws Exception
    {
        CsvMapper mapper = mapperForCsv();
        mapper.configure( JsonGenerator.Feature.IGNORE_UNKNOWN, true );
        CsvSchema schema = CsvSchema.builder()
                .addColumn( "people" ) // here I'm skipping phoneNumber so I need to use IGNORE_UNKNOWN feature
                .addColumn( "address" )
                .build()
                .withHeader();

        Person firstPerson = new Person( "Barbie", "Benton" );
        Person secondPerson = new Person( "Veltto", "Virtanen" );
        Set<Person> people = new HashSet<>();
        people.add( firstPerson );
        people.add( secondPerson );
        MyClass myClass = new MyClass();
        myClass.setPeople( people );
        myClass.setAddress( "AAA" );
        myClass.setPhoneNumber( "123" );

        String result = mapper.writer( schema ).writeValueAsString( myClass );
        int numberOfLines = result.split( "\n" ).length;
        assertEquals( 2, numberOfLines ); // header and data (here fails with 3)
    }

Version is 2.9.0
A workaround would be to write a custom serializer, but with a class with a lot of fields it was very hard to find which field has broken silently the csv.
Is there any way to use the toString rappresentation for cases like this one?
If you prefer I can open a PR with that code instead of paste it here.

In enterprise contexts, CSV formats can take on really strange formats, and in my experience those formats commonly include sub-headers, headers that do not correspond to lower row values, and placeholders for CSVs that serve as templates to be filled in.

Jackson doesn't seem to currently support anything except the idea of a single header corresponding to the data that will be in a CSV.

There are libraries that do accomplish this -- I've used beanio -- but are just archaic in almost every other way. We use Jackson so much for other dataformats and use-cases, I'd love to keep to Jackson for CSVs as well.

Looks like a commonly supported option with CSV (family of formats) is to allow skipping of empty rows; that is, ones that only contain white-space.

Fails to recognise explicit binary type tag !!binary (tag:yaml.org,2002:binary)

Also fails to decode to spec as defined in http://yaml.org/type/binary.html.

Binary data is serialized using the base64 format as defined by RFC2045 (MIME), with the following notes:

• The content is not restricted to lines of 76 characters or less.

• Characters other than the base64 alphabet, line breaks and white space are considered an error.

Sample test case the demonstrates the issues:

From "Example 2.23. Various Explicit Tags" from spec (1.1 and 1.2) - http://www.yaml.org/spec/1.2/spec.html

---
picture: !!binary |
 R0lGODlhDAAMAIQAAP//9/X
 17unp5WZmZgAAAOfn515eXv
 Pz7Y6OjuDg4J+fn5OTk6enp
 56enmleECcgggoBADs=

@Test
public void testBinary() throws IOException {
    final ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
    try (final InputStream inputStream = getClass().getResourceAsStream("yaml1.3-example2.23.yaml")) {
        final JsonNode bean = mapper.readTree(inputStream);
        final JsonNode picture = bean.get("picture");
        assertEquals(JsonNodeType.BINARY, picture.getNodeType()); // fails
        final byte[] gif = picture.binaryValue(); // also fails 
        assertEquals(65, gif.length);
        final byte[] actualFileHeader = Arrays.copyOfRange(gif, 0, 6);
        final byte[] expectedFileHeader = new byte[]{'G', 'I', 'F', '8', '9', 'a'};
        assertArrayEquals(expectedFileHeader, actualFileHeader);
    }
}

(note: based on https://github.com/FasterXML/jackson-dataformat-yaml/issues/62)

If possible, it would be great to be able to support anchors for simple scalar values.

The original test case to show this would be:

public class YamlParserTest {
    private ObjectMapper objectMapper = new ObjectMapper(new YAMLFactory());

    public static class Test1 {
        public Long value1;
        public Long value2;
    }

    @Test
    public void test() throws IOException {
        String yaml =
        "value1: &test_1 123\n" +
        "value2: *test_1";
        Test1 test1 = objectMapper.readValue(yaml, Test1.class);
        assertEquals(123L, test1.getValue1().longValue());
        assertEquals(123L, test1.getValue2().longValue());
    }
}

(note: moved from https://github.com/FasterXML/jackson-dataformat-properties/issues/5)

Comma-separated lists are quite common in property or *.ini style configuration files. The only way I found to generate/parse these with JavaPropsMapper is to annotate a custom value class with @JsonCreator and @JsonValue, but this also affects other data formats that do have a proper list representation. If I want to convert between *.ini and json/yaml/xml, it gets quite messy.

I would like to propose (and help with) the following feature: Arrays or lists of primitive types can be configured to be serialized and deserialized as CSV (Split at configurable delimiter, strip whitespace around delimiter, do not split or remove whitespace within quoted strings, remove quotes from quoted strings).

This would make *.ini style configuration a real alternative to json/yaml/xml for some use cases.

(from FasterXML/jackson-dataformat-csv#118 by @balbusm)

CsvSchema schema = CsvSchema.emptySchema().withHeader();
FileReader fileReader = new FileReader(file);
MappingIterator<Map<String,String>> it = mapper.readerFor(Map.class)
  .with(schema)
  .readValues(fileReader);
while (it.hasNext()) {
  try {
    Map<String,String> rowAsMap = it.next();
    System.out.println(rowAsMap);
  } catch (Exception e) {
    System.out.println(e);
  }
}

Example input:

1,Head2,Head3,Head4,"Head5
2,Head2,Head3,Head4,Head5
3,Head2,Head3,Head4,Head5

If I have a CSV with a header row, the header names seem to be trimmed. For example:

"Padded ,Unpadded\nOne,Two"

The above will produce a map with the key "Padded" instead of "Padded ".

I'm looking for a feature similar to Spring, HandleBar, etc that would allow me to create a yaml (or any format) file that includes a reference to system variables during read. Having to "pre-process" a file using templating tools before launching an application is cumbersome and error-prone vs having a standard by which variables can be defined by the user (vs the developer) when crafting the files and then merged into the content delivered to the application. I realize the request is "extending" the yaml, json, etc standard but the point of libraries such as Jackson is to help establish new standards.

example:

system:
   host: ${HOSTNAME:defaulthost}
   port: ${PORT:8080}

if you wanted to allow the control of the source (System ENV vs Java Property) you could use a syntax as follows.

system:
   port: ${ENV:PORT:8080}
   port: ${PROPERTY:PORT:8080}

(from FasterXML/jackson-dataformat-csv#145 by @robmoore)

We have a case where we're reading documents produced by a third-party which contains trailing empty lines. It appears that the reader attempts to create empty POJO entries for these lines rather than ignoring them.

A work around for this is to simply read each value individually rather than using readValues(). Nevertheless, for the streaming case it would be nice to be able to use readValues().

Is it please possible to emit comment lines with the keys? I could imagine this being done with an annotation for the respective class member. Also, adding whitespace, e.g. empty line would be a nice feature as well.

If this is not currently possible, could anyone please point me to the code where this magic would be happening so that I can add this feature?

Many thanks! :)

(from https://github.com/FasterXML/jackson-dataformat-yaml/issues/52 by @cruisercoder)

Jackson provides some control over the generated YAML, such as MINIMIZE_QUOTES. It it would also be nice also have control over the block/flow style at any point during the generation. This can be implemented by controlling the value of the style boolean that is passed to the snakeyaml Emitter.emit call.

Hi, is there a way to get this:

    kibana_hide_apps: ["readonlyrest_kbn", "timelion"]

instead of this:

    kibana_hide_apps:
    - "readonlyrest_kbn"
    - "timelion"

(moved from: https://github.com/FasterXML/jackson-dataformat-yaml/issues/24)

While Native Object Id handling works just fine in general (as well as when buffering with TokenBuffer), it does not work when reading contents as trees (JsonNode), then converting.

We're thinking of writing a jackson-dataformats-rdf that uses http://rdf4j.org, akin to jackson-dataformat-yaml which uses SnakeYAML.

What are the odds that it would be accepted? (If not, that's OK, instead of forking jackson-dataformats-text we'll just create our own repo.)

Thanks!

(from FasterXML/jackson-dataformat-csv#141 by @jefferyyuan)

It's common that we use multiple-characters delimiter to separate columns in CVS: such as: [|]
Sometimes, we have no choice as third-party defines the csv format like that.

Sample data:

FirstName[|]LastName[|]Address
John[|]Smith[|]123 Main St.

FasterXML/jackson-dataformat-csv#92 supports to use multiple characters as array element separator, It would be great if same can be done for withColumnSeparator.

I did some google search: seems there is no java library that supports use multiple-characters as the column separator.

But the C# CsvHelper does
https://github.com/JoshClose/CsvHelper

• Jackson-dataformat-csv version 2.8.8

When using csvMapper.readValues(file), a new FileinputStream is created, but if an error occur during the parsing, this inputStream is never closed.

Java code:

@JsonIdentityInfo(generator=ObjectIdGenerators.IntSequenceGenerator.class)
class Student {
	private String name;
	private Integer age;

	@JsonCreator
        public Student(@JsonProperty("age") int age, @JsonProperty(value = "name") String name){
		this.name = name;
		this.age = age;
	}

        public String toString(){
            return "Student [ name: "+name+", age: "+ age+ " ]";
    }
}

public static void main(String args[]){
        try {
            YAMLMapper mapper = new YAMLMapper(new YAMLFactory());

            Student student = mapper.readValue(new File("student_yaml.yml"), Student.class);

	        System.out.println(student);
        }
        catch (Exception e) {
	        e.printStackTrace();
        }
    }

student_yaml.yml:

---
&1 name: "billy"
age: 5

Runtime Error:

com.fasterxml.jackson.databind.JsonMappingException: No Object Id found for an instance of Student, to assign to property '@id'
 at [Source: student_yaml.yml; line: 3, column: 7]
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
	at com.fasterxml.jackson.databind.DeserializationContext.reportUnresolvedObjectId(DeserializationContext.java:1259)
	at com.fasterxml.jackson.databind.deser.impl.PropertyValueBuffer.handleIdValue(PropertyValueBuffer.java:241)
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:143)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:471)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1198)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1168)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:146)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3798)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2740)
	at JacksonTester.main(JacksonTester.java:50)

So for this, when attempting to use the YAML anchor notation, it doesn't seem to be recognized. Everything works fine when I make Student stop using JsonCreator (and make the fields public, of course), or change student_yaml.yml to

---
'@id': 1
name: "billy"
age: 5

YAMLGenerator does not quote empty strings if MINIMIZE_QUOTES is enabled.

The YAML 1.2 spec says that plain (unquoted) scalars cannot be empty i.e. an empty string must always be quoted.

      YAMLFactory f = new YAMLFactory();
      f.configure(YAMLGenerator.Feature.MINIMIZE_QUOTES, true);

      YAMLMapper mapper = new YAMLMapper(f);

      Map<String, Object> content = new HashMap<>();
      content.put("key", "");
      String yaml = mapper.writeValueAsString(content).trim();

      assertEquals("---\nkey: \"\"", yaml); // fails - actual output is "---\nkey:"

(from FasterXML/jackson-dataformat-csv#35 @ffbit)

It'd be good to have the jackson-jaxrs-cvs-provider derived from this artifact.

It would be great if there was a way to specify a non-string property value when mapping to the generic ObjectNode type. For example:

Properties props = new Properties();

props.setProperty("booleanExample", "true");
props.setProperty("integerExample", "4");
props.setProperty("stringExample", "some string");
    
ObjectNode root = new JavaPropsMapper().readValue(props, ObjectNode.class);

System.out.println(new ObjectMapper().writeValueAsString(root));

This code outputs

{"integerExample":"4","booleanExample":"true","stringExample":"some string"}

But it would be great if there was some configuration or way to escape property values such that the output inferred integer or boolean types from the values themselves:

{"integerExample":4,"booleanExample":true,"stringExample":"some string"}

Uber-project for (some) standard Jackson textua format backends: csv, properties, yaml (xml to be added in future)

That's all. (Well, I'd write the future.)

In RFC-4180 it is said that CSV implementations may choose to include a line break at the end of the file or not.

It could be nice to add an option to choose. For instance something like this :

schema.withEndingLineSeparator();
schema.withoutEndingLineSeparator();

Hi,

There is a use case where we know the CSV column names but not the order.
It would be great if there was a way to map headers in the CSV to POJO properties.

Something like:

Map<String, String> csvHeaders = new HashMap<String, String>();
csvHeaders.put("headerX", "propertyX");
csvHeaders.put("headerY", "propertyY");
CsvSchema bootstrapSchema = CsvSchema.emptySchema().withHeaderMap(csvHeaders);
...

Possibly with an option to either ignore extra columns or throw an error if there are missing columns.

Does anyone know if this is possible/likely?

Thanks for a great library.

I am trying to convert properties to Json and here is what I am seeing:

//String properties = "ƛ Ɯ Ɲ ƞ Ɵ Ơ ơ Ƣ = ƛ Ɯ Ɲ ƞ Ɵ Ơ ơ Ƣ";
String properties = "\u019B \u019C \u019D \u019E \u019F \u01A0 \u01A1 \u01A2 = \u019B \u019C \u019D \u019E \u019F \u01A0 \u01A1 \u01A2";
JsonNode node = new JavaPropsMapper().readValue(properties, JsonNode.class);
System.out.println(node.toString());

Expected output:
{"ƛ Ɯ Ɲ ƞ Ɵ Ơ ơ Ƣ" : "ƛ Ɯ Ɲ ƞ Ɵ Ơ ơ Ƣ"}

Observed output:
{"ƛ":"Ɯ Ɲ ƞ Ɵ Ơ ơ Ƣ = ƛ Ɯ Ɲ ƞ Ɵ Ơ ơ Ƣ"}

I am not sure if I am not using the library correctly or if I need to do more pre-processing of that String (maybe escape it or something?), or if its something else. Any help is appreciated.

(see https://github.com/FasterXML/jackson-dataformat-yaml/issues/67 reported by @ondrej-kvasnovsky for discussion)

Looks like SnakeYAML (1.18 and below) have an issue with indentation, causing generation of invalid YAML in cases where array values are not scalars.

I would like to escape the QuoteChar when it is doubled.

1, " the next word is ""quoted"""

The following snippet seems to give the expected result:

CsvSchema schema = CsvSchema.emptySchema()
.withQuoteChar('"')
//if we have a quote char, allow to escape it (see rfc4180)
.withEscapeChar('"')
...

But, the csv-compatibility section states :

If enabled, character immediately followed will be used as-is, except for a small set of "well-known" escapes (\n, \r, \t, \0)

Hi when trying to create an instance of CsvMapper I am getting following issue

CsvMapper mapper = new CsvMapper();

Error:
java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.dataformat.csv.CsvFactory at com.fasterxml.jackson.dataformat.csv.CsvMapper.<init>(CsvMapper.java:39) at com.bluetrack.report.services.LeadReportService.csvWriter(LeadReportService.java:154) at com.bluetrack.report.services.LeadReportService.generateScheduledLeadReport(LeadReportService.java:140) at com.bluetrack.report.services.LeadReportService$$FastClassBySpringCGLIB$$418c5dc5.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) at com.bluetrack.report.services.LeadReportService$$EnhancerBySpringCGLIB$$7c047677.generateScheduledLeadReport(<generated>) at com.bluetrackmedia.scheduledreporting.ReportGenerator.generateLeadReport(ReportGenerator.java:56) at com.bluetrackmedia.scheduledreporting.ReportGenerator.generateReport(ReportGenerator.java:42) at com.bluetrackmedia.scheduledreporting.ReportGenerator$$FastClassBySpringCGLIB$$72f6c360.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) at com.bluetrackmedia.scheduledreporting.ReportGenerator$$EnhancerBySpringCGLIB$$7f6c3bb2.generateReport(<generated>) at com.bluetrackmedia.scheduledreporting.scheduledtask.checkForPendingScheduledReports(scheduledtask.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)

My Pom.xml:
<dependency> <groupId>com.fasterxml.jackson.dataformat</groupId> <artifactId>jackson-dataformat-csv</artifactId> <version>2.8.8</version> </dependency>

When I see maven dependencies, I am able to see all dependent jars of jackson.

Thanks,
Pavan

When adding Json sub types management to a class to deserialize, the references to anchors are unresolved.

The following sample is derived from ObjectIdTest.

Here is new Node class (with type attribute and JsonSubTypes annotations) :

	@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, 
			include = As.PROPERTY, 
			property = "type", 
			defaultImpl = NodeWithStringId.class)
	@JsonSubTypes({ 
		@JsonSubTypes.Type(value = SubNodeWithStringId.class, name= "subnode"),
		@JsonSubTypes.Type(value = NodeWithStringId.class, name = "node") 
	})
    @JsonIdentityInfo(generator=PrefixIdGenerator.class)
    static class NodeWithStringId
    {
        public String name;
        public String type;

        public NodeWithStringId next;
        
        public NodeWithStringId() { }
        public NodeWithStringId(String name) {
            this.name = name;
        }
    }	

And a related Node subtype :

    static class SubNodeWithStringId extends NodeWithStringId
    {
    }

Now the test is :

	String yml =
	            "---\n"
                    +"&id1 name: \"first\"\n"
                    +"type: \"node\"\n"
                    +"next:\n"
                    +"  &id2 name: \"second\"\n"
                    +"  next: *id1"
	        ;


	NodeWithStringId node  = mapper.readValue(yml, NodeWithStringId.class);


        assertNotNull(node);
        assertEquals("first", node.name);
        assertNotNull(node.next);
        assertEquals("second", node.next.name);
        assertNotNull(node.next.next);
        assertSame(node, node.next.next);	

The test fails with :

com.fasterxml.jackson.databind.deser.UnresolvedForwardReference: Unresolved forward references for: 
 at [Source: java.io.StringReader@3302035b; line: 6, column: 13]Object id [id1]

If the sub type annotations are removed, the test is ok

I have a yaml file that I want to read into Map<String, String>:

    events:
      key1: val1
      key2: val2
      key3: val3

This piece of code is working fine.

public class EventGenerator {    
     private Map<String, String> events;
   
     public Map<String, String> getEvents() {
       return this.events;
     }
   
     public void setEvents() {
       ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
           try {
               FileInputStream fis = new FileInputStream("file.yaml");
               EventGenerator eventGenerator = mapper.readValue(fis, EventGenerator.class);
               fis.close();
               System.out.print(eventGenerator.getEvents());
           } catch (IOException e) {
               e.printStackTrace();
           }
   
     }
   
   }

But when I pass path of yaml file as a String (and/or File) argument

    public void setEvents(String filePath) {
    // same code
    FileInputStream fis = new FileInputStream(filePath)
    // same code
    }

I'm getting an exception

jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String

Also, when I override the setEvents() method and have both setEvents() and setEvents(String filePath) then neither are working and both are giving the above exception.

UPD (sorry, forgot to add pom)
jackson pom dependencies:

        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-yaml</artifactId>
            <version>2.9.1</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.9.1</version>
        </dependency>

Background:
We're in need for a fast and feature-rich CSV Parser and have zeroed-in on using jackson-dataformats-text library. We will be passing the 'Schema' object as an input (which is populated from a schema-file we receive from an upstream system). For each field, we will provide a 'Name' and 'Data Type'. Additionally, we would like to provide an input 'isRequired' so if a field is marked isRequired=true but there is no data in that field in the incoming CSV file, we need to raise an exception.

Requirement:
So far, we could not find support for 'isRequired' check. Please can you advise if this is already present or if not, could this be taken as a new requirement to add the feature in an upcoming release ?

Thanks.

(from: https://github.com/FasterXML/jackson-dataformat-yaml/issues/61 by @andrey-polyakov)

I need keys in map structures to be checked for duplicates and rejected if found. How to enable this, of course, if it is implemented?

To be specific, I want to avoid overwriting of values in java.util.Map.

1. Wiki link broken in https://github.com/FasterXML/jackson-dataformats-text/tree/master/csv
2. Wiki link redirects to the same page in https://github.com/FasterXML/jackson-dataformats-text

YAML binary data http://yaml.org/type/binary.html

This code

  public void yamlBinaryTest() throws IOException {
    StringWriter out = new StringWriter();
    YAMLFactory f = new YAMLFactory();
    f.disable(Feature.WRITE_DOC_START_MARKER);
    f.enable(Feature.MINIMIZE_QUOTES);
    YAMLGenerator gen = f.createGenerator(out);
    gen.writeStartObject();
    gen.writeFieldName("files");
    gen.writeStartArray();
    for(int i=0; i<=2; i++) {
      gen.writeStartObject();
      gen.writeStringField("path", "/tmp/"+i+".txt");
      byte[] data = "foobar".getBytes("UTF-8");
      gen.writeFieldName("content");
      gen.writeBinary(Base64Variants.MIME, data, 0, data.length);
      gen.writeEndObject();
    }
    gen.writeEndArray();
    gen.writeEndObject();
    gen.close();
    System.out.println(out);
  }

Returns

files:
- path: /tmp/0.txt
  content: "Zm9vYmFy"
- path: /tmp/1.txt
  content: "Zm9vYmFy"
- path: /tmp/2.txt
  content: "Zm9vYmFy"

The expected result

files:
- path: /tmp/0.txt
  content: !!binary | 
    Zm9vYmFy
- path: /tmp/1.txt
  content: !!binary | 
    Zm9vYmFy
- path: /tmp/2.txt
  content: !!binary | 
    Zm9vYmFy

The YAMLGenerator does not support for binary literal.
SnakeYAML(actual low-level decoder/encoder) is having this feature https://bitbucket.org/asomov/snakeyaml/src/190f378138123971777405fbabeab704724b1860/src/main/java/org/yaml/snakeyaml/representer/SafeRepresenter.java?at=default&fileviewer=file-view-default#SafeRepresenter.java-430

In documentation it is written that we can change Linefeed character. But when I try with different Linefeed character, it is not parsing correctly. Below is test case

    void testWithDifferentLinefeeds(boolean useBytes) throws Exception {

        String CSV = "data11,data12,data13|data21,data22,data23";

        CsvMapper mapper = new CsvMapper();
        mapper.disable(CsvParser.Feature.WRAP_AS_ARRAY);

        CsvParser csvParser = mapper.getFactory().createParser(CSV);

        CsvSchema csvSchema = CsvSchema.builder()
                .setLineSeparator("|")
                .setColumnSeparator(',')
                .build();

        csvParser.setSchema(csvSchema);

        MappingIterator<Object[]> mappingIterator = mapper.readerFor(Object[].class).readValues(csvParser);

        assertTrue(mappingIterator.hasNext());
        Object[] record = mappingIterator.nextValue();
        assertNotNull(record);
        assertEquals("data11", record[0]);
        assertEquals("data12", record[1]);
        assertEquals("data13", record[2]);
        assertEquals(3, record.length);

        assertTrue(mappingIterator.hasNext());
        record = mappingIterator.nextValue();
        assertNotNull(record);
        assertEquals("data21", record[0]);
        assertEquals("data22", record[1]);
        assertEquals("data23", record[2]);
        assertEquals(3, record.length);

        assertFalse(mappingIterator.hasNext());
        mappingIterator.close();
    }

But same test case is working when I change the Linefeed char to "\n" in my data. e.g.,

    void testWithDifferentLinefeeds(boolean useBytes) throws Exception {

        String CSV = "data11,data12,data13\ndata21,data22,data23";
        CsvMapper mapper = new CsvMapper();
        mapper.disable(CsvParser.Feature.WRAP_AS_ARRAY);
        CsvParser csvParser = mapper.getFactory().createParser(CSV);
        CsvSchema csvSchema = CsvSchema.builder()
                .setLineSeparator("\n")
                .setColumnSeparator(',')
                .build();

        csvParser.setSchema(csvSchema);

        MappingIterator<Object[]> mappingIterator = mapper.readerFor(Object[].class).readValues(csvParser);

        assertTrue(mappingIterator.hasNext());
        Object[] record = mappingIterator.nextValue();
        assertNotNull(record);
        assertEquals("data11", record[0]);
        assertEquals("data12", record[1]);
        assertEquals("data13", record[2]);
        assertEquals(3, record.length);

        assertTrue(mappingIterator.hasNext());
        record = mappingIterator.nextValue();
        assertNotNull(record);
        assertEquals("data21", record[0]);
        assertEquals("data22", record[1]);
        assertEquals("data23", record[2]);
        assertEquals(3, record.length);

        assertFalse(mappingIterator.hasNext());
        mappingIterator.close();
    }

It means that currently Jackson Csv parser only supports "\n" as a line delimiter. right?

(from: https://github.com/FasterXML/jackson-dataformat-yaml/issues/37 by @jvantuyl)

The use of BigInteger for really big integers introduces an interesting behavior. For a very large zero-prefixed number, it won't see a leading zero as indicating octal (because BigInteger's constructor doesn't mirror Integer.parseInt or Long.parseLong).

In the long run, also doesn't support the 0o644 notation from YAML 1.2, which I'm starting to see floating about.

CSV data may have a syntax error. The user may wich to continue with the next record. When using CSV parsing with
import com.fasterxml.jackson.dataformat.csv.CsvMapper; ... it = mapper.readerFor(Map.class).with(schema).readValues(csvFile);
it doesn't appear to be able to continue with the next record.

how to append data in CSV (JSON)

List list = new ArrayList<>();
list.add(populateLoginR());

  CsvMapper mapper = new CsvMapper();
  mapper.configure(JsonGenerator.Feature.IGNORE_UNKNOWN, true);
   CsvSchema schema = CsvSchema.builder().addColumn("home_page").addColumn("message").addColumn("full_name").build();
   ObjectWriter writer = mapper.writerFor(LoginR.class).with(schema);

  File tempFile = new File("G://green pyramid//output.csv");
    writer.writeValues(tempFile).writeAll(list);
    System.out.println("Success");
 }

  private LoginR populateLoginR() {
	  LoginR o1 = new LoginR();
  o1.setHomePage(lclasscatch.getHomePage());
  o1.setMessage(lclasscatch.getMessage());
  o1.setFullName(lclasscatch.getFullName());
  

  return o1;
 }