zamblauskas / scala-csv-parser Goto Github PK

View Code? Open in Web Editor NEW

25.0 25.0 5.0 58 KB

CSV parser library.

License: MIT License

Scala 100.00%

csv parsing scala

scala-csv-parser's People

Contributors

Stargazers

Watchers

Forkers

idctim luis3m lightningpayments isgasho raboof

scala-csv-parser's Issues

Code generators for `ApplicativeBuilderX`

At the moment we have up to ApplicativeBuilder6 written by hand.
Adding additional is error prone and time consuming.
Instead rewrite using code generators.

Parser inside polymorphic methods.

Can I use Parser inside a polymorphic method like this?

  def readCsv [A] (str: String) : Queue[A] = {
    if (!RFile(str).exists) {
      logger.info(s"File not exists: $str")
      return Queue.empty[A]
    }
    val csv = readFileContent(str)
    Parser.parse[A](csv) match {
      case Left(failure: Failure) => {
        logger.error(s"Syntax error: ${failure.message}")
        Queue.empty[A]
      }
      case Right(list: Seq[A]) => list.to(Queue)
    }
  }

If I try to execute it providing A as some sealed case class, then I got error on compile time:

Couldn't find sh.srv.docsp.services.core.FileHelper.A constructor.
Error while trying to generate zamblauskas.csv.parser.ColumnReads[sh.srv.docsp.services.core.FileHelper.A].
Either fix the error or provide zamblauskas.csv.parser.ColumnReads[sh.srv.docsp.services.core.FileHelper.A].

Bintray no longer available

@zamblauskas Hi, with Bintray down permanently there's no artifact available to use this library.

As far as I know, bintray going to switch off the repository on the 1st May (https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/). I would be nice to publish artifacts somewhere else (possible sonatype).

`Alternative` for `ColumnReads`

Need to implement Alternative for ColumnReads to enable picking between multiple possible columns.

We can't use it for csv without column name?!

I like the simplicity of this lib, but there's no column name on my csv, so I'd like to use it without column name.
I solve the issue by adding the header in a string format in my code, but I think there should be a more elegant solution!

Scala 3 support

It would be great to have Scala 3 support. Unfortunately it'd mean rewriting the ReadsMacro - splitting the scala2 and scala3 implementation of that macro is easy enough with https://docs.scala-lang.org/scala3/guides/migration/tutorial-macro-cross-building.html , but I'm not familiar enough with Scala 3 macro's yet to port it. My half-baked start at an attempt is at https://github.com/raboof/scala-csv-parser/tree/scala3 ;)

Integration with `Akka Streams`

Could be split into:

modularizing the code into core (domain/dsl/functional stuff) and backend (current parse whole string at once)
a new backend which would produce an Akka Flow

Issue with email

I have one CSV file that contains one column with email address.

I get the following error during parsing :

myCsv.csv:3:20 expected (Semis | &"}" | end-of-input)
xx;xx;[email protected];xxxx
                   ^

Is there one tip to enable to parse column with email address

No implicit support for Long

When trying to convert .csv data from a stream or file that has made it to a comma and new line delimited string into a case class used by an application, it would be good (in certain circumstances, essential) to have an expanded list of "native" Scala types. Presently, there is support for the following types:

String
Int
Float
Double
Boolean

I propose adding Long to this list and have created a Pull Request for this.

Why is this essential? When dealing with AWS S3 Objects, the document info needed to fully describe an entry includes a Long type for the content-length. The other types that might be considered important to support are BigDecimal and BigInt, but they are more involved since they do not have implicit converters in Scala's StringOps class. BigDecimal is used for money a lot in many projects including Squants, and it is also used extensively in default situations for Slick and Postgres. Big Int could come along for the ride. I will work on a Pull Request for the Bigs, but Long is terribly time sensitive, so if the Long pull request could be approved very soon, that would be most appreciated!

Cache Travis ivy dependencies

Identical to how it's done in https://github.com/akka/alpakka/blob/master/.travis.yml

Setup `sbt-doctest` to test examples in the README

https://github.com/tkawachi/sbt-doctest

scalaVersion defined twice in build.sbt and .travis.yml

Ideally would like it defined in one place.
Check for a solution.

Setup automatic publish to Bintray on every commit to master

Column limit

There seems to be a limit on how many columns can be parsed which I don't see mentioned in documentation.

My case class:

private case class DeckLevelDescriptorFromCsv(
                                               levelId: Int,
                                               isHuman: Boolean,
                                               playerIndexInTeam: Int,
                                               avatarId: Int,
                                               frameId: Int,
                                               card01Id: Int,
                                               card01Count: Int,
                                               card02Id: Int,
                                               card02Count: Int,
                                               card03Id: Int,
                                               card03Count: Int,
                                               card04Id: Int,
                                               card04Count: Int,
                                               card05Id: Int,
                                               card05Count: Int,
                                               card06Id: Int,
                                               card06Count: Int,
                                               card07Id: Int,
                                               card07Count: Int,
                                               card08Id: Int,
                                               card08Count: Int,
                                               card09Id: Int,
                                               card09Count: Int,
                                               card10Id: Int,
                                               card10Count: Int,
                                               card11Id: Int,
                                               card11Count: Int,
                                               card12Id: Int,
                                               card12Count: Int,
                                               card13Id: Int,
                                               card13Count: Int,
                                               card14Id: Int,
                                               card14Count: Int,
                                               card15Id: Int,
                                               card15Count: Int,
                                               card16Id: Int,
                                               card16Count: Int,
                                               card17Id: Int,
                                               card17Count: Int,
                                               card18Id: Int,
                                               card18Count: Int,
                                               card19Id: Int,
                                               card19Count: Int,
                                               card20Id: Int,
                                               card20Count: Int
                                             )

reading:

      val parsedDecks = Parser.parse[DeckLevelDescriptorFromCsv](decksFileContent)

error:

error: value and is not a member of zamblauskas.functional.ApplicativeBuilder22[zamblauskas.csv.parser.ColumnReads,Int,Boolean,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int]
[ant:scalac]       val parsedDecks = Parser.parse[DeckLevelDescriptorFromCsv](decksFileContent)
[ant:scalac]                                                                 ^
[ant:scalac] <macro>:1: error: implementation restricts functions to 22 parameters
[ant:scalac] new tk.purplegames.shaeloarena.gui.screens.levelSelection.DeckLevelDescriptorFromCsv(_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_)
[ant:scalac] ^

I have also tried defining ColumnReads:

      implicit val deckRead: ColumnReads[DeckLevelDescriptorFromCsv] = (
        column("levelId").as[Int] and
          column("isHuman").as[Boolean] and
          column("playerIndexInTeam").as[Int] and
          column("avatarId").as[Int] and
          column("frameId").as[Int] and
          column("card01Id").as[Int] and
          column("card01Count").as[Int] and
          column("card02Id").as[Int] and
          column("card02Count").as[Int] and
          column("card03Id").as[Int] and
          column("card03Count").as[Int] and
          column("card04Id").as[Int] and
          column("card04Count").as[Int] and
          column("card05Id").as[Int] and
          column("card05Count").as[Int] and
          column("card06Id").as[Int] and
          column("card06Count").as[Int] and
          column("card07Id").as[Int] and
          column("card07Count").as[Int] and
          column("card08Id").as[Int] and
          column("card08Count").as[Int] and
          column("card09Id").as[Int] and
          column("card09Count").as[Int] and
          column("card10Id").as[Int] and
          column("card10Count").as[Int] and
          column("card11Id").as[Int] and
          column("card11Count").as[Int] and
          column("card12Id").as[Int] and
          column("card12Count").as[Int] and
          column("card13Id").as[Int] and
          column("card13Count").as[Int] and
          column("card14Id").as[Int] and
          column("card14Count").as[Int] and
          column("card15Id").as[Int] and
          column("card15Count").as[Int] and
          column("card16Id").as[Int] and
          column("card16Count").as[Int] and
          column("card17Id").as[Int] and
          column("card17Count").as[Int] and
          column("card18Id").as[Int] and
          column("card18Count").as[Int] and
          column("card19Id").as[Int] and
          column("card19Count").as[Int] and
          column("card20Id").as[Int] and
          column("card20Count").as[Int]
        ) (DeckLevelDescriptorFromCsv)

which resulted in:

error: value and is not a member of zamblauskas.functional.ApplicativeBuilder22[zamblauskas.csv.parser.ColumnReads,Int,Boolean,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int]

I'll probably change the file format (perhaps join card id and count), but I thought it would be good to write it here, so others are aware of this limitation. Case classes are no longer limited by 22, but other things are (I think tuples and functions, not sure though).

Setup `https://github.com/dwijnand/sbt-dynver`

Unused import warning with Scala 2.11

Hi!

I've found that compiling with -Ywarn-unused-import produces really weird warning:

[warn] /home/limansky/projects/myproject/src/main/scala/org/mypackage/MyClass.scala:1:0: Unused import
[warn] package org.mypackage
[warn] ^

It looks like the issue reproduces only with Scala 2.11. I've googled little bit, it seems it related to macros, and play-json faced with same issue (they were able to solve it).