Code Monkey home page Code Monkey logo

Comments (3)

mjakubowski84 avatar mjakubowski84 commented on June 16, 2024

This looks strange, indeed. It looks like a bug.

from parquet4s.

mjakubowski84 avatar mjakubowski84 commented on June 16, 2024

Hi @struong

Given proto

syntax = "proto3";

option java_package = "com.github.mjakubowski84";

message LoginEvent {
  enum EventType {
    LOGIN_EVENT_UNSPECIFIED = 0;
    LOGIN_SUCCESSFUL = 1;
  }

  int64 timestamp = 1;
  EventType eventType = 2;

}

and Scala code:

package com.github.mjakubowski84

import com.github.mjakubowski84.parquet4s._
import ScalaPBImplicits._

import java.time.Instant

object PBTest extends App {

  val outFile = InMemoryOutputFile(initBufferSize = 4800)
  ParquetWriter.of[LoginEvent].writeAndClose(
    file = outFile,
    data = Seq(
      LoginEvent(
        timestamp = Instant.now().toEpochMilli,
        eventType = LoginEvent.EventType.LOGIN_SUCCESSFUL
      )
    )
  )

  val inFile = InMemoryInputFile.fromBytes(outFile.take())

  val protoResult = ParquetReader.as[LoginEvent].read(inFile)
  // prints LoginEvent(1701085450638,LOGIN_SUCCESSFUL,UnknownFieldSet(Map()))
  println(protoResult.head)

  val genericResult = ParquetReader.generic.read(inFile)
  // prints Some(LOGIN_SUCCESSFUL)
  println(genericResult.head.get[String]("eventType", ValueCodecConfiguration.Default))

}

The created record is RowParquetRecord(timestamp=LongValue(1701085521641),eventType=BinaryValue(Binary{16 constant bytes, [76, 79, 71, 73, 78, 95, 83, 85, 67, 67, 69, 83, 83, 70, 85, 76]})) with eventType being a binary encoding a String value of the enum.

Everything looks good to me with enums and deserialisation.

It seems that you have a bug somewhere else.

from parquet4s.

struong avatar struong commented on June 16, 2024

@mjakubowski84 thank you for your prompt reply!

I believe it's an issue with my VSCode plugin to read parquet files, I used an online reader and indeed the rows look good. Thank you for investigating!

from parquet4s.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.