Code Monkey home page Code Monkey logo

datus's Introduction

datus enables you to define a conversion process between two data structures in a fluent functional API. Define conditional processing, do some business relevant work or just pass values from a to b - just about everything is possible for both mutable and immutable data structures.

Replace your conversion factories/helpers with datus and focus on what to map, not how: define your conversion, eliminate the need to test the conversion aspects of your application and focus on what really matters - your project.

Overview

  1. Why use datus?
  2. Examples
  3. Sample projects
  4. User guide
  5. Dependency information (Maven etc.)
  6. Supported JVM languages
  7. Development principles
  8. Supporting datus development

Why use datus?

Using datus has the following benefits:

  • separation of concerns: write your mapping in datus while using the business logic of other components (no more monolithic classes that do both)
  • reducing dependencies: enable your businesslogic to operate on parts of a data structure instead of depending on the whole object (e.g. upper casing a persons name in a .map-step)
  • cleaner abstractions: programming against an interface instead of concrete classes
  • declarative/functional-programming approach: focus on what to map, not how to do it
  • simplicity: compared to Lombok and Apache MapStruct no additional IDE or build system plugins are needed
  • explicitness: no black magic - you define what to map and how (compile-time checked), not naming conventions, annotations or heuristics
  • low coupling: leave your POJO/data classes 'as is' - no need for annotations or any other modifications
  • less classes: no more 'dumb'/businesslogic less Factory-classes that you have to unit test
  • cross-cutting concerns: easily add logging (or other cross-cutting concerns) via spy or process(see below for more information about the full API)
  • rich mapping API: define the mapping process from A -> B and get Collection<A> -> Collection<B>, Collection<A> -> Map<A, B> and more for free
  • focus on meaningful unit tests: no need to unit test trivial but necessary logic (e.g. null checking, which once fixed won't be a problem at the given location again)
  • clarity: (subjectively) more self-documenting code when using datus mapping definitions
  • no reflection or code generation: datus implementation is just plain java, use GraalVM Native Image or any JVM out there - datus will always run out of the box without any further configuration or limitations

Examples

The following examples outline the general usage of both the immutable and mutable API of datus. Please refer to the USAGE.md for an extensive guide on datus that includes a fictional, more complex scenario with changing requirements.

Immutable object API example

class Person {
    //getters + constructor omitted for brevity
    private final String firstName;
    private final String lastName;
   
}

class PersonDTO {
    //getters omitted for brevity
    private final String firstName;
    private final String lastName;
    
    public PersonDTO(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }
}

//define a building process for each constructor parameter, step by step
//the immutable API defines constructor parameters in their declaration order
Mapper<Person, PersonDTO> mapper = Datus.forTypes(Person.class, PersonDTO.class).immutable(PersonDTO::new)
    .from(Person::getFirstName).to(ConstructorParameter::bind)
    .from(Person::getLastName).nullsafe()
        .given(String::isEmpty, "fallback").orElse(ln -> ln.toUpperCase())
        .to(ConstructorParameter::bind)
    .build();
    
Person person = new Person();
person.setFirstName("firstName");
person.setLastName(null);
PersonDTO personDto = mapper.convert(person);
/*
    personDto = PersonDTO [
        firstName = "firstName",
        lastName = null
    ]
*/
person.setLastName("");
personDto = mapper.convert(person);
/*
    personDto = PersonDTO [
        firstName = "firstName",
        lastName = "fallback"
    ]
*/
person.setLastName("lastName");
personDto = mapper.convert(person);
/*
    personDto = PersonDTO [
        firstName = "firstName",
        lastName = "LASTNAME"
    ]
*/

Mutable object API example

class Person {
    //getters + setters omitted for brevity
    private String firstName;
    private String lastName;
}

class PersonDTO {
    //getters + setters + empty constructor omitted for brevity
    private String firstName;
    private String lastName;
}

//the mutable API defines a mapping process by multiple getter-setter steps
Mapper<Person, PersonDTO> mapper = Datus.forTypes(Person.class, PersonDTO.class).mutable(PersonDTO::new)
    .from(Person::getFirstName).into(PersonDTO.setFirstName)
    .from(Person::getLastName).nullsafe()
        .given(String::isEmpty, "fallback").orElse(ln -> ln.toUpperCase())
        .into(PersonDTO::setLastName)
    .from(/*...*/).into(/*...*/)
    .build();
    
Person person = new Person();
person.setFirstName("firstName");
person.setLastName(null);
PersonDTO personDto = mapper.convert(person);
/*
    personDto = PersonDTO [
        firstName = "firstName",
        lastName = null
    ]
*/
person.setLastName("");
personDto = mapper.convert(person);
/*
    personDto = PersonDTO [
        firstName = "firstName",
        lastName = "fallback"
    ]
*/
person.setLastName("lastName");
personDto = mapper.convert(person);
/*
    personDto = PersonDTO [
        firstName = "firstName",
        lastName = "LASTNAME"
    ]
*/

Sample projects

There are two sample projects located in the sample-projects directory that showcase most of datus features in two environments: framework-less and with Spring Boot.

Hop right in and tinker around with datus in a compiling environment!

User guide

Please refer to the USAGE.md for a complete user guide as the readme only serves as a broad overview. The user guide is designed to take at most 15 minutes to get you covered on everything about datus and how to use it in different scenarios.

Dependency information

Maven

<dependency>
  <groupId>com.github.roookeee</groupId>
  <artifactId>datus</artifactId>
  <version>1.5.0</version>
</dependency>

or any other build system via Maven Central

Maven Central

Supported JVM languages

datus currently only supports Java as every other popular statically typed language on the JVM has fundamental issues with datus core workflow. Non-statically typed languages like Clojure are not tested for compatability.

Kotlin issues

Kotlin has a myriad of issues when it comes to the type-inference chain of datus builder-like API which makes 1:1 translations of Java to Kotlin (e.g. immutable class to data class) uncompilable. The encoding of (non-) nullability in Kotlins type system makes using method references with datus nearly impossible when using a nullable type and referencing Java functions from Kotlin (e.g. something simple as Long::parseLong on a Long?).

Scala issues

Scala offers no way to obtain a method reference to a constructor of an object which forces datus users to provide a lambda which delegates to the corresponding constructor. This lambda has to be fully typed as Scalas overload resolution / type-inference cannot handle it otherwise. This makes using the immutable API of datus cumbersome which is especially bad for a language like Scala that is focused around immutability.

The aforementioned issues are not exhaustive and more issues are likely to arise when working around them. Therefor it is discouraged to use datus in any other language than Java right now.

Development principles

This section is about the core principles datus is developed with.

Benchmarking

Every release of datus is monitored for performance regressions by a simple JMH suite that checks the core functionality of datus (simple mapping from a->b). See com.github.roookeee.datus.performance.PerformanceBenchmarkTest for further insight.

Mutation testing

datus uses pitest to secure the quality of all implemented tests and has no surviving mutations outside of Datus helper functions (which only aid type inference and are thus not tested), some constructors of the immutable API that only delegate and are thus not explicitly tested and pitest edge-cases which are not testable / programmatically producible.

Branching

The master branch always matches the latest release of datus while the develop branch houses the next version of datus that is still under development.

Semver

datus follows semantic versioning (see https://semver.org/) starting with the 1.0 release.

Licensing

datus is licensed under The MIT License (MIT)

Copyright (c) 2019-2023 Nico Heller

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Supporting datus development

Like datus ? Consider buying me a coffee :)

Buy Me A Coffee

datus's People

Contributors

roookeee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

datus's Issues

Add tests for other JVM languages

Currently datus cannot be used in Kotlin as outlined here - Kotlins compiler can't infer the generic type parameters as accurately as Java.

We should add some tests for Scala and Kotlin (once the inference is more capable) to ensure datus really works with these languages.

Add a nullsafe mode

As much as datus is about explicitness: traversing nested objects which can be null is quite cumbersome as it stands.

We have to find a way to avoid something like this:

.from(Order::getCustomer)
    .given(Objects::nonNull, Customer::getBillingAddress).orElseNull()
    .given(Objects::nonNull, Address::getCity).orElseNull()
    .into(OrderDTO::setBillingCity)

I personally would like to veto any variant that would make every mapping step of a mapping definition nullsafe - it's too implicit.

Concept: add the ability to map to multiple outputs

Say you have a Customer object as a field inside your Order object and want to extract both the cutomers name and adress to two separate fields of the output object. With datus you have to do getCustomer() twice which is cluttered and quite cumbersome.

See this benchmark suite with the following manual code:

public final class ManualMapper implements OrderMapper {

    @Override
    public OrderDTO map(Order source) {
        OrderDTO target = new OrderDTO();
        Customer customer = source.getCustomer();
        if (customer != null) {
            target.setCustomerName(customer.getName());
            if (customer.getBillingAddress() != null) {
                target.setBillingCity(customer.getBillingAddress().getCity());
                target.setBillingStreetAddress(customer.getBillingAddress().getStreet());
            }
            if (customer.getShippingAddress() != null) {
                target.setShippingCity(customer.getShippingAddress().getCity());
                target.setShippingStreetAddress(customer.getShippingAddress().getStreet());
            }
        }
        if (source.getProducts() != null) {
            List<ProductDTO> targetProducts = new ArrayList<ProductDTO>(source.getProducts().size());
            for (Product product : source.getProducts()) {
                targetProducts.add(new ProductDTO(product.getName()));
            }
            target.setProducts(targetProducts);
        }
        return target;
    }

}

which currently looks like this:

    private static final Mapper<Product, ProductDTO> productMapper = Datus.forTypes(Product.class, ProductDTO.class)
            .immutable((String name) -> new ProductDTO(name))
            .from(Product::getName).to(ConstructorParameter::bind)
            .build();


    private static final Mapper<Order, OrderDTO> orderMapper = Datus.forTypes(Order.class, OrderDTO.class)
            .mutable(OrderDTO::new)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getName).orElseNull()
                .into(OrderDTO::setCustomerName)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getBillingAddress).orElseNull()
                .given(Objects::nonNull, Address::getCity).orElseNull()
                .into(OrderDTO::setBillingCity)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getBillingAddress).orElseNull()
                .given(Objects::nonNull, Address::getStreet).orElseNull()
                .into(OrderDTO::setBillingStreetAddress)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getShippingAddress).orElseNull()
                .given(Objects::nonNull, Address::getCity).orElseNull()
                .into(OrderDTO::setShippingCity)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getShippingAddress).orElseNull()
                .given(Objects::nonNull, Address::getStreet).orElseNull()
                .into(OrderDTO::setShippingStreetAddress)
            .from(Order::getProducts)
                .given(Objects::nonNull, productMapper::convert).orElseNull()
                .into(OrderDTO::setProducts)
            .build();

We have to do something about this and null checking is a problem too :/

Conditionally apply a transformation

Suppose I have two DTOs:

public class In {

    public String getAge() {
        return age;
    }

    public void setAge(String age) {
        this.age = age;
    }

    private String age;
}

and

public class Out {

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    private Integer age;
}

And I want to conditionally apply a transformation: I want to map String age only if it is made from digits, otherwise simple don't map it.

The only way I found to do this is:

        In in = new In();
        in.setAge("no");

        Out out = Datus.forTypes(In.class, Out.class)
                       .mutable(Out::new)
                       .from(In::getAge).given(Pattern.compile("\\d+").asPredicate(), Function.identity())
                       .orElseNull().map(x -> x == null ? null : Integer.parseInt(x)).into(Out::setAge)
                       .build()
                       .convert(in);

        Assertions.assertNotNull(out);
        Assertions.assertNull(out.getAge());

which looks weird, at best. Did I miss some documentation and there is a better way? If no, I think I might submit a PR with my idea to handle this; unless you already have something like this in motion.

Reevaluate terminal operations in the immutable API

Goal of this issue is to document any other approaches to the immutable API which offer a better usablity than the current implementation.
The following parts of the immutable API come to mind:

  • ImmutableConstructionStep map vs mapTo
  • ImmutableConstructionStep when has to be followed by a .to(Function.identity())

I am open for suggestions but releasing 1.0 does not necessitate a new immutable API.

EDIT: solution was found, see below

Improve datus documentation

This tickets serves as the central point for anything related to datus documentation that could be improved.

  • What is missing in datus documentation?
  • How could datus strength be shown in a better way?
  • Is there a need for a more complex sample project?
  • Are there parts that are confusing and/or badly worded?

Let me know which parts you would like to see improved :)

source to destination conversion.

Hi,
What if i want to convert a property from source byte to destination string this future is currently supported by map struct.

Examine having multiple inputs for generating one output object

There are numerous scenarios where an output object is constructed out of multiple input objects. datus does not support this concept as it has severe implications:

  • Can't auto generate convert(Collection<In1>, Collection<In2>) as it does not correlate the two collections, we would need a map then, which seems cumbersome
  • Both mutable and immutable API would have to be copy pasted as I don't see a way to do it otherwise

Maybe datus should define Tuple2...X types, these would easily fix this problem by unifying multiple input types into one, though the user has to create them by hand. Great care should be taken when deciding this as I don't want to implement too many concepts that are not directly related to data-mapping.

Feedback: Add a shortcut for .from(Function.identity())

A user of datus has noted that .from(Function.identity() feels clumsy when a mapping step has to combine multiple getters of the input type into one setter / parameter of the output type.

We should consider adding a shortcut. Naming seems to be the hardest part of this task as the implementation is pretty straightforward.

Consider renaming when

When is a reserved keyword in Kotlin which forces Kotlin users to use the when function of the builder APIs with backticks - annoying.

What is PerformanceBenchmarkTest exactly supposed to do?

I am a little confused about PerformanceBenchmarkTest.

Anyway, these are nitpicks, mainly. The idea is that I hoped this test would have some samples against some popular frameworks. I have seen https://github.com/arey/java-object-mapper-benchmark, though. It's just that your results rely on those tests and I am not sure about their correctness (without understanding them, those are meaningless numbers: look how the benchmark presents that "direct" mapping is slower than "map-struct"). I wonder if I should provide a PR against some data mappings libraries inside (or a separate project/child from datus)?

Apply PECS where it is missing

As a library datus should follow PECS. Most parts of datus are already PECS compliant but some functions that were added just before releasing 1.0 don't comply yet (or they have just slipped my attention).

I will categorize this issue as semver-minor because applying PECS is backwards compatible and not being able to use functions/predicates/etc. that would work with PECS is more of a bug than a new feature.

  • Mapper interface

Add a JMH testsuite

datus was just integrated here but the main repository could use some fancy images + benchmark statistics itself. The benchmark should focus on other mapping frameworks that don't use code generation but we should definitely mention those that do and how they compare to datus perf

Implement a sample project that shows all of datus features

The existing documentation (readme.md & usage.md) may be enough but I myself like to play around with code (that compiles out of the box) instead of reading about a library. Let's add a simple example project that highlights datus features.

note: as this won't affect any release related code I don't know how to handle semantic versioning here - as it stands I won't do a new release after finishing the sample project.

  • implement the sample project
  • reference it from the readme / usage files

Exchange the explicit mapper classes through lambdas

datus currently implements the Mapper<A,B> in two distinct classes for the mutable and immutable API. This is unnecessary as the Mapper<A,B> interface is a functional interface. As the mapper implementation classes are package private this is a non-breaking change that reduces the overall code size of datus while also opening up for future JVM optimizations in regards to lambdas. Some local JMH testing also showed minor performance benefits when using lambdas.

Support generic types as input and output types

Currently the Datus class expects two Class<T> parameters to infer the input and output types. These cannot be passed if e.g. T is a List<T> as List<T>.class cannot be created / obtained.

We should add an overload to Datus.forTypes() without any parameters that leaves the type definition to the caller via Datus.<List<Integer>, List<String>>forTypes() (it is ugly indeed, but it works, don't think there is a better way).

Thanks to #33 for noticing this issue ! :)

Add an orElse variant that uses a null value directly

When using orElse(null) the java compiler complains about an ambiguous method call (supplier variant and value variant are equally specific) and some user might prefer orElse(null) over orElse(Function.identity()) when doing a null check.

We should add another orElse variant that uses a null value directly, e.g. orElseNull() - naming should be straight forward.

Clarify Mapper.convertToMap behaviour on key collisions

The JavaDoc does not state what happens when there is a key-collision for a given input collection. We should clarify that the last mapped value is retained in the result map and any other behaviour should be implemented via the conversionStream function which offers all other alternatives through the Collectors.toMap collector

Use of mutation testing in datus - Help needed

Hello there!

My name is Ana. I noted that you use the mutation testing tool in the project.
I am a postdoctoral researcher at the University of Seville (Spain), and my colleagues and I are studying how mutation testing tools are used in practice. With this aim in mind, we have analysed over 3,500 public GitHub repositories using mutation testing tools, including yours! This work has recently been published in a journal paper available at https://link.springer.com/content/pdf/10.1007/s10664-022-10177-8.pdf.

To complete this study, we are asking for your help to understand better how mutation testing is used in practice, please! We would be extremely grateful if you could contribute to this study by answering a brief survey of 21 simple questions (no more than 6 minutes). This is the link to the questionnaire https://forms.gle/FvXNrimWAsJYC1zB9.

Drop me an e-mail if you have any questions or comments ([email protected]). Thank you very much in advance!!

Improve datus performance by reducing megamorhpic call sites

datus makes extensive use of lambda-chaining (.map-steps) and wrapping other lambdas (e.g. to support implicit nullsafeness). The resulting wrapper-functions share their byte-code / method-body with all calls to these functions (multiple .map-steps) which makes them mega-morphic and thus un-inlineable.

This affects the following places:

  • ìnto` in the mutable API
  • map in both APIs
  • nullsafe() in both APIs

I have asked on SO on how to solve this and Holger supplied a solution which improves datus performance in the benchmark suite from roughly 8 million op/s to 23 million op/s.

The following points need to be considered while implementing this feature:

  • either add an automatic-detection if re-loading a class via a new classloader is possible (e.g not in SubstrateVM) or add a feature toggle to disable / enable the feature
  • Reference Holger's answer
  • make sure that the resulting helper classes are not accessible from outside of datus (we don't want to leak such a functionality as it's not the core functionality of datus)
  • profile what kind of impact these new classloader instantiations have

Add a helper class for defining mappers for recursive data structures

I reconsidered adding this functionality to datus as there are quite a few scenarios wherin simple data structures are recursively included in themselfs (parent child relations like categories or products containing more subproduct variants).

There is an utility class outlined in https://github.com/roookeee/datus/blob/master/USAGE.md#advanced-usage--faq to fix this problem. Implementing the given class once (+ unit tests) is far superior to letting users handle it themselves.

Let's add it :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.