Not a specific issue, but rather a comment/critique of how partial results are handled in some cases.

The EitherCodec always tries to decode the left first, but always returns the right in the case of a partial result. This is even more problematic when the either has the same type, and is used to encode/decode in two different manners, perhaps then a concatenation of the two partials is wanted.

The collections codec that use apply2stable also have a problem, outlined in #55 , which discards entries after the first no-partial DataResult. Something else that might be worth considering is that currently the collection always has a partial result, but should it contain only contain the completely succesful elements of the collection? Or should it also contain the ones that have a partial result? Also, since they always have a partial result, even if all elements have failed completely, the collection will have an empty collection as a partial, which might not be desired.

See MinecraftForge/MinecraftForge#7582 for the issue that arises from minecraft's use of the EitherCodec.

https://github.com/Mojang/DataFixerUpper/blob/master/src/main/java/com/mojang/datafixers/types/Type.java#L40

Please apply a cache eviction policy to this REWRITE_CACHE, thanks.

There's currently no support for Sets and other things. Also there's no flexibility for what kinds of maps you can get out of the thing - there's no support for IdentityHashMaps either.

Have you considered porting Serde to Java?

Hello, thanks for the code release!

I found it fascinating that this library is using such advanced technology from the category theory/functional programming world!

Could you tell us a bit about how you learned about these tools and how you decided to use them in practice?

All the best

I find Codecs to be a really powerful and useful tool that I would like to be able to use in my own projects without also pulling in everything else in DFU. Would you please split DFU up into a couple of modules similar to how Fast Util has a core module and a full module?

A small example showing the problems this solves, and how to apply them would be great.

Tests would be a nice addition.

https://docs.oracle.com/javase/specs/jls/se17/html/jls-15.html#jls-15.27.4

This implies that the identity of the result of evaluating a lambda expression (or, of serializing and deserializing a lambda expression) is unpredictable, and therefore identity-sensitive operations (such as reference equality (§15.21.3), object locking (§14.19), and the System.identityHashCode method) may produce different results in different implementations of the Java programming language, or even upon different lambda expression evaluations in the same implementation.

Yet this behaviour is relied upon in the hashCode/equals functions in TypeRewriteRule.OrElse

This was found when testing Minecraft 1.13.21 on IKVM, an alternative JVM ikvmnet/ikvm#342

When mineraft crashes cause of codec's it should give more details like where it crashed and the reason why

It would be great if DFU had native support for multiple different sets of transformations at the same time. Allowing this would greatly benefit mods, as they could easily have breaking changes in versions that don't line up with Minecraft versions.

I would love a way to support more than 16 params for Codecs since rn it's limited to 16 cause of the Products and Kind1 class only going to 16. Aint there a better way to do it?

The rendering engine could do with open sourcing plus could use newer opengl again to reduce clock speed needed again since the slightest lag in clock shows

Figure out what is the strategy for adding a new primitive codec/serialization operation, and implement missing ones

public static void main(String[] args) {
	JsonObject j = new JsonObject();
	j.add("test", JsonNull.INSTANCE);
	
	System.out.println(JsonOps.INSTANCE.convertTo(JsonOps.INSTANCE, j)); //should return a copy of `j`
}

fails with

Exception in thread "main" java.lang.NullPointerException: Cannot invoke "com.google.gson.JsonElement.getAsJsonPrimitive()" because "input" is null
	at com.mojang.serialization.JsonOps.convertTo(JsonOps.java:50)
	at com.mojang.serialization.JsonOps.convertTo(JsonOps.java:24)
	at com.mojang.serialization.DynamicOps.lambda$convertMap$27(DynamicOps.java:255)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at com.mojang.serialization.JsonOps.createMap(JsonOps.java:275)
	at com.mojang.serialization.JsonOps.createMap(JsonOps.java:24)
	at com.mojang.serialization.DynamicOps.convertMap(DynamicOps.java:254)
	at com.mojang.serialization.JsonOps.convertTo(JsonOps.java:42)
	at agency.highlysuspect.apathy.rule.CodecUtil.main(CodecUtil.java:145)

It appears there is a check for instanceof JsonNulls:

But it doesn't work, because DynamicOps#getJsonValues, as well as other methods like getStream and getList, filter JsonNulls to actual nulls.

A small QOL thing would be to have a Codec<Set<A>> implementation, when unique values are desired. It would be nice to be able to do .uniqueListOf() on any codec.

So I was toying around with DFU and I discovered a bug while playing with Schemas to register my type references.

So if you create a DataFixer through the DataFixerBuilder and don't add a DataFix to any registered schema version (adding a DataFix to any schema fixes the issue), when you build and then update a Dynamic through the update method, you get an NPE from the DataFixer inside of private int getLowestFixSameVersion(final int versionKey) within DataFixerUpper at Line 115 where it checks for fixerVersions.

The intended functionality I thought would occur is the DFU would just pass out the Dynamic unchanged because there are no fixes at all to modify the Dynamic with, however the update seems to attempt to fix the data even if there are not any fixes to actually apply.

Sorry about the horrible wording it is about 10:45 at night here. Just ask me if you need any clarification about any more explanation.

There are very few documentation comments, and the ones that are there are not very useful. Did you all strip the comments from the repository before publishing DFU?

As of 1.0.20, Schema contains registerEntities and registerBlockEntities methods, and registerTypes contains entityTypes and blockEntityTypes parameters.

pls

Issue from https://bugs.mojang.com/browse/MC-135413

I found a problem with the class: com.mojang.datafixers.Dynamic
When I try to convert a boolean value from a JsonElement it always turns up as false.

Base a basic UnitTest to explain my scenario.

import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import com.mojang.datafixers.Dynamic;
import com.mojang.datafixers.types.JsonOps;
import net.minecraft.server.v1_13_R1.DynamicOpsNBT;
import net.minecraft.server.v1_13_R1.NBTTagCompound;
import org.junit.Test;

import static org.junit.Assert.*;

public class DynamicTest {
    @Test
    public void convertTest() {
        JsonObject element = new JsonObject();
        element.addProperty("string", "hello world");
        element.addProperty("boolean_true", true);
        element.addProperty("boolean_false", false);
        element.addProperty("byte_1", (byte) 1);
        element.addProperty("byte_0", (byte) 0);
        element.addProperty("double", 10.5);
        element.addProperty("int", 11);
        element.addProperty("long", 12L);
        element.addProperty("float", 13.5F);
        element.addProperty("short", (short) 100);

        JsonObject properties = new JsonParser().parse(element.toString()).getAsJsonObject();

        assertEquals("hello world", properties.get("string").getAsString());
        assertEquals(true, properties.get("boolean_true").getAsBoolean());
        assertEquals(false, properties.get("boolean_false").getAsBoolean());
        assertEquals((byte) 1, properties.get("byte_1").getAsByte());
        assertEquals((byte) 0, properties.get("byte_0").getAsByte());
        assertEquals(10.5, properties.get("double").getAsDouble(), 10.5);
        assertEquals(11, properties.get("int").getAsInt());
        assertEquals(12L, properties.get("long").getAsLong());
        assertEquals(13.5F, properties.get("float").getAsFloat(), 13.5F);
        assertEquals((short) 100, properties.get("short").getAsShort());

        NBTTagCompound dynamic = ((NBTTagCompound) Dynamic.convert(JsonOps.INSTANCE, DynamicOpsNBT.a, properties));

        assertEquals("hello world", dynamic.getString("string"));
        assertEquals(true, dynamic.getBoolean("boolean_true")); // Fails here?? Dynamic.class can't convert true to a boolean.
        assertEquals(false, dynamic.getBoolean("boolean_false"));
        assertEquals((byte) 1, dynamic.getByte("byte_1"));
        assertEquals((byte) 0, dynamic.getByte("byte_0"));
        assertEquals(10.5, dynamic.getDouble("double"), 10.5);
        assertEquals(11, dynamic.getInt("int"));
        assertEquals(12L, dynamic.getLong("long"));
        assertEquals(13.5F, dynamic.getFloat("float"), 13.5F);
        assertEquals((short) 100, dynamic.getShort("short"));
    }
}

The result when we run the UnitTest is

java.lang.AssertionError: 
Expected :true
Actual   :false
 <Click to see difference>


	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at DynamicTest.convertTest(DynamicTest.java:43) // The line is commented in the example above.
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...

Where will this bug be a problem?

When you set the generator-settings in the server.properties to a JSON object it will always turn up in the level.dat as:

useCaves: 0
useRavines: 0

When you set the generator-settings to {"useCaves":true,"useRavines":false}

Description

When creating a new schema, buildTypes() is called. In that method templates.stream().reduce(DSL::or).get();.reduce(DSL::or).get(); At this time templates only includes the entries in the RECURSIVE_TYPES map. The .get() method is troublesome, as if the .reduce() function didn't return anything, then the method reports a NoSuchElementException and breaks. If you don't add any recursive types, then .get() throws the error.

Error Log:

Exception in thread "main" java.util.NoSuchElementException: No value present at java.util.Optional.get(Optional.java:135) at com.mojang.datafixers.schemas.Schema.buildTypes(Schema.java:50) at com.mojang.datafixers.schemas.Schema.<init>(Schema.java:38) at DataFixerUpperExamples.schemas.Version1.<init>(Version1.java:16) at com.mojang.datafixers.DataFixerBuilder.addSchema(DataFixerBuilder.java:40) at com.mojang.datafixers.DataFixerBuilder.addSchema(DataFixerBuilder.java:34)

Details of how to reproduce

Create a new class that extends Schema with the regular override of functions registerEntities(), registerBlockEntities() and registerTypes(), but don't register a recursive type, (i.e. never schema.registerType(true, ...); ) For my schema class, see below:

public class Version1 extends Schema {

public Version1(int versionKey, Schema parent) {
	super(versionKey, parent);
}

public Version1() {
  super(1, null);
}

public void registerTypes(final Schema schema, final Map<String, Supplier<TypeTemplate>> entityTypes, final Map<String, Supplier<TypeTemplate>> blockEntityTypes) {
	schema.registerType(false, TypeReferences.LEVEL, DSL::remainder);
}

public Map<String, Supplier<TypeTemplate>> registerEntities(final Schema schema) {
    Map<String, Supplier<TypeTemplate>> map = Maps.newHashMap();
	return map;
}

public Map<String, Supplier<TypeTemplate>> registerBlockEntities(final Schema schema) {
	Map<String, Supplier<TypeTemplate>> map = Maps.newHashMap();		
	return map;
}

}

Current Situation:
The project currently utilizes RecordCodecBuilder for the storage of Map Entries which supports heterogeneous data types. However, there seems to be a gap when it comes to storing elements of Lists in a similar heterogeneous manner.

Proposed Feature:
I have developed a feature named TupleCodec aimed at addressing this gap. The TupleCodec is designed to facilitate the storage of heterogeneous list elements.

Inquiry:
Is there an interest in integrating such a feature into the project? I believe TupleCodec could significantly enhance the project's data handling capabilities, especially for complex data structures involving lists of mixed data types. If the project community is open to this addition, I am willing to contribute by opening a pull request.

I look forward to hearing your thoughts on this proposal.