Many Custom Shader parameters missing from the Inspector panel. about godot_heightmap_plugin HOT 7 OPEN

Some uniforms are hidden because they are managed by the plugin, having them exposed could lead to mistakes, tools not working, and scene serialization issues. This is similar to all the built-in uniforms of Godot such as the screen texture, depth texture, canvas texture or TIME.

They are hidden based on this:

Regarding higher-res normalmap, you could actually override its uniform at runtime I think, using a script calling terrain.set_shader_param(...), it "might" just work because matching texture resolutions and editable formats mostly matter in the editor only to allow edition. Havent tested doing that tho.
You could also have your own uniform and replace all usages of the built-in uniform in your custom shader. But the main issue with all this, is the fact the original built-in texture will still get loaded... to actually replace it, you'd have to really modify the PNG file next to other maps in your terrain data folder. Although, the plugin doesnt expect this so it might error in some areas (there are debug checks).

If you change this in the plugin directly, you'll be on your own, because tools of the plugin assumes nothing can mess with them, so you might see (or worse, not see) side-effects of that, like texture duplication (since I assume you want to use your own but then the plugin still has its own stored in an internal resource, and wouldn't know when to use one or the other), tools not working since they work on the textures the plugin knows about and not what's in the shader uniform of every chunk in the rendering part; tools also have to workaround importing logic to make textures editable, which is not possible if you override them, or that burden would end up on you; or not supporting varying resolutions.

You are the first one since the beginning of this plugin to express a need for having one of the terrain maps be higher-resolution. If you want this to be part of the plugin, it has to be an option supported all the way from HTerrainData and related tools with UI to change it, it can't just be exposed by making the shader parameter public. Otherwise... you'll have to make your own modifications disregarding all of the tools.

from godot_heightmap_plugin.

Thanks,

I am still confused though. Why is it called Custom shader when the shader isn't expected to be customized? I mean once the user gets a full access to the terrain shader, they can mess up a lot more things, so hiding some of the parameters from them is hardly a measure.

I don't expect tools to work, in fact I'd almost want them to intentionally not work, because if I am supplying a procedurally generated HeightMap, NormalMap and Masks, I want to minimize the chance of making an accidental scult or texture paint brush stroke to 0. Right now it's a bit hard, because as long as the HTerrain object selected, it's not possible to be outside of all tools. I am always in some tool so I have to be very careful of not clicking. Maybe some lock button which would be a non-tool that would also hide the bottom toolbar.

I am a bit horrified I am the first one to request this since this is a common workflow in the industry for almost a decade. Only heightmap is usually in these comically small resolutions because of performance reason for both collision as well as mesh drawing, but NormalMaps are usually used in much higher resolutions, not only to make normals more detailed but also to make slope based terrain blending more detailed. Here's an example of that workflow in practice in my RTS game (in Unreal Engine):
https://www.youtube.com/watch?v=a93FSexjjDE The entire map is just 1017x1017 heightmap including out of the play area buffer zone, yet it looks much more detailed because of this workflow.

Here's an example of it in Godot, with the original NormalMap:

And the highres one:

If the landscape really has to look good, manually sculpting and texture painting the landscape just won't cut it in this decade.

Also, btw, not sure if this is a bug but the custom shader never works out of the box because the include path seems relative, but not functional:

It needs to be made relative to the game res root folder in order to work:

from godot_heightmap_plugin.

I am still confused though. Why is it called Custom shader when the shader isn't expected to be customized?

It is expected to be customized. But customizing the source of some of the textures isn't, because it means modifying more than just the shader. It's really a matter of support in the plugin, because tools really don't expect this texture to be modified here. At some point there was even the hypothesis that terrain maps could be split in many tiles, which means you would not even be able to assume there is only one texture to assign (that idea hasn't been implemented tho, and probably wont in the near future).

so hiding some of the parameters from them is hardly a measure.

It is, because those textures don't come from the shader. They come from a HTerrainData resource in the terrain node (which is exposed as a folder path currently), so even if they were just present in the inspector, it would not solve the problem that all these textures still get loaded, and that's the textures all the tools are based on editing.

In addition, these textures even get temporarily replaced in some situations, because by default Godot does not import them as ImageTextures (i.e they might be compressed and their pixels aren't available to the CPU). So they get converted by the plugin, which means replacing their instance because Godot uses different classes for this (a texture can't just be converted while keeping the same object instance). In fact, the heightmap in particular needs to be an ImageTexture, because heights are needed to create the collider and allow several useful functions to work. So in the hypothesis that you could just assign it in the inspector, you would have to make sure it is imported as such.

They also each need to have specific formats, that the tools are tuned to work with. If a format isn't supported, tools won't work.

On top of that, if they were exposed, they would end up saved too inside the scene where the terrain is, which is not expected because they are supposed to be in the HTerrainData resource, not in every terrain using that resource. And if you "dare" saving the scene while they are in editable format (ImageTexture), Godot will basically embed all the pixels into the scene file, bloating it tremendously. If TSCN is used (which is by default to be VCS-friendly), that's even worse because it will be in text form. This, and some other issues, are the reason HTerrainData exists.

Not saying what you wan't can't ever be done, just that the way the plugin is currently designed doesn't allow it to be done by solely exposing this through uniforms.

I don't expect tools to work, in fact I'd almost want them to intentionally not work, because if I am supplying a procedurally generated HeightMap, NormalMap and Masks, I want to minimize the chance of making an accidental scult or texture paint brush stroke to 0. Right now it's a bit hard, because as long as the HTerrain object selected, it's not possible to be outside of all tools. I am always in some tool so I have to be very careful of not clicking. Maybe some lock button which would be a non-tool that would also hide the bottom toolbar.

You can turn off the plugin in ProjectSettings, and edit hterrain.gd to have access to uniforms you want. This way only the logic in hterrain.gd will run and nothing else.
Unfortunately, previewing in the editor relies on a few things that only an editor plugin can provide, such as where the camera is to know where to focus LOD. It sounds stupid but nodes really can't tell in the editor, get_camera() returns null. So instead input events of the plugin grab the camera from a different API just to bring it to the terrain node.
Also again, editing uniforms isn't the solution, because HTerrainData IS the actual source of the textures, not the uniforms. Even if you replace the normalmap with an exposed uniform, there will still be normalmap in HTerrainData... which is the one you actually need to replace.

I am a bit horrified I am the first one to request this

The only reason you'd end up wanting this, is because one of your textures needs to have different resolution, which nobody asked yet, and the plugin doesn't support out of the box. Outside of this, providing procedural textures is already covered, because you should be able to provide them using a script by building an HTerrainData resource yourself. Importing existing images is also possible in the editor, and I say "importing" once again because textures must be in specific formats for the plugin to work.

If the landscape really has to look good, manually sculpting and texture painting the landscape just won't cut it in this decade.

Totally agree, but manual sculpting must still be supported for everyone not needing realistic-looking terrain. I can't throw away all of the tools and make this "just a renderer" for the sake of it, and there are likely existing project using this. Also, there are more tools than just the caveman "raise" and "lower", to apply brushes, generate, smooth or erode heights. I also have a draft of a "noise brush" locally, which can sculpt continuous details.

I totally see the benefit of high-res normalmap, I'm just not sure yet how to support your use case because it kinda requires to make a huge bypass in many locations as long as dimensions don't match, since a low-res heightmap can't possibly be used to update a high-res normalmap, therefore the heightmap can't be edited by any tool that affects heights.

Also, btw, not sure if this is a bug but the custom shader never works out of the box because the include path seems relative

I fixed that in the master branch 380b82b, it's not yet on a version in the asset library.

from godot_heightmap_plugin.

The only reason you'd end up wanting this, is because one of your textures needs to have different resolution, which nobody asked yet, and the plugin doesn't support out of the box. Outside of this, providing procedural textures is already covered, because you should be able to provide them using a script by building an HTerrainData resource yourself. Importing existing images is also possible in the editor, and I say "importing" once again because textures must be in specific formats for the plugin to work.

Now I totally see the benefit of high-res normalmap, I'm just not sure yet how to support your use case because it kinda requires to make a huge bypass in many locations as long as dimensions don't match, since a low-res heightmap can't possibly be used to update a high-res normalmap, therefore the heightmap can't be edited by any tool that affects heights.

For a second I thought indeed I am going about it wrong, and the right way would be to supply the terraindata such as normal myself by building the HTerrainData as you suggested, but in any case, I would want to use the Custom shader still. I would not want to paint the textures using some limited texture layer system. I would want to do things like normal slope blending and build my own heightmap blending layers. In such case, it becomes pointless to even touch anything outside the .gdshader as I'd get the same control but scattered over multiple places.

Right now, it just works if I don't break the parts of the shader that do displacement. I've managed to cut down the custom shader to this while still not breaking anything:

shader_type spatial;

#include "res://addons/zylann.hterrain/shaders/include/heightmap.gdshaderinc"

uniform sampler2D u_normalmap;

uniform sampler2D u_terrain_heightmap;
uniform mat4 u_terrain_inverse_transform;
uniform mat3 u_terrain_normal_basis;

varying vec2 fragment_UV;


vec3 unpack_normal(vec4 rgba) {
	vec3 n = rgba.xzy * 2.0 - vec3(1.0);
	// Had to negate Z because it comes from Y in the normal map,
	// and OpenGL-style normal maps are Y-up.
	n.z *= -1.0;
	return n;
}

void vertex() {
	vec4 wpos = MODEL_MATRIX * vec4(VERTEX, 1);
	vec2 cell_coords = (u_terrain_inverse_transform * wpos).xz;
	cell_coords += vec2(0.5);

	// Normalized UV
	UV = cell_coords / vec2(textureSize(u_terrain_heightmap, 0));
	fragment_UV = cell_coords / (vec2(textureSize(u_terrain_heightmap, 0)) - vec2(1.5));
	

	// Height displacement
	float h = sample_heightmap(u_terrain_heightmap, UV);
	VERTEX.y = h;
	wpos.y = h;

	NORMAL = u_terrain_normal_basis * unpack_normal(texture(u_normalmap, UV));
}

void fragment() {	
	ALBEDO = vec3(0.5);

	vec3 terrain_normal_world = u_terrain_normal_basis * unpack_normal(texture(u_normalmap, fragment_UV));
	vec3 normal = normalize(terrain_normal_world);

	NORMAL = (VIEW_MATRIX * (vec4(normal, 0.0))).xyz;
}

I guess the best way to support my use case would be to simply introduce another new Node3D type, something like "HTerrainBasic" (but with possibly better name):

1. This node would not have any tools
2. This node would not support built in texture sets
3. This node would not have Shader Type, it'd just have Shader slot which would create super basic shader similar to one above.
4. The node would have 2 image slots for supplied heightmap and normal map.
5. The node would have Min height and Max height parameters to defined the range of the normal map.

I assume the step 4 is necessary so that the collision generation part of the plugin can retrieve the heightmap for the collision. But if GDScript can access shader uniforms and retrieve image resource paths from them, the step 4 could be completely omitted and and the HeightMap and NormalMap could just be regular uniform parameters of the pre-created simple shader template. But if the user then wrote their own hightmap remapping function, the collision mesh would not be aware of that, so maybe that's not the best idea.

But in any case, the idea is just to have the addon do the heighmap mesh with distance LODing and heightmap collision of a simple, basic terrain square for me, and let me do everything else (shading). The approach of creating new simpler node sounds more elegant to me than adding some "Truly Custom" shader type and then scattering many checks all over the code and switching features such as parameter hiding based on them.

from godot_heightmap_plugin.

I reluctant having to maintain a separate node that is a subset of the other... and it's still not as simple as it sounds, mostly because of the heightmap. I wish the core system would just be able to handle that case normally.

Since the plugin generates the normalmap and doesn't do that much with it, I could perhaps add a custom_normalmap checkbox to instead allow providing one manually, which would make the field available (since it can then be modified manually), remove the normalmap from HTerrainData, and disable the automatic normalmap baker. Sculpting tools would still work, but would obviously no longer affect the normalmap, so initially it's just a matter of not using them. Then they can eventually get greyed out to prevent mistakes when that option is on, which needs work, but doesnt have to be done at the same time.

If you want to have this working right now with no changes to the plugin, you can already define your normalmap as a custom uniform and use it in places of the built-in one, and it will do what you want. The only downside is extra samplers, and the fact the built-in normalmap is still loaded.

from godot_heightmap_plugin.

I reluctant having to maintain a separate node that is a subset of the other... I wish the core system would just be able to handle that case normally. Since the plugin generates the normalmap and doesn't do that much with it, I could perhaps add a custom_normalmap checkbox to instead allow providing one manually, which would make the field available (since it can then be modified manually), remove the normalmap from HTerrainData, and disable the automatic normalmap baker.

If you want to have this working right now with no changes to the plugin, you can already define your normalmap as a custom uniform and use it in places of the built-in one, and it will do what you want. The only downside is extra samplers, and the fact the built-in normalmap is still loaded.

Yes, I am aware that I can do it myself. It's just the huge amount of complexity to dig through. I need to ignore like 90% of the HTerrain featureset, but I either need to have all that complexity in the way or I need to spend a lot of setup time removing it. You are familiar with the thousands lines of code you've been writing over the years but I am not, so for me as well as any average user it's going to take quite a lot of time and reverse engineering.

Defining my own normal and replacing it is exactly what I did in the code snippet above.

I also wish that Godot would have something basic to handle it natively, so that I don't need to download some external plugin just to avoid writing heightmap rendering with distance LOD and collision from scratch.

Just please don't add any more checkboxes. It won't be useful for me, and probably not for any other users either. There's already way more complexity than necessary. I have it working already. It's just the amount of work and knowledge of obscure information it takes to create something that usually takes couple of clicks in other engines.

I do understand that you are a Godot developer though, not 3rd party, so I am aware I should not expect this addon to be designed for more generic use cases.

from godot_heightmap_plugin.

I also wish that Godot would have something basic to handle it natively

I wish too, but at the same time, if it did, I have a feeling that it would still not have supported what you want, because I haven't seen this idea floating anywhere in discussions I've seen. But that's speculation and I could be wrong, I havent been in all discussions hehe

Just please don't add any more checkboxes

Knowing the code I have to go through, it's easier for me to do that for the normalmaps case, than to copypaste all the core code and adapt it just for your case. If you expect something else, it will take more time, because I have other projects in progress.

I do understand that you are a Godot developer though, not 3rd party, so I am aware I should not expect this addon to be designed for more generic use cases.

I technically am a "Godot developer" in the sense I made a few PRs to the engine and use Godot, but I'm not sure what 3rd party would mean here. If "Godot developer" means developping the engine directly and official addons, then I'm a 3rd party. There is no terrain plugin "officially endorsed" by Godot, and I'm not even working full-time on this, only in small steps when I have free time.

from godot_heightmap_plugin.