0beqz / realism-effects Goto Github PK
View Code? Open in Web Editor NEWSSGI, Motion Blur, TRAA - Effects to enhance your three.js scene's realism
Home Page: https://realism-effects-obeqz.vercel.app/
License: MIT License
SSGI, Motion Blur, TRAA - Effects to enhance your three.js scene's realism
Home Page: https://realism-effects-obeqz.vercel.app/
License: MIT License
Hi @0beqz ,
Thanks for making the realism-effects. SSGI works great with HDRI images.
Does it support three js Lights? I tested it with three js Sunlamp, SSGI does not seems to work.
Difficult to setup and integrate into an existing three.js project. Setting up the Offscreen Canvas to use elsewhere is not explained. Also doesn't seem to work at all on mobile because of this?
Hey, congrats on this release! I'm excited to give it a quick try at least, but it seems the VelocityDepthNormalPass
is currently not being bundled/exported?
I used GPUComputationRenderer to compute the position and velocity for particles and save them as texture, and pass them to instancedMesh for rendering.
<instancedMesh>
<boxGeometry args={[0.2, 0.2, 0.6]}>
<instancedBufferAttribute attach="attributes-uvs" args={[uvs, 3]} />
</boxGeometry>
<ThreeCustomShaderMaterial/>
</instancedMesh>
In the vertex shader of the material for instancedMesh, I did something like this:
void main() {
vec3 pos = texture2D(positionTex, uvs.xy).xyz;
csm_PositionRaw = projectionMatrix * modelViewMatrix * vec4( position + pos, 1.0 );
},
When I apply the SSGI, all particles become bunch of boxes in the origin, and the positions I modified with the positionTex are not correctly reflected.
Is it some kind of the limit of SSGI?
I get a couple of errors when i'm calling SSGI in the latest version of the repo. I built this myself since npm was a little behind. I also tested this on the older npm version and i get the same errors. I'm using a couple gltf's and a boxGeometry. I thought this had to do with uv's but it seems boxGeometry has accurate uv's so that can't be the issue. And my GLTF's rendered fine in the previous version but now they don't render at all with the updated build.
THREE.WebGLProgram: Shader Error 0 - VALIDATE_STATUS false
Material Name: undefined
Material Type: MeshPhysicalMaterial
Program Info Log: Vertex shader is not compiled.
VERTEX
ERROR: 0:400: 'uvundefined' : undeclared identifier
ERROR: 0:400: 'constructor' : not enough data provided for construction
395: void main() {
396: #if defined( USE_UV ) || defined( USE_ANISOTROPY )
397: vUv = vec3( uv, 1 ).xy;
398: #endif
399: #ifdef USE_MAP
> 400: vMapUv = ( mapTransform * vec3( MAP_UV, 1 ) ).xy;
401: #endif
402: #ifdef USE_ALPHAMAP
403: vAlphaMapUv = ( alphaMapTransform * vec3( ALPHAMAP_UV, 1 ) ).xy;
404: #endif
405: #ifdef USE_LIGHTMAP
406: vLightMapUv = ( lightMapTransform * vec3( LIGHTMAP_UV, 1 ) ).xy;
I tried these configurations
Latest Github Repo build:
<postEffectsComposer ref={composerRef} args={[gl]}>
<postRenderPass args={[scene, camera]} attach={(parent, self) => parent.addPass(self)} />
<velocityDepthNormalPass
ref={velocityDepthNormalPassRef}
args={[scene, camera]}
attach={(parent, self) => parent.addPass(self)}
/>
{velocityDepthNormalPassRef.current && composerRef.current && (
<postEffectPass
args={[camera, new SSGIEffect(composerRef.current, scene, camera, velocityDepthNormalPassRef.current)]}
attach={(parent, self) => parent.addPass(self)}
/>
)}
Previous NPM install 1.1.2:
<postEffectsComposer ref={composerRef} args={[gl]}>
<postRenderPass args={[scene, camera]} attach={(parent, self) => parent.addPass(self)} />
<velocityDepthNormalPass
ref={velocityDepthNormalPassRef}
args={[scene, camera]}
attach={(parent, self) => parent.addPass(self)}
/>
{velocityDepthNormalPassRef.current && (
<postEffectPass
args={[camera, new SSGIEffect(scene, camera, velocityDepthNormalPassRef.current)]}
attach={(parent, self) => parent.addPass(self)}
/>
)}
)
One thing i want to note is that i'm able to successfully utilize the following:
{velocityDepthNormalPassRef.current && composerRef.current && (
<postEffectPass
args={[
camera,
// new HBAOEffect(composerRef.current, camera, scene),
new MotionBlurEffect(velocityDepthNormalPassRef.current),
new SharpnessEffect(0.5),
// new TRAAEffect(scene, camera, velocityDepthNormalPassRef.current),
(() => {
const effect = new SMAAEffect({
preset: SMAAPreset.MEDIUM,
edgeDetectionMode: EdgeDetectionMode.COLOR,
predicationMode: PredicationMode.DEPTH,
});
if (effect.edgeDetectionMaterial) {
const edgeDetectionMaterial = effect.edgeDetectionMaterial;
edgeDetectionMaterial.edgeDetectionThreshold = 0.02;
edgeDetectionMaterial.predicationThreshold = 0.002;
edgeDetectionMaterial.predicationScale = 1;
}
return effect;
})(),
]}
attach={(parent, self) => {
parent.addPass(self);
return () => {
};
}}
/>
)}
specifically MotionBlur and SharpnessEffect
continuing this thread from twitter https://twitter.com/vis_prime/status/1632602124269023233?s=20
I'm noticing trails being formed which does not happen on the demo page
Using OffscreenCanvas seems to result in issues when using Safari, so it should maybe be removed from the demo.
Not exactly sure how to title this issue correctly;
while testing a few more "clean" models to learn more about how realism-effects works I ran into a strange phenomenon:
I encountered that a couple of times after refreshs as well, but I don't have clear reproduction steps unfortunately.
The model was this one:
AmbientOcclusionTest.glb.zip
Changing "color" in the material on runtime won't work.
It seems like propsPrimitive
didn't have color property inherited from Material
class.
(https://github.com/0beqz/realism-effects/blob/main/src/gbuffer/material/GBufferMaterial.js)
let props = Object.keys(gBufferMaterial)
props = props.filter(
key => !key.startsWith("_") && !key.startsWith("is") && key !== "uuid" && key !== "type" && key !== "transparent"
)
const propsPrimitive = props.filter(
key => typeof gBufferMaterial[key] === "string" || typeof gBufferMaterial[key] === "number"
)
Hey there! Thanks for releasing such an amazing project!!
Are emissive textures supposed to work? They seem to have no effect, even when creating some random messages to test it out.
I'm trying to get a video texture to reflect on the floor like in one of your old examples but nothing seems to pick up light from emissive textures.
I just want to make sure I'm not missing something!
Just testing a bit on adjusting texture and material with realism effects SSGI passes
I notice the texture repeat, center, rotation and offset is not working, also in material normal strength, env map internsity also not working
There is quite some discrepancy between what's outlined in Usage and what's done in the example page.
Is there a more basic example somewhere that just shows minimal setup, all options at defaults, using a basic EXR skybox and one model?
If not, it would be great to have that – I'm not able to get something fully working from the Usage example code (e.g. no IBL, motion blur / taa don't actually work, ...).
Here is a minimal example.
https://codepen.io/benzsuankularb/pen/WNLzMyg
Basically, I've setup 2 canvas
One with Realism-Effect and one with normal MeshStandardMaterial.
If both canvases are enabled then the env map in MeshStandardMaterial won't work.
Both canvases didn't share any resources on the application layer.
I'm not sure if this is some issue on the Three / WebGL layer.
Or there are some modifications on the Realism-Effect applied to the MeshStandardMaterial shader.
Hello! I have a problem on my ipad pro with m1 chip when i bring the camera close to an object
In the example, when changing the camera from perspective to orthographic, tracing does not seem to work, and because of this, the light creates a completely different picture, slightly overexposed and without such shadows as with a perspective camera.
// const camera = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 0.01, 250)
const frustumSize = 10
const aspect = window.innerWidth / window.innerHeight
const camera = new THREE.OrthographicCamera(
-frustumSize * aspect,
frustumSize * aspect,
frustumSize,
-frustumSize,
1,
1000,
)
scene.add(camera)
Perspective (default example)
Orthographic
Is it possible to achieve a similar result with an orthographic camera as a perspective one? Or is it not supported yet?
I have 2 WebGLRenderer(Canvas) in my web.
However, The second one didn't make use of realism effects.
But the env map goes black when realism effects were used in the first canvas.
I've dug a bit and found the patches of ShaderChunk.envmap_physical_pars_fragment
(https://github.com/0beqz/realism-effects/blob/main/src/ssgi/utils/Utils.js)
I also found the SSGIEffect is trying to disable the patch at the end of each frame.
(https://github.com/0beqz/realism-effects/blob/main/src/ssgi/SSGIEffect.js)
But that's not working when another renderer tries to use envmap_physical_pars_fragment
along the way.
globalIblRadianceDisabledUniform.value = true
this.rAF = requestAnimationFrame(() => {
this.rAF2 = requestAnimationFrame(() => {
globalIblRadianceDisabledUniform.value = false
})
})
First of all, amazing what you have been doing here! Thank you for all your work here.
I was wondering if there will also be support for this on mobile. I tested with the AO effects and there seem to be issues on my Android device. Do you know a workaround for this?
Hi, great work on this!
I have the following code
const camera = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 0.01, 250)
const velocityDepthNormalPass = new PostProcessing.VelocityDepthNormalEffect(this.getThreeScene(), camera);
I am currently getting the following error
I have no doubt that it is completely my problem....however, I am stumped at where to start to troubleshoot it so any suggestions would be very much appreciated. Im using Threejs r155
Hello, my statement is more a question than an issue but i didn't know how to reach you so please, forgive me if you find this entry too much intrusive.
Basically, i can't understand the current status of your SSR effect. I mean, i've seen the other repository which at the moment seems to be deprecated due to an incompatibility with the last version of three.js.
At the moment, is the SSR effect implemented in you SSGI code? I've seen the SSR quote in the realism-effects description page but there is no description and so for me is a little bit misleading...
Anyway, thanks for your work, i really appreciate it!
If you run the demo and change the model to cyber_samurai, the whole model turns black within a few seconds.
I am running this on Mac OS 13.5.1 with M2 chip and google chrome 116.0.5845.140. Here is the
Hi,
First-of-all, thanks for this great library of effects! I'd like to try-out some recent realism-effects which are not yet available in the module version published to npm 1.1.2
. I see you have deployments set-up for production/preview - which might just be hooks to vercel, and publish up-to-date demo pages after any commit.
Could you also push the updated realism-effects build to npm within the same vercel deployment task? Or set-up a Github Action to do so, eventually on versioned? Would be great to test new featues without building the lib ourselves.
Is there a recommended way to integrate the SSGI with @react-three/fiber
? I tried adapting your code like this, but it did not work:
const SSGI = () => {
const { camera, composer, scene } = useContext( EffectComposerContext )
const velocityDepthNormalPass = new VelocityDepthNormalPass( scene, camera )
composer.addPass(velocityDepthNormalPass)
const ssgiEffect = new SSGIEffect( scene, camera, velocityDepthNormalPass, options )
const traaEffect = new TRAAEffect( scene, camera, velocityDepthNormalPass )
const motionBlurEffect = new MotionBlurEffect( velocityDepthNormalPass )
return <EffectComposer>
<primitive object={motionBlurEffect}/>
<primitive object={traaEffect}/>
<primitive object={ssgiEffect}/>
</EffectComposer>
}
Hello, I'm having an issue setting up the AO (both HBAO and SSAO) with varying scene sizes.
In this first example, I have a pretty standard setup with the HBAO effect. Everything works as expected.
When I now introduce a scalingFactor
that scales all elements in the scene as in this second example, the results of the AO are not consistent. I tried to adjust the distance
, thickness
and radius
by the scalingFactor
with no success.
Is there a way to achieve consistent AO results depending on the scene scale?
So helpers like TransformControls crash the render loop (same issue as the old SSR )
and THREE.Lines create black overlay/glitch problems when SSGI is turned on
So if we can use the selection class from postprocessing that would be nice to have eliminate these problems
I tried to test this pr(pmndrs/react-postprocessing#211) for test in react-postprocessing
I found when SSGIEffect instance created, Javascript VM instance has added so I think it is kind of workers but I can't find where the worker created in chrome Mac OS
Since recently, three version was restricted a lot more, is that absolutely required?
Line 53 in e6e4f57
I ask this because currently @pmndrs/postprocessing
for example only works with r148, but make other libs will benefit from a more loose peerDep?
When clicking through the sample models on https://realism-effects.vercel.app/, I noted the Swordsman looks wrong around the cape / cutout parts:
I think either transparency or cutout may not be working correctly with SSGI on.
EDIT: even more apparent on "cyber samurai" and can also be seen on "amg" on the headlights, on "motorbike" on the glass parts, on "cyberpunk_bike" on the floor.
I understand there are limitations with depth-based effects and transparency - would be great if those objects would render kind of correct though.
While the GLTFLoader always guaranteed only single material per mesh.
Multiple materials per mesh are natively supported by three.js.
It'd be nice if we cloud add this support. :)
I tested both locally and online, with your example code, as well as my own working code, and experienced a memory leak: the same model, loaded multiple times, and each load cleared the memory of the previous model, but the memory continued to rise. When I use normal rendering, this problem does not occur. The memory comparison diagram is as follows: The leak points to Realistic-Effects, and I think it may be that the Realistic-Effects rendered real particle geometry memory is not fully cleared
If "NORMAL_IN_RGB" is not defined, the following line in "poissionDenoise.frag" causes a compilation error:
float center = texel.rgb;
For a solution see #21
Thanks for the great project. I've just started using it and spent a lot of time trying to integrate it with react-three-fiber (Success!). Now, I'm just having fun.
I have found the issue that covers this one while I was writing it. :/ Feel free to close. I still think it's good for future reference if somebody has the same issue.
The related issue is https://github.com/0beqz/realism-effects/blob/eb03a9e4a50bd7f0242bafb77d8e9b0c4c73543b/src/ssgi/utils/Utils.js#L238C24-L238C24
First of all, I'm using realism-effects with react-three-fiber and in general it works just fine. I discovered an issue though, that is architectural.
VelocityDepthNormalPass does deep replacement of materials in the whole scene. That's quite an intrusive operation as it's impossible to tell if some user of replaced material relies on custom property that is went missing.
In my case it's "color" on one of the handlers of TransformControls. The code tries to clone a color without doing undefined checks.
I've temporary fixed it by adding "color" to
const materialProps = ["vertexTangent", "vertexColors", "vertexAlphas", "vertexUvs", "uvsVertexOnly", "supportsVertexTextures", "instancing", "instancingColor", "side", "flatShading", "skinning", "doubleSided", "flipSided"];
Proxy can be used to fix at least part of property access errors but it is still going to be brittle. In my opinion it's good to implement both approches.
Partly related #14 as it will allow to exclude some of the offending objects in scene graph.
TransformControls:
https://github.com/pmndrs/three-stdlib/blob/4c04593ee49bb0b022025718844f3ce2b21f67bf/src/controls/TransformControls.ts#L1301C40-L1301C40
ssgi/.../Util:
https://github.com/0beqz/realism-effects/blob/eb03a9e4a50bd7f0242bafb77d8e9b0c4c73543b/src/ssgi/utils/Utils.js#L238C24-L238C24
Although, am not using any environment map for the scene, I quite love the way SSGI lifts up the scene. Is there a way to avoid the trailing effect at longer distance?
Am using this settings as option
options = {
"distance": 1.5,
"thickness": 20,
"autoThickness": false,
"maxRoughness": 1,
"blend": 0.95,
"denoiseIterations": 1,
"denoiseKernel": 3,
"denoiseDiffuse": 20,
"denoiseSpecular": 0,
"depthPhi": 14.022,
"normalPhi": 0,
"roughnessPhi": 0,
"envBlur": 0.55,
"importanceSampling": true,
"directLightMultiplier": 1,
"maxEnvLuminance": 50,
"steps": 8,
"refineSteps": 1,
"spp": 3,
"resolutionScale": 2,
"missedRays": false
};
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.