nikolaiwarner / aframe-chromakey-material Goto Github PK

Greeting,
I am quite new in this HTML and AFrame, with ZAPPAR Universal SDK.
If I make any mistakes, please forgive me.

The problem I facing now is when I try to use this material, the video will be invisible completely.
(on desktop, and on mobile)

Here are my snippet, if inform me if I am wrong or need more information.

<a-asset-item` id="target-file" `src="example-tracking-image.zpt"/>

<video id="Green_Screen" loop="false" src="Green_Screen.mp4"></video>
</a-assets>

<a-entity zappar-image="target: #target-file" id="image-group">

<a-video src="#Green_Screen" width="4" height="2.25" position="0 0 0.1" scale= "0.8 0.8 1" material="shader: chromakey; src: #Green_Screen; color: 0 0 0" webkit-playsinline playsinline></a-video>
</a-entity>

Please forgive me if I do it wrong.

Dear all,

upgrading to a recent release of aframe leads to error messages when evaluating the chromykey fragment shader. This is due to ThreeJS.ShaderMaterial expecting glsl 3 level code now, in which 'texture' is a function name, thus cannot be used as a variable any longer.

Here's a similar problem and discussion: mrdoob/three.js#19627

For anyone looking for a quick fix: simply changing the variable name for the texture2d uniform in the fragmentShader to eg. 'myTexture', then reflecting the change in the ShaderMaterial binding code fixes the problem.

Best regards,
Jan

The shader works nice, but the video doesn't autoplay in Chrome. How can I fix this?

I can't get it to not loop the video.
Even though i set it to loop="false"
Any tips?

the chromakey shader silently fails when material.src is updated.

I need to show transparent videos on a-frame that work both on android and iphone. I'm using your component: https://www.npmjs.com/package/aframe-chromakey-material

This chromakey component is working perfect with mp4 videos on desktop PC, but on mobile android, chrome shows the video as a black rectangle.Why is it working on Desktop PC with chrome, but failing on Android mobile with chrome, the code is exactly the same.

Btw I am using a button so that user clicks first to make sure video activates well on mobile. Without the chromakey component the mp4 video plays all well on mobile. It's only when I activate the shader that it goes all black only on mobile, on desktop all is good.

This happens with any mp4 video. The video uses H264 codec. Resolution is 1500x1500, no errors on console. Without the shader active, the mp4 video shows perfectly on both mobile and desktop. With the chromakey shader active, the video shows great without background on desktop but on mobile it shows a black rectangle and it fails

Currently using to overlay alpha video on top of an image, but where the image (background) should be showing, I am only seeing the camera feed (essentially looking through the background image). Is there anyway to see the background image properly with alpha transparent video on top?

I'm having trouble getting this to work on my android device (works find on desktop) and it appears to be related to the codec used to create the mp4. The symptom is a black plane being rendered instead of the movie.

The example provided with the component works fine. My problematic movie file was created via:

ffmpeg -i video01.mov -vf "scale=iw/2:ih/2,crop=335:536:326,fps=30" video01-green.mp4

And has this output from ffmpeg

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video01-green.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
encoder         : Lavf58.29.100
Duration: 00:00:35.38, start: 0.000000, bitrate: 282 kb/s
Stream #0:0(und): Video: h264 (High 4:2:2) (avc1 / 0x31637661), yuv422p10le, 334x536 [SAR 1:1 DAR 167:268], 144 kb/s, 30 fps, 30 tbr,15360 tbn, 60 tbc (default)
Metadata:
handler_name    : Core Media Video
timecode        : 00:00:00:00
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name    : Core Media Audio
Stream #0:2(eng): Data: none (tmcd / 0x64636D74)
Metadata:
handler_name    : Core Media Video
timecode        : 00:00:00:00

I'm having issues where the shader is not 100% correctly applied when running on Desktop Chrome or Android Chrome. It works flawless on Chrome in iOS. On Android and Desktop, the chroma key is not removed completely but only somewhat transparent. There are no errors logged in the console.

Running on android:

Running on iOS:

(funnily enough you can see the same issue when taking a screenshot on iPhone... Which is why I had to take a photo of the phone to show that it's working as intended...)

Hi there,

It seems like this component is not supported in version 1.2.0 of AFRAME.

I tried using it following the readme file and got this error:

THREE.WebGLProgram: shader error: 0 35715 false gl.getProgramInfoLog invalid shaders� THREE.WebGLShader: gl.getShaderInfoLog() fragment ERROR: 0:96: 'texture' : function name expected ERROR: 0:96: 'rgb' : field selection requires structure, vector, or interface block on left hand side ERROR: 0:96: '=' : dimension mismatch ERROR: 0:96: '=' : cannot convert from 'const mediump float' to 'highp 3-component vector of float' �1: #version 300 es 2: #define varying in 3: out highp vec4 pc_fragColor; 4: #define gl_FragColor pc_fragColor 5: #define gl_FragDepthEXT gl_FragDepth 6: #define texture2D texture 7: #define textureCube texture 8: #define texture2DProj textureProj 9: #define texture2DLodEXT textureLod 10: #define texture2DProjLodEXT textureProjLod 11: #define textureCubeLodEXT textureLod 12: #define texture2DGradEXT textureGrad 13: #define texture2DProjGradEXT textureProjGrad 14: #define textureCubeGradEXT textureGrad 15: precision highp float; 16: precision highp int; 17: #define HIGH_PRECISION 18: #define SHADER_NAME ShaderMaterial 19: #define GAMMA_FACTOR 2 20: #define USE_SHADOWMAP 21: #define SHADOWMAP_TYPE_PCF 22: uniform mat4 viewMatrix; 23: uniform vec3 cameraPosition; 24: uniform bool isOrthographic; 25: #define DITHERING 26: 27: vec4 LinearToLinear( in vec4 value ) { 28: return value; 29: } 30: vec4 GammaToLinear( in vec4 value, in float gammaFactor ) { 31: return vec4( pow( value.rgb, vec3( gammaFactor ) ), value.a ); 32: } 33: vec4 LinearToGamma( in vec4 value, in float gammaFactor ) { 34: return vec4( pow( value.rgb, vec3( 1.0 / gammaFactor ) ), value.a ); 35: } 36: vec4 sRGBToLinear( in vec4 value ) { 37: return vec4( mix( pow( value.rgb * 0.9478672986 + vec3( 0.0521327014 ), vec3( 2.4 ) ), value.rgb * 0.0773993808, vec3( lessThanEqual( value.rgb, vec3( 0.04045 ) ) ) ), value.a ); 38: } 39: vec4 LinearTosRGB( in vec4 value ) { 40: return vec4( mix( pow( value.rgb, vec3( 0.41666 ) ) * 1.055 - vec3( 0.055 ), value.rgb * 12.92, vec3( lessThanEqual( value.rgb, vec3( 0.0031308 ) ) ) ), value.a ); 41: } 42: vec4 RGBEToLinear( in vec4 value ) { 43: return vec4( value.rgb * exp2( value.a * 255.0 - 128.0 ), 1.0 ); 44: } 45: vec4 LinearToRGBE( in vec4 value ) { 46: float maxComponent = max( max( value.r, value.g ), value.b ); 47: float fExp = clamp( ceil( log2( maxComponent ) ), -128.0, 127.0 ); 48: return vec4( value.rgb / exp2( fExp ), ( fExp + 128.0 ) / 255.0 ); 49: } 50: vec4 RGBMToLinear( in vec4 value, in float maxRange ) { 51: return vec4( value.rgb * value.a * maxRange, 1.0 ); 52: } 53: vec4 LinearToRGBM( in vec4 value, in float maxRange ) { 54: float maxRGB = max( value.r, max( value.g, value.b ) ); 55: float M = clamp( maxRGB / maxRange, 0.0, 1.0 ); 56: M = ceil( M * 255.0 ) / 255.0; 57: return vec4( value.rgb / ( M * maxRange ), M ); 58: } 59: vec4 RGBDToLinear( in vec4 value, in float maxRange ) { 60: return vec4( value.rgb * ( ( maxRange / 255.0 ) / value.a ), 1.0 ); 61: } 62: vec4 LinearToRGBD( in vec4 value, in float maxRange ) { 63: float maxRGB = max( value.r, max( value.g, value.b ) ); 64: float D = max( maxRange / maxRGB, 1.0 ); 65: D = clamp( floor( D ) / 255.0, 0.0, 1.0 ); 66: return vec4( value.rgb * ( D * ( 255.0 / maxRange ) ), D ); 67: } 68: const mat3 cLogLuvM = mat3( 0.2209, 0.3390, 0.4184, 0.1138, 0.6780, 0.7319, 0.0102, 0.1130, 0.2969 ); 69: vec4 LinearToLogLuv( in vec4 value ) { 70: vec3 Xp_Y_XYZp = cLogLuvM * value.rgb; 71: Xp_Y_XYZp = max( Xp_Y_XYZp, vec3( 1e-6, 1e-6, 1e-6 ) ); 72: vec4 vResult; 73: vResult.xy = Xp_Y_XYZp.xy / Xp_Y_XYZp.z; 74: float Le = 2.0 * log2(Xp_Y_XYZp.y) + 127.0; 75: vResult.w = fract( Le ); 76: vResult.z = ( Le - ( floor( vResult.w * 255.0 ) ) / 255.0 ) / 255.0; 77: return vResult; 78: } 79: const mat3 cLogLuvInverseM = mat3( 6.0014, -2.7008, -1.7996, -1.3320, 3.1029, -5.7721, 0.3008, -1.0882, 5.6268 ); 80: vec4 LogLuvToLinear( in vec4 value ) { 81: float Le = value.z * 255.0 + value.w; 82: vec3 Xp_Y_XYZp; 83: Xp_Y_XYZp.y = exp2( ( Le - 127.0 ) / 2.0 ); 84: Xp_Y_XYZp.z = Xp_Y_XYZp.y / value.y; 85: Xp_Y_XYZp.x = value.x * Xp_Y_XYZp.z; 86: vec3 vRGB = cLogLuvInverseM * Xp_Y_XYZp.rgb; 87: return vec4( max( vRGB, 0.0 ), 1.0 ); 88: } 89: vec4 linearToOutputTexel( vec4 value ) { return LinearToLinear( value ); } 90: 91: uniform sampler2D texture; 92: uniform vec3 color; 93: varying vec2 vUv; 94: void main(void) 95: { 96: vec3 tColor = texture2D( texture, vUv ).rgb; 97: float a = (length(tColor - color) - 0.5) * 7.0; 98: gl_FragColor = vec4(tColor, a); 99: }

Is there anyway you could provide some support here?

Much appreciated in advance!

Hello M8 first off awesome material, works good on my flat VR project on iOS/Android/desktop, but on oculus 2 I only hear sound but video isn't showin, not even black screen.

Does it work for you?

Thank you!