Key for the ES256 algorithm must be one of type KeyObject or CryptoKey. Received an instance of Uint8Array about backstage HOT 22 CLOSED

I see it fails on CatalogAuthResolverContext.ts when tries to get catalog entities await this.catalogApi.getEntityByRef(entityRef, { token });

from backstage.

Are you deployed behind a reverse proxy like for example oauth2proxy? Could it be configured to inject authorization headers of its own? We've seen some cases like that, where an intermediary proxy was ruining the Backstage token passing, so that the backend receiving the calls could not recognize the auth it received. Reconfiguring the proxy made the problem go away in those instances

from backstage.

the problem is it fails not on external call, but when tries to get catalog entity after successful auth in okta, so it fails on inter-module communication

from backstage.

"It" is the browser here, making a request to a backend plugin (catalog), am I understanding that correctly?

from backstage.

nope, auth plugin makes request to catalog plugin

from backstage.

When you did this upgrade to the new Backstage release, did you update both the auth plugin and the catalog plugin to the newest versions?

Is your backend instance migrated to the new backend system?

Did you have a custom TokenManager in place in your backend?

from backstage.

What do you mean by upgrading plugin to the newest version ? if backstage deps then yes

I migrated backend to the new backend system, but some internal plugins are still not migrated

I don't have custom TokenManager

from backstage.

Can you run yarn why jose and paste the output here also, and what database are you using?

from backstage.

@benjdlambert

├─ @backstage/backend-app-api@npm:0.7.0
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/backend-common@npm:0.21.7
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/backend-common@npm:0.21.7 [bd11c]
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-backend-module-aws-alb-provider@npm:0.1.9
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-backend-module-azure-easyauth-provider@npm:0.1.0
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-backend-module-cloudflare-access-provider@npm:0.1.0
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-backend-module-microsoft-provider@npm:0.1.12
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-backend-module-oauth2-proxy-provider@npm:0.1.10
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-backend@npm:0.22.4
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
├─ @backstage/plugin-auth-node@npm:0.4.12
│  └─ jose@npm:5.2.3 (via npm:^5.0.0)
│
└─ openid-client@npm:5.6.4
   └─ jose@npm:4.15.5 (via npm:^4.15.4)

I'm using Postgres

from backstage.

Do you do anything special with configuring the auth.identityTokenAlgorithm in app-config.yaml?

from backstage.

nope, nothing

from backstage.

@benjdlambert do you have any idea what could be wrong ?

from backstage.

btw, I needed to override okta provider, so it looks like this

authProviders.registerProvider({
          providerId: 'okta',
          factory: createOAuthProviderFactory({
            authenticator: oktaAuthenticator,
            signInResolverFactories: {
              emailLocalPartMatchingUserEntityName:
                catalogNotFoundFallbackResolverFactory(
                  commonSignInResolvers.emailLocalPartMatchingUserEntityName(),
                ),
            },
          }),
        });

from backstage.

@benjdlambert I see that PluginTokenHandler during initialization sets up ES256 algorithm and no possibility to change that, not sure what was supposed, I see previously HS256 was used.

from backstage.

@IlyaSavich would it be possible to dig around and see if you can get a stack trace for the "Key for the ES256 algorithm must be one of type KeyObject or CryptoKey. Received an instance of Uint8Array" error? That'd help narrow things down quite a lot because there are a couple of different places it could be happening.

The switch from HS256 to ES256 is intentional, although it could be related

from backstage.

@Rugvip

TypeError: Key for the ES256 algorithm must be one of type KeyObject or CryptoKey. Received an instance of Uint8Array
    at asymmetricTypeCheck (/Users/isavich/projects/productivity/backstage/node_modules/jose/dist/node/cjs/lib/check_key_type.js:17:15)
    at checkKeyType (/Users/isavich/projects/productivity/backstage/node_modules/jose/dist/node/cjs/lib/check_key_type.js:44:9)
    at flattenedVerify (/Users/isavich/projects/productivity/backstage/node_modules/jose/dist/node/cjs/jws/flattened/verify.js:78:37)
    at compactVerify (/Users/isavich/projects/productivity/backstage/node_modules/jose/dist/node/cjs/jws/compact/verify.js:18:60)
    at Object.jwtVerify (/Users/isavich/projects/productivity/backstage/node_modules/jose/dist/node/cjs/jwt/verify.js:8:58)
    at ServerTokenManager.authenticate (/Users/isavich/projects/productivity/backstage/node_modules/@backstage/backend-common/src/tokens/ServerTokenManager.ts:190:19)
    at AuthCompat.authenticate (/Users/isavich/projects/productivity/backstage/node_modules/@backstage/backend-common/src/auth/createLegacyAuthAdapters.ts:107:30)
    at HttpAuthCompat.extractCredentialsFromRequest_fn (/Users/isavich/projects/productivity/backstage/node_modules/@backstage/backend-common/src/auth/createLegacyAuthAdapters.ts:208:23)
    at HttpAuthCompat.getCredentials_fn (/Users/isavich/projects/productivity/backstage/node_modules/@backstage/backend-common/src/auth/createLegacyAuthAdapters.ts:213:7)
    at HttpAuthCompat.credentials (/Users/isavich/projects/productivity/backstage/node_modules/@backstage/backend-common/src/auth/createLegacyAuthAdapters.ts:223:31)
    at <anonymous> (/Users/isavich/projects/productivity/backstage/node_modules/@backstage/plugin-catalog-backend/src/service/createRouter.ts:201:39)
    at handleReturn (/Users/isavich/projects/productivity/backstage/node_modules/express-promise-router/lib/express-promise-router.js:24:23)
    at /Users/isavich/projects/productivity/backstage/node_modules/express-promise-router/lib/express-promise-router.js:64:7
    at handleReturn (/Users/isavich/projects/productivity/backstage/node_modules/express-promise-router/lib/express-promise-router.js:24:23)
    at /Users/isavich/projects/productivity/backstage/node_modules/express-promise-router/lib/express-promise-router.js:64:7
    at Layer.handle [as handle_request] (/Users/isavich/projects/productivity/backstage/node_modules/express/lib/router/layer.js:95:5)

from backstage.

@Rugvip @benjdlambert something wrong happens on ServerTokenManager:190, it looks like it's written the way it expects to work with symmetrical algorithms, while PluginTokenHandler is configured with default ES256 which leads to constant fail.

Not sure if I miss some configuration, but I didn't find any customizable place

from backstage.

@Rugvip @benjdlambert maybe this place is something where something strange happens DefaultAuthService:140

    const targetSupportsNewAuth =
      await this.pluginTokenHandler.isTargetPluginSupported(targetPluginId);

Inside I see it checks for keys in response

const res = await fetch(
          `${await this.discovery.getBaseUrl(
            targetPluginId,
          )}/.backstage/auth/v1/jwks.json`,
        );

but it doesn't check for empty array. I guess there is the issue

from backstage.

@IlyaSavich ServerTokenManager:190 is a red herring, it's actually :223 where the error happens, don't know why the sourcemaps are off :/

So yep this is clearly a token from the new auth service being validated by the old token manager. Do you happen to be using a mix of migrated backend plugins and some installed via the legacyPlugin helper? I suspect that might trigger this issue.

from backstage.

@IlyaSavich about the isTargetPluginSupported, it needs to consider an empty array valid since the array will be empty if the plugin hasn't made any requests yet, because it hasn't generated any keys. It's for sure a potential solution though. Gonna have a think on how best to solve this, but I think we've figured out the issue.

from backstage.

oh, yeah, I have some plugins not migrated yet and catalog is one of them. Ok, I will try to migrate it, hope it will help :)

from backstage.

Suggested a fix in #24676

from backstage.