env variables aren't populated even though process.env exists about roadmap HOT 12 CLOSED

Astro users struggle with environment variables so I am eager to arrive on a solution as well. I want to bring the full picture into the conversation first.

We have dev, build, and runtime to care of.

• dev - easy enough: we fill process.env same as import.meta.env
• build - same as dev but also some variables are statically replaced with the values, some with process.env.*
• runtime - this is where I think we have people with varying needs. "What's their .env.* convention?" "Is there even going to be a filesystem?" "Will the process global exist?" "Can we make our solution behave the same as dev for the most part?"

I am currently of the opinion that we should let the adapter - if not the user themselves - make the decision to add to process.env. import.meta.env is astro/vite's jurisdiction, process.env is not.

from roadmap.

@pi0 I don't find your comment sounding strong, and same here, trying to have a healthy discussion 🙂

About import.meta, I don't mean that Vite is setting a baseline or "taking-over" it though. I mean import.meta.env specifically which Vite "claims" and uses it for its own purpose.

I don't think it's likely we'll see import.meta.env taking adoption in runtimes too because (as mentioned) import.meta contains metadata for the current ES module. Env vars are fundamentally information of the overall program and not per-module. Hence, Vite has gotten criticism about import.meta.env in the past which I can understand.

if we can do std-envsupport any better for Astro users (like having an import condition or even specific logic), i am open to it. What we make in unjs, aims to be usable for more people not to make barriers.

I think IMO std-env is rightfully doing its best to acquire env vars as a dependency. But if we go down to a vanilla Node.js script standpoint, it's not always possible to assume .env files will be loaded. Hence, I don't think std-env can always be 100% confident that env vars can be acquired, unless it loads .env files itself. (I guess this is my only suggestion)

I don't think there's something std-env can do to support Astro better at the moment (besides the above suggestion). For SvelteKit as well, it's not possible unless you import SvelteKit-specific modules (but that's risky). And I think frameworks can rightfully do so too because the scope of env vars extend much beyond than process.env.*, e.g. you have a matrix of public/private static/runtime env vars.

from roadmap.

Some ecosystem examples that fill process.env from .env file:

• dotenv with 36M monthly downloads populates process.env.
• Node.js 20.6 with --env-file flag, populates process.env
• Bun, fills proces.env (docs)
• Deno fills their native way of accessing env variables Deno.env with it which means in Node.js compact mode, process.env is also filled

Vite, uses import.meta.env which I guess is understandable since vite aim to avoid Node.js API dependency but still fills their standard place.

std-env supports them all, when there is no single standard and surprisingly now I see even more.

I think for sake of ecosystem compatibility for Astro, it really makes sense that if process.env is available, it at least consistently works with rest.

from roadmap.

import.meta.env is a Vite feature, or bundler-specific if they implement it.

No, import.meta is a generic context for JavaScript modules.

Vite and also runtimes like jiti we already adopted it to help migration from Node.js process.env API which i guess similar merits of making a standard.

Yes, I suppose there is no need from Node.js to endorse a new convention of import.meta.env because they already have one. Unless we push to make a neural alternative like import.meta.env, it won't happen and I am thankful for Vite that does it.

But it works in Vite today because Vite is a bundler. It may unintentionally work in dependencies today, but we don't want to encourage libraries built to be Vite-specific, they should be agnostic.

Again, a love Vite but I don't belive Vite alone can set a baseline like this. import.meta is not a Vite specific API, to set boundaries for, neither import.meta was designed for Bundlers, it is designed for JavaScript modules.

For example, std-env won't work in Vite SSR dev today because Vite doesn't load .env files into process.env.*. It also only injects import.meta.env = {} to the top of every file that Vite processes, which excludes dependencies like std-env, unless you configure ssr.noExternal to tell Vite to handle it, which means you're opt-ing in to Vite-specific APIs.

Yes and that's probably a limitation for Vite and is solvable. Nuxt, uses vite and supports it. Your point is that dotenv does not works in Vite without configuration and I agree, yes today it won't.

I don't think every metaframework needs to make .env automatically work

Maybe, i am also not always fan of .env files but lets agree the main players (all runtimes i mentioned above), do + vite do support it.

@bluwy in the end, sorry if my comments sound strong. I think we are coming from different contexts and what i am chasing for std-env, is to find a baseline standard of accessing environment variables. JavaScript is strange enough and admitibly, solutions we make are just as silly as JS is. So if you like to have a chat in discord, i would love to ❤️

Also for Astro's sake, while probably it is totally not my position to say any comments or what is best or not, since process.env is available, and for the sake of ecosystem compatibility, and consistency, i belive @juliusmarminge is correct that process.env could be filled -- however again i might be totally missing part of a context, in which case, if we can do std-envsupport any better for Astro users (like having an import condition or even specific logic), i am open to it. What we make in unjs, aims to be usable for more people not to make barriers.

from roadmap.

Thanks for your points @bluwy.

Yes, we are mainly talking about "accessing environment variables" at least for std-env that initiated this issue. How they arrive (by OS, by platform ,or by a .env file that could be or not loaded) is another topic probably beyond this and more of a preference :)

Yes I also agree that import.meta.env is scoped to each module and i wish there were more better place for it but really there is no better-alternative that is not dependent on Node.js in entire JS universe. (globalThis is too risky to rely too) -- If you ever wanted to open a topic to imagine a better platform/runtime agnostic standard for accessing environment variables count me in!

As for std-env, it is a semi-standard solution until standard arrives and i think we are responsible to handle more edge cases like today how astro is filling the env sources) -- i think we have some ideas in upstream to try to always iterate all sources (unjs/std-env#115 (comment)) as a middle-ground to make it working with Astro 👍🏼

from roadmap.

If the library requires process.env.SOMETHING, shouldn't its documentation mention that users have to set that when using the library? Astro doesn't document populating the process.env.* from .env files. process.env.* is a standard Node.js feature too and not all tooling needs to load-up the values from .env files.

The blessed way Astro supports env vars is through import.meta.env.*. Even so, libraries shouldn't rely on this because it's a Vite/Astro specific feature. Libraries should use an agnostic way for env vars, e.g. process.env.* or via parameters.

I don't think this is a bug in Astro. Maybe there's a confusion with how env vars work here that we could improve on though.

from roadmap.

Libraries should use an agnostic way for env vars, e.g. process.env.* or via parameters.

Yes, that's what we do. We use process.env.XXX_API_KEY (but also provide an option to override it like {config: { apiKey: import.meta.env.XXX_API_KEY }}). We even use std-env to support accessing process.env.XXX_API_KEY without hassle.

For "normal" Vite frameworks, this works since they only have import.meta.env and no process so this check makes us look at the import.meta.env.XXX_API_KEY even if we access process.env.XXX_API_KEY in the library:

https://github.com/unjs/std-env/blob/main/src/env.ts#L6-L7

We'd ultimately like to offer Astro user a good experience with 0 config (just have the variable set in your .env) but that's just not possible without heavy special treatments to resolve the variable

from roadmap.

std-env should definitely not use import.meta.env, but I guess that's a bit too late now. I think the way to support Astro is to pass it through the config explicitly, and I'm not really sure if it's cumbersome as it's only one extra step after setting in .env? (that is adding apiKey: import.meta.env.XXX_API_KEY)

We can definitely look into a feature request to populate process.env.*, but having multiple ways to access things is probably not ideal. There's also discussions that we might move to how SvelteKit handles env vars, and given that it's likely that std-env won't work in SvelteKit too.

from roadmap.

std-env should definitely not use import.meta.env

wondering why you belive this?

from roadmap.

The environment story definitely needs to be revisited soon. However, should automatically adding things to process.env be the ideal solution?

from roadmap.

However, should automatically adding things to process.env be the ideal solution?

Yes - it's literally the most used in the industry...

from roadmap.

wondering why you belive this?

import.meta.env is a Vite feature, or bundler-specific if they implement it. AFAIK I don't see it endorsed by Node.js or the spec. Similarly import.meta.glob doesn't always work in dependencies unless you specifically configure Vite to process that file.

import.meta.* in essence is also metadata for the current file. You can't initialize import.meta.env = {} in one file, and access the values in another. But it works in Vite today because Vite is a bundler. It may unintentionally work in dependencies today, but we don't want to encourage libraries built to be Vite-specific, they should be agnostic.

For example, std-env won't work in Vite SSR dev today because Vite doesn't load .env files into process.env.*. It also only injects import.meta.env = {} to the top of every file that Vite processes, which excludes dependencies like std-env, unless you configure ssr.noExternal to tell Vite to handle it, which means you're opt-ing in to Vite-specific APIs.

I think for sake of ecosystem compatibility for Astro, it really makes sense that if process.env is available, it at least consistently works with rest.

I don't think every metaframework needs to make .env automatically work. It's a choice the user can opt into too, either via dotenv, or the JS runtime like you mentioned. Astro could add it (not that I'm proposing), but I don't think it's feasible to request every metaframework to do it.

from roadmap.