astronomer / airflow-chart Goto Github PK
View Code? Open in Web Editor NEWA Helm chart to install Apache Airflow on Kubernetes
License: Other
A Helm chart to install Apache Airflow on Kubernetes
License: Other
Facing issue while using CeleryExecutor.
while the working is launching, its crashing with the below error:
PermissionError: [Errno 13] Permission denied: '/usr/airflow/logs/scheduler'
can we have a PVC or something?
postgres is stuck in a loop:
postgresql 15:52:19.71 INFO ==> ** Starting PostgreSQL **
2020-03-27 15:52:20.199 GMT [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-03-27 15:52:20.199 GMT [1] LOG: listening on IPv6 address "::", port 5432
2020-03-27 15:52:20.312 GMT [1] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2020-03-27 15:52:21.240 GMT [140] LOG: database system was interrupted; last known up at 2020-03-27 15:52:18 GMT
2020-03-27 15:52:33.866 GMT [147] FATAL: the database system is starting up
2020-03-27 15:52:43.657 GMT [154] FATAL: the database system is starting up
2020-03-27 15:52:53.223 GMT [160] FATAL: the database system is starting up
2020-03-27 15:52:53.669 GMT [167] FATAL: the database system is starting up
2020-03-27 15:53:03.209 GMT [173] FATAL: the database system is starting up
2020-03-27 15:53:03.676 GMT [180] FATAL: the database system is starting up
2020-03-27 15:53:11.002 GMT [140] LOG: database system was not properly shut down; automatic recovery in progress
2020-03-27 15:53:11.398 GMT [140] LOG: invalid record length at 0/1000178: wanted 24, got 0
2020-03-27 15:53:11.398 GMT [140] LOG: redo is not required
2020-03-27 15:53:12.890 GMT [1] LOG: database system is ready to accept connections
2020-03-27 15:53:12.890 GMT [184] FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
2020-03-27 15:53:12.891 GMT [186] FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
2020-03-27 15:53:13.000 GMT [1] LOG: autovacuum launcher process (PID 184) exited with exit code 1
2020-03-27 15:53:13.000 GMT [1] LOG: terminating any other active server processes
2020-03-27 15:53:13.000 GMT [1] LOG: background worker "logical replication launcher" (PID 186) exited with exit code 1
2020-03-27 15:53:13.154 GMT [1] LOG: all server processes terminated; reinitializing
2020-03-27 15:53:13.537 GMT [187] LOG: database system was interrupted; last known up at 2020-03-27 15:53:12 GMT
2020-03-27 15:53:14.086 GMT [193] FATAL: the database system is in recovery mode
2020-03-27 15:53:14.178 GMT [200] FATAL: the database system is in recovery mode
2020-03-27 15:53:23.215 GMT [206] FATAL: the database system is in recovery mode
2020-03-27 15:53:23.660 GMT [213] FATAL: the database system is in recovery mode
This is 1.15.9 and a plain vanilla install via helm3 into namespec airflow:
helm install airflow --namespace airflow astronomer/airflow
postgres will shutdown and then the container will restart and the cycle repeats. There is a postgres create user job that is forever stuck as well since postgres never appears ready.
Hello π§βπ folks,
While using this chart, I'm trying to configure Google Cloud Storage remote logging: https://airflow.apache.org/docs/apache-airflow-providers-google/stable/logging/gcs.html
As per the documentation says, I need to:
a. Tweak the Airflow config to pass the proper params (Service Account PATH, GCS bucket name, ...)
b. Write the JSON credentials Key of a Google Service Account alongside Airflow (e.g. in the Airflow Pods).
To configure Airflow (a.), it's quite simple, I use the Airflow config env variables (AIRFLOW__LOGGING__XXXX
) that I inject in this Chart through the .Values.env
value. π
However, I don't see any way of writing the Google Service Account credentials JSON file into the Airflow Pods. I currently have the credentials file stored into a Kubernetes secret and I would like to inject it as a file into Airflow Pods.
Things that I tried but didn't work:
1/ I found ways to bind Kubernetes Secrets to env variables with this Chart (.Values.secrets[]
) but I need a file, not an Env variable (Google auth libs wants to have the credentials into Files, not Env vars, see: https://cloud.google.com/docs/authentication/production#automatically).
2/ I saw that there was a way to mount files into the worker and scheduler (.Values.workers.extraVolumeMounts
/ .Values.workers.extraVolumes
/ .Values.scheduler.extraVolumeMounts
/ .Values.scheduler.extraVolumes
) but I don't see it possible for the webserver
and as I want the Airflow UI to fetch the logs from GCS when reading a task log, I guess that it won't work (Airflow would write the logs but wouldn't be able to read them)
Did I miss anything?
If needed I can open a PR to propose a way to mount any volume, such as .Values.extraVolumeMounts
/ .Values.extraVolumes
.
tried this chart using k3d
scheduler, worker and webserver are unable to start
stuck waiting for container wait-for-airflow-migrations
, which fails with this error
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) ERROR: No such database: airflow-metadata
after some investigation, i see that the issue is the value for pgbouncer.ini
in the airflow-pgbouncer-config
secret.
here's what it looks like:
airflow-metadata = host=<no value> dbname=postgres port=5432 pool_size=3
airflow-result-backend = host=<no value> dbname=postgres port=5432 pool_size=2
[pgbouncer]
pool_mode = transaction
listen_port = 6543
server_tls_sslmode = prefer
listen_addr = *
auth_type = md5
auth_file = /etc/pgbouncer/users.txt
stats_users = postgres
ignore_startup_parameters = extra_float_digits
max_client_conn = 100
verbose = 0
log_disconnections = 0
log_connections = 0
the problem is the host=<no value>
if we look up the IP for the pgbouncer service and update the secret, it resolves the issue.
i am not sure if this is specific to k3d.
Immediate failure when running the following command:
helm install . --generate-name --debug
Error: found in Chart.yaml, but missing in charts/ directory: postgresql
helm.go:75: [debug] found in Chart.yaml, but missing in charts/ directory: postgresql
helm.sh/helm/v3/pkg/action.CheckDependencies
/home/circleci/helm.sh/helm/pkg/action/install.go:608
main.runInstall
/home/circleci/helm.sh/helm/cmd/helm/install.go:193
main.newInstallCmd.func1
/home/circleci/helm.sh/helm/cmd/helm/install.go:116
github.com/spf13/cobra.(*Command).execute
/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
main.main
/home/circleci/helm.sh/helm/cmd/helm/helm.go:74
runtime.main
/usr/local/go/src/runtime/proc.go:203
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1357
Hello,
Astronomer helm chart version: 0.25.8
////// Config
astronomer:
astroUI:
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "1024Mi"
houston:
resources:
requests:
cpu: "500m"
memory: "1024Mi"
limits:
cpu: "1000m"
memory: "2048Mi"
config:
executor: "CeleryExecutor"
publicSignups: true
email:
enabled: true
reply: "[email protected]"
deployments:
helm:
workers:
additionalVolume:
enabled: true
capacity: 100Gi
storageClassName: "efs"
mountPath: "/data"
volumePlugin: {}
And I get the following error in houston pod logs:
astronomer-houston-upgrade-deployments-b7wk7 post-upgrade-job 2021-10-13T17:58:25 INFO Response from #updateDeployment: {"result":{"message":"YAML parse error on airflow/templates/workers/worker-persistentvolume.yaml: error converting YAML to JSON: yaml: line 20: did not find expected key"},"deployment":{}}
https://github.com/astronomer/airflow-chart/blob/v0.20.1/templates/workers/worker-persistentvolume.yaml -> this is the template that throws the error. Am I doing something wrong or is the template the problem?
Chart assumes clients will only want to add environment variables as name/values.
Client won't be able to do something like
- name: ENV_VAR
valueFrom:
configMapKeyRef:
name: configmap-name
key: configmap-field
Is it possible to use this chart and astronomerinc/ap-airflow
image without an astronomer license?
I believe privateRegistry.*
is no longer used anywhere. It looks like 887a4c0 removed it from the templates.
This might be a simple fix, but what should I alter in the chart/templates to be able to access the task logs in the Airflow Web UI?
Currently using the default KubernetesExecutor and getting a "Log file does not exist:" error. Also not seeing a Persistent Volume Claims for the workers?
The markup version of the readme that is displayed for the main page in this repo contains the following link:
Status code [404:NotFound] - Link: https://github.com/astronomer/astronomer/tree/master/docker/airflow
(The link in the readmeβs raw markdown may appear in a different format to the link above
This bad link was found by a tool I very recently created as part of an new experimental hobby project: https://github.com/MrCull/GitHub-Repo-ReadMe-Dead-Link-Finder
I (a human) verified that this link is broken and have manually logged this Issue (i.e. this Issue has not been created by a bot).
If this has been in any way helpful then please consider giving the above Repo a Star.
If you have any feedback on the information provided here, or on the tool itself, then please feel free to share your thoughts and pass on the feedback.
How could I add Spark binaries to use SparkSubmitOperator?
Should the default imagePullPolicy for all images be Always
to enable a more secure default installation?
Doing a simple helm install with KEDA and a custom image (simple astro dev init) as specified in the README ends with scheduler erroring like this:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 91, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py", line 64, in scheduler
job.run()
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/base_job.py", line 245, in run
self._execute()
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1305, in _execute
self._run_scheduler_loop()
File "/usr/local/lib/python3.7/site-packages/astronomer/airflow/version_check/plugin.py", line 29, in run_before
fn(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1398, in _run_scheduler_loop
num_queued_tis = self._do_scheduling(session)
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1494, in _do_scheduling
self._create_dagruns_for_dags(guard, session)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/retries.py", line 76, in wrapped_function
for attempt in run_with_db_retries(max_retries=retries, logger=logger, **retry_kwargs):
File "/usr/local/lib/python3.7/site-packages/tenacity/__init__.py", line 390, in __iter__
do = self.iter(retry_state=retry_state)
File "/usr/local/lib/python3.7/site-packages/tenacity/__init__.py", line 356, in iter
return fut.result()
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/local/lib/python3.7/site-packages/airflow/utils/retries.py", line 85, in wrapped_function
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1585, in _create_dagruns_for_dags
self._create_dag_runs(query.all(), session)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3373, in all
return list(self)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column dag.last_parsed_time does not exist
LINE 1: ...AS dag_is_subdag, dag.is_active AS dag_is_active, dag.last_p...
^
[SQL: SELECT dag.dag_id AS dag_dag_id, dag.root_dag_id AS dag_root_dag_id, dag.is_paused AS dag_is_paused, dag.is_subdag AS dag_is_subdag, dag.is_active AS dag_is_active, dag.last_parsed_time AS dag_last_parsed_time, dag.last_pickled AS dag_last_pickled, dag.last_expired AS dag_last_expired, dag.scheduler_lock AS dag_scheduler_lock, dag.pickle_id AS dag_pickle_id, dag.fileloc AS dag_fileloc, dag.owners AS dag_owners, dag.description AS dag_description, dag.default_view AS dag_default_view, dag.schedule_interval AS dag_schedule_interval, dag.concurrency AS dag_concurrency, dag.has_task_concurrency_limits AS dag_has_task_concurrency_limits, dag.next_dagrun AS dag_next_dagrun, dag.next_dagrun_create_after AS dag_next_dagrun_create_after
FROM dag
WHERE dag.is_paused IS false AND dag.is_active IS true AND dag.next_dagrun_create_after <= now() ORDER BY dag.next_dagrun_create_after
LIMIT %(param_1)s FOR UPDATE OF dag SKIP LOCKED]
[parameters: {'param_1': 10}]
(Background on this error at: http://sqlalche.me/e/13/f405)
Of course in normal airflow a simple airflow db upgrade
would suffice but this is k8s where I cannot just run a command in the pod and that will persist
Hello! Thanks for helm for airflow
helm repo add astronomer https://helm.astronomer.io
helm install --set airflowVersion=1.10.14,defaultAirflowTag=1.10.14-buster,ingress.enabled=true,ingress.baseDomain=192.168.22.7.sslip.io airflow astronomer/airflow
logs
kubectl logs airflow-run-airflow-migrations-g58jq
Waiting for host: airflow-postgresql.default.svc.cluster.local. 5432
DB: postgresql://postgres:***@airflow-postgresql.default.svc.cluster.local.:5432/postgres?sslmode=disable
[2021-03-19 05:21:13,487] {db.py:378} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
yield
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 364, in _upgrade_revs
revs = list(revs)
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 765, in _iterate_revisions
requested_lowers = self.get_revisions(lower)
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 319, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 319, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 324, in get_revisions
for rev_id in resolved_id
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 324, in <genexpr>
for rev_id in resolved_id
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 397, in _revision_for_ident
resolved_id,
alembic.script.revision.ResolutionError: No such revision or branch 'e959f08ac86c'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 37, in <module>
args.func(args)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 81, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 1681, in upgradedb
db.upgradedb()
File "/usr/local/lib/python3.7/site-packages/astronomer/airflow/version_check/plugin.py", line 29, in run_before
fn(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 386, in upgradedb
command.upgrade(config, 'heads')
File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 97, in <module>
run_migrations_online()
File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 91, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 511, in run_migrations
for step in self._migrations_fn(heads, self):
File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 287, in upgrade
return script._upgrade_revs(revision, rev)
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 369, in _upgrade_revs
for script in reversed(list(revs))
File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 194, in _catch_revision_errors
compat.raise_from_cause(util.CommandError(resolution))
File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 308, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 301, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
yield
File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 364, in _upgrade_revs
revs = list(revs)
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 765, in _iterate_revisions
requested_lowers = self.get_revisions(lower)
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 319, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 319, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 324, in get_revisions
for rev_id in resolved_id
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 324, in <genexpr>
for rev_id in resolved_id
File "/usr/local/lib/python3.7/site-packages/alembic/script/revision.py", line 397, in _revision_for_ident
resolved_id,
alembic.util.exc.CommandError: Can't locate revision identified by 'e959f08ac86c'
KEDA v2 supports not only Deployment
but also StatefulSet
. PR #731 Currently, this chart forces users to use deployment for Airflow workers instead of statefulsets. Is there any plan on auto-scaling Airflow workers as statefulsets via KEDA v2? If not can I contribute to this Chart?
When running this chart on its own, it worked great for me as expected. However, when I defined it as a dependency in my Chart.yaml, the airflow containers fail to start up correctly, and I get something like this:
It seems to be related to this helm secret yaml file, but beyond that I'm having trouble debugging. I tried specifying different values for fernetKeySecretName
and fernetKey
in the helm values config, but haven't had any luck getting passed this.
Any suggestions?
Hi Folks,
We've been evaluating Apache Airflow and are going ahead with it :)
I recently came across @schnie post on dev@airflow so started looking at this repository. You guys have done an EXCELLENT job! kudos.
In parallel with having our SRE prototype installation into our test env, I wanted our developers to be able to write, deploy and test their own DAG's even before they interact with the test env. The following therefore documents what I had to do to get this working.
Install astronomer/airflow helm chart on docker-deskop (Kubernetes)
brew install helm
kubectl create namespace airflow
helm repo add astronomer https://helm.astronomer.io
helm repo update
helm install airflow astronomer/airflow
kubectl get pods --all-namespaces
helm list -n airflow
kubectl port-forward svc/airflow-webserver 8080:8080 -n airflow
# access Airflow dashboard via localhost:8080
Deploy container DAG's into Airflow
brew install astronomer/tap/astro
mkdir sample-airflow-project && cd sample-airflow-project
astro dev init
docker build -t sample-airflow-dag:0.0.1 .
helm upgrade airflow -n airflow --set images.airflow.repository=sample-airflow-dag --set images.airflow.tag=0.0.1 astronomer/airflow
kubectl port-forward svc/airflow-webserver 8080:8080
#access Airflow dashboard via localhost:8080
I'm not sure what you guys want to do with this info... maybe a wiki entry?
When I check for logs by attempts, I get this, anyone have any idea on this?
its not persisting i think ,
*** Log file does not exist: /usr/local/airflow/logs/candidate_sharing/train/2020-07-31T08:55:14.478919+00:00/1.log
*** Fetching from: http://candidatesharingtrain-38b4497e1ac045b683f6b0bd2ce15d23:8793/log/candidate_sharing/train/2020-07-31T08:55:14.478919+00:00/1.log
*** Failed to fetch log file from worker. HTTPConnectionPool(host='candidatesharingtrain-38b4497e1ac045b683f6b0bd2ce15d23', port=8793): Max retries exceeded with url: /log/candidate_sharing/train/2020-07-31T08:55:14.478919+00:00/1.log (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdbf1227990>: Failed to establish a new connection: [Errno -2] Name or service not known'))
I have used the secret field in value like this:
secret:
- envName: BH_DB_HOST
secretName: airflow-secrets
secretKey: BH_DB_HOST
- envName: BH_DB_NAME
secretName: airflow-secrets
secretKey: BH_DB_NAME
I'm using this: https://github.com/astronomer/ap-airflow/tree/master/1.10.10
and it gives me this kind of error in the scheduler,
any idea of this?
According to README.md#parameters, secret
name/key pairs can be mount into Airflow pods.
Did not a find a way to do the same with imagePullSecret
key.
Since, I was not able to set imagePullSecret
key as a parameter and, in order to make pods Pull an Image from a Private Registry, I had to manually add image pull secret to all Airflow K8s service accounts.
This implies, in a "from-scratch" deployment:
helm install
to get all resources deployed (Helm release status: Failed)ImagePullBackOff
status so they are recreated with imagePullSecret
key correctly sethelm upgrade
to get a successful Helm releaseimagePullSecret
name/key pairs can be mount into Airflow pods as a parameter, exactly like secret
name/key pairs.
This would prevent users from opening up and editing current Helm charts.
I'm using : https://github.com/astronomer/ap-airflow/tree/master/1.10.12
Seems like one more column it needs to create?
Logs from my scheduler:
Hi,
i have a nicely working airflow cluster using your chart, its running in GKE, in the same namespace i run the Elasticsearch helm chart from elastic. Both work nicely, and can talk to each other.
This is the relevant yaml part
elasticsearch:
enabled: true
connection:
host: elasticsearch-master
port: 9200
But i dont see the logs, could you point me to what i am doing wrong?
Hi Astronomer Team,
We are deploying this chart in our K8 cluster to run airflow. We are using nginx proxy to route to airflow. All pods & services are running. But while accessing the airflow URL we are getting Airflow 404 = Lots of circles Error. Is this because we have not set ingress.baseDomain while installing chart?
Please help.
Thanks,
Kapil.
Hi I don't see any where in the values.yml file config for gitSync. But in the image in the scheduler a gitSync sidecar is running.
What are the differences with Helm's official chart and apache/airflow-on-k8s-operator in terms of the architecture/design?
What are the pros and cons other than helm support?
Running:
helm install airflow \
--set ingress.enabled=true \
--namespace airflow \
astronomer/airflow
returns:
Error: Ingress.extensions "airflow-airflow-ingress" is invalid: [spec.rules[0].host: Invalid value: "deployments.%!s(<nil>)": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.rules[1].host: Invalid value: "airflow-airflow.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts: Invalid value: "deployments.%!s(<nil>)": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[1].hosts: Invalid value: "airflow-airflow.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]
Am I missing something?
Hi! I'm trying to upgrade to Airflow 2.0 with this chart. When I upgrade, I get an error:
airflow command error: argument GROUP_OR_COMMAND: invalid choice: 'serve_logs'
Here is where serve_logs
is getting invoked:
But AFAICT, this command does not exist in Airflow 2.0.
Am I missing something? Or is this chart not compatible with Airflow 2.0?
Thanks!
When I run "helm install airflow -n airflow astronomer/airflow" it fails saying waiting on condition.
When I run "helm install airflow -n airflow astronomer/airflow --debug --wait" I get the above error.
Any idea how to troubleshoot this?
Is it possible to sync from several repositories into the dags foder using git-sync, example below. The intention being, to have a sort of multi-tenent aiflow cluster, where dags can be deployed from several git repositories
extraContainers:
- name: gitsync-repo-1
image: k8s.gcr.io/git-sync/git-sync:v3.2.2
imagePullPolicy: IfNotPresent
env:
- name: GIT_SYNC_REPO
value: https://github.com/company/repo-1.git
- name: GIT_SYNC_ROOT
......
volumeMounts:
- mountPath: /usr/local/airflow/dags
name: dags
readOnly: false
extraVolumeMounts:
- name: dags
mountPath: /usr/local/airflow/dags
extraVolumes:
- name: dags
emptyDir: {}
- name: gitsync-repo-2
image: k8s.gcr.io/git-sync/git-sync:v3.2.2
imagePullPolicy: IfNotPresent
env:
- name: GIT_SYNC_REPO
value: https://github.com/company/repo-2.git
- name: GIT_SYNC_ROOT
......
volumeMounts:
- mountPath: /usr/local/airflow/dags
name: dags
readOnly: false
extraVolumeMounts:
- name: dags
mountPath: /usr/local/airflow/dags
extraVolumes:
- name: dags
emptyDir: {}
Hello! Thanks for helm airflow!
Please update airflow version to 1.10.14
I used commit short SHA to tag image. When the image tag is integer the image tag being passed to kubernetes resources are invalid.
Example:
I have image tag 5060547
helm template
helm template astronomer-main astronomer/airflow --set images.airflow.tag=5060547
Result:
...
containers:
- name: create-user
image: quay.io/astronomer/ap-airflow:%!s(int64=5060547)
...
I think it is because of this helper implementation
airflow-chart/templates/_helpers.yaml
Lines 113 to 115 in 90b0911
I tried following the instructions for keda install (installing locally in k3d)
I got this error:
β― helm install airflow \ <aws:edm-dstandish>
--set executor=CeleryExecutor \
--set workers.keda.enabled=true \
--set workers.persistence.enabled=false \
--namespace airflow \
astronomer/airflow
Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ScaledObject" in version "keda.k8s.io/v1alpha1"
Hello! Thanks for helm chart for airflow
I try install helm chart or airflow from astronomer
helm repo add astronomer https://helm.astronomer.io
helm repo update
helm install --set webserver.defaultUser.password=new_password,airflowVersion=2.0.0,defaultAirflowTag=2.0.0-buster,ingress.enabled=true,ingress.baseDomain=192.168.22.7.sslip.io airflow astronomer/airflow
I login to airflow as admin:admin.
But i think login as admin:new_password
Hello,
I noticed that several issues have been outstanding in this repo for several months. I was looking to adopt this helm chart. Are you all still maintaining this?
Otherwise, is there an alternative you can recommend to get started with?
We should investigate using helm-docs to generate our parameters table, which is out of date and does not include all of our parameters.
When I set webserver replicas to 2 or more. it gives a CSRF error.
any idea on this?
with the same, if I set replicas 1, it does work as it supposes to be.
Ingress exposing the airflow Web server UI to <SUB>.<DOMAIN>.COM/<RELEASE-NAME>/airflow
I wanted to set it to <SUB>.<DOMAIN>.COM/
instead,
I tried to do it by editing the ingress path, but it's still pointing to the same. (I have tried deleting and reinstalling the ingress object)
According to the README.md,
The recommended way to update your DAGs with this chart is to build a new docker image with the latest code
Would it be possible to update DAGs without building a new Airflow Docker image?
does the latest release support Airflow 2.0.0?
We have a specific requirement where in airflow DAG reads a file from S3 to determine how to tasks to instantiate, so trying to access S3 using RBAC, possibility to attach custom service accounts would make this easier.
scheduler:
logGroomerSidecar:
enabled: True
retentionDays: 7
args: ["/usr/local/bin/clean-airflow-logs"]
Setting the following I would expect the logGroomer to clean the logs of anything older than 7 days.
This translates to setting an envvar in the sidecar of AIRFLOW__LOG_RETENTION_DAYS
, however in /usr/local/bin/clean-airflow-logs
its looking for RETENTION=${ASTRONOMER__AIRFLOW__WORKER_LOG_RETENTION_DAYS:-15}
Is there a setting I'm missing to create the astronomer envvar for the logGroomerSidecar?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.