Comments (10)
Can you please post the dag or a minimal example to reproduce this?
from airflow.
I am unable to reproduce this locally on main branch with the given dag code. The dag is scheduled hourly for me. Since this is a common cron expression I am wondering if there is something else wrong specific to the installation.
python -m airflow dags details gh39435
property_name | property_value
================================+=======================================================================================
dag_id | gh39435
dag_display_name | gh39435
root_dag_id | None
is_paused | False
is_active | True
is_subdag | False
last_parsed_time | 2024-05-06T16:26:38.541571+00:00
last_pickled | None
last_expired | None
scheduler_lock | None
pickle_id | None
default_view | grid
fileloc | /home/karthikeyan/airflow/dags/gh39435.py
file_token | Ii9ob21lL2thcnRoaWtleWFuL2FpcmZsb3cvZGFncy9naDM5NDM1LnB5Ig._hmr8O0KN7-ndOioQCJjf0dqsfo
owners | ggomiz
description | None
schedule_interval | {'__type': 'CronExpression', 'value': '0 * * * *'}
timetable_description | Every hour
tags | {'name': 'dbservers'},{'name': 'redis'},{'name': 'backups'}
max_active_tasks | 16
max_active_runs | 16
max_consecutive_failed_dag_runs | 0
has_task_concurrency_limits | False
has_import_errors | False
next_dagrun | 2024-05-06T16:00:00+00:00
next_dagrun_data_interval_start | 2024-05-06T16:00:00+00:00
next_dagrun_data_interval_end | 2024-05-06T17:00:00+00:00
next_dagrun_create_after | 2024-05-06T17:00:00+00:00
from airflow.
@Taragolis, setting TZ env var to "America/Buenos_Aires", as described in #37869 worked perfectly.
I've had already in docker compose file set the following env vars:
AIRFLOW__CORE__DEFAULT_TIMEZONE: 'America/Buenos_Aires'
AIRFLOW__WEBSERVER__DEFAULT_UI_TIMEZONE: 'America/Buenos_Aires'
but didn't know that also must set TZ env var.
Weird thing is that on airflow version 2.7.3 worked ok without setting TZ env var.
Thanks for your help!
from airflow.
#!/usr/bin/python3
from airflow import DAG
from datetime import datetime
from airflow.operators.bash import BashOperator
from airflow.hooks.base import BaseHook
default_args = {
'owner': 'ggomiz',
'depends_on_past': False,
'start_date': datetime(2024,5,6),
'email': ['[email protected]'],
'email_on_failure': True,
'email_on_retry': False
}
# hourly redis dag
dag_hourly = DAG(dag_id='redis_test', default_args=default_args, schedule='0 * * * *', catchup=False, tags=["dbservers", "backups", "redis"])
task_args = {
'retries': 0,
'trigger_rule': 'all_done'
}
# hourly backup task
task_daily = BashOperator(task_id="test_redis_master",
bash_command="cronjob.sh -q -u root /srv/redis/scripts/redisbackup.sh master",
dag=dag_hourly, queue = "galadriel", default_args=task_args)
This minimal DAG to make an hourly backup of our redis database is getting incorrect schedule.
from airflow.
As additional information, I've several other DAGs with hourly schedule definition, but also are incorrectly scheduled on a 4 hour interval, but they are not executing. Last run time is before the upgrade.
from airflow.
@GabrielGomiz Could you provide also information about your timezones configurations: in OS and in Airflow Configuration?
from airflow.
@GabrielGomiz Could you provide also information about your timezones configurations: in OS and in Airflow Configuration?
Ok. We're in Argentina, so timezone is -3 ART as you can see in the timestamps above: "2024-05-06T12:00:00-03:00"
Inside the docker containers TZ is -3 ART also:
[/opt/airflow] [docker] airflow > date
Mon May 6 18:52:14 -03 2024
from airflow.
I more interested in values which are setting up, not a result. Invalid configuration may lead to some errors, e.g. Airflow expected to work with IANA timezones, if OS misconfigured it also might return invalid values.
from airflow.
This might be similar to this one #37869 (comment) or it could be something new and different, without understand what is your setup it is hard to guess what is going wrong here, because I could also confirm the fact, that with default configurations it is works without any issues.
from airflow.
Ok, I'll look into timezone configuration and report back asap. Thanks @Taragolis
from airflow.
Related Issues (20)
- Use custom port makes FTP connection fail HOT 4
- `dag.test` compatible with local executors
- Status of testing of Apache Airflow Helm Chart 1.14.0rc1 HOT 4
- Support logs pulling in `KubernetesJobOperator`
- Incorrect run gets started HOT 4
- Airflow Dags not visible in the UI -2.5.3,Dag File processor was not visible in the logs HOT 1
- Add annotations for Redis StatefulSet resource in values.yaml is not possible HOT 1
- Number of queries to Airflow database in "DAG File Processing Stats" HOT 6
- *** Could not read served logs: [Errno -3] Temporary failure in name resolution HOT 1
- execution_date format HOT 2
- Handle toctree contains reference to nonexisting document `_api/` errors better when building docs HOT 2
- scheduler_job_runner starving log additional information HOT 1
- OpenLineage defaults in airflow.cfg incorrect HOT 5
- Add deferrable support to `LambdaInvokeFunctionOperator` HOT 10
- OpenLineage Provider issue: scheduler shuts down after pickle of OpenLineageListener fails HOT 5
- Version 2.9.2 airflow db migrate unsupported queries for mysql HOT 8
- Mismatched Xcom Map Index when Dynamic Mapping over TaskGroup and not all mapped tasks have run
- After deleting a dag from the dags directory, it is still displayed in the UI HOT 3
- Syntax highlight and copy button for entries in XCom tab
- Deferred task behaviour is different from normal task HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airflow.