Comments (7)
There is exception handling for write_logs
:
def write_logs(self, pod: k8s.V1Pod):
try:
logs = self.pod_manager.read_pod_logs(
pod=pod,
container_name=self.base_container_name,
follow=False,
)
for raw_line in logs:
line = raw_line.decode("utf-8", errors="backslashreplace").rstrip("\n")
self.log.info("Container logs: %s", line)
except HTTPError as e:
self.log.warning(
"Reading of logs interrupted with error %r; will retry. "
"Set log level to DEBUG for traceback.",
e,
)
The error that bubbles up is an kubernetes.client.exceptions.ApiException
though... I wonder if this should be included in the exception handling?
from airflow.
I think simply handling the ApiException won't be enough. We should perform additional checks because during cleanup, we also read the pod.
from airflow.
Yes, I would like it but It's my first time so I reading how do it
from airflow.
Hello @RNHTTR ,
Yes, I belive that too. I am doing a PR including this exception in that point but It is my firts PR and it's taking me longer.
from airflow.
Something like this:
def write_logs(self, pod: k8s.V1Pod):
try:
logs = self.pod_manager.read_pod_logs(
pod=pod,
container_name=self.base_container_name,
follow=False,
)
for raw_line in logs:
line = raw_line.decode("utf-8", errors="backslashreplace").rstrip("\n")
self.log.info("Container logs: %s", line)
except (HTTPError, ApiException) as e:
self.log.warning(
"Reading of logs interrupted with error %r; will retry. "
"Set log level to DEBUG for traceback.",
e if not isinstance(e, ApiException) else e.reason,
)
from airflow.
Hi @pankajastro ,
Of course. I have added the following in the _clean:
def _clean(self, event: dict[str, Any]):
if event["status"] == "running":
return
istio_enabled = self.is_istio_enabled(self.pod)
# Skip await_pod_completion when the event is 'timeout' due to the pod can hang
# on the ErrImagePull or ContainerCreating step and it will never complete
if event["status"] != "timeout":
try:
self.pod = self.pod_manager.await_pod_completion(
self.pod, istio_enabled, self.base_container_name
)
except ApiException as e:
if e.status == 404:
self.pod = None
else:
raise e
if self.pod is not None:
self.post_complete_action(
pod=self.pod,
remote_pod=self.pod,
)
I have done this due to the await_pod_completion can throw the same ApiException than write_logs. Handling this exception not is necessary make the post_complete_action. What do you think?
from airflow.
would you like to create a PR?
from airflow.
Related Issues (20)
- Mismatched Xcom Map Index when Dynamic Mapping over TaskGroup and not all mapped tasks have run HOT 3
- After deleting a dag from the dags directory, it is still displayed in the UI HOT 3
- Syntax highlight and copy button for entries in XCom tab
- Deferred task behaviour is different from normal task HOT 1
- Request to Reintroduce Tasks Duration Multi Line Graph HOT 3
- The task was stuck, but when marking it as successful, it continues to run the remaining part of the job. HOT 2
- Mark Success but remote task continue running HOT 4
- Add on_kill to SSHOperator HOT 4
- Airflow scheduler crashes with a psycopg2.errors.GroupingError due to an incorrect SQL query when detecting orphaned datasets
- Support extraInitContainers for migrateDatabaseJob HOT 1
- ObjectStoragePath should be template-able HOT 2
- After deleting a dag from the dags directory, it is still displayed in the UI HOT 2
- values.yaml `dags.gitSync.containerLifecycleHooks` Does not working HOT 2
- Improve SFTPOperator with directory transfer and DELETE operation HOT 7
- add ower to TaskInstance class HOT 1
- Status of testing Providers that were prepared on June 22, 2024 HOT 38
- openlineage: Non-local executor's initializer breaking the Airflow DB connection HOT 6
- ODBC Provider Configuration
- admin canot see admin components HOT 2
- Azure Datalake Storage V2 ObjectStoragePath connection issues HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airflow.