[FEATURE] Print out the task shuffle write time statistics by table format in log about incubator-uniffle HOT 10 OPEN

Feel free to pick this up.

from incubator-uniffle.

Are you interested on this? @myandpr

from incubator-uniffle.

Are you interested on this? @myandpr

Fine，please assign to me, thanks @zuston !

from incubator-uniffle.

Should we use Spark's metrics instead of log?

from incubator-uniffle.

Should we use Spark's metrics instead of log?

This may be also valid for MR and Tez. BTW, the task metrics for low shuffle-server don't exist. I haven't seen this metric.

from incubator-uniffle.

Should we use Spark's metrics instead of log?

This may be also valid for MR and Tez. BTW, the task metrics for low shuffle-server don't exist. I haven't seen this metric.

This should be a metric instead of log. The log seems weird.

from incubator-uniffle.

So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want

from incubator-uniffle.

So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want

Spark metrics system allows user to add extra metrics. You can refer to https://simhadri-g.medium.com/custom-metrics-source-in-apache-spark-ca30a3b362dd

from incubator-uniffle.

So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want

Spark metrics system allows user to add extra metrics. You can refer to https://simhadri-g.medium.com/custom-metrics-source-in-apache-spark-ca30a3b362dd

Looks good. ping @myandpr If you have any idea, feel free to discuss

from incubator-uniffle.

So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want

Spark metrics system allows user to add extra metrics. You can refer to https://simhadri-g.medium.com/custom-metrics-source-in-apache-spark-ca30a3b362dd

Looks good. ping @myandpr If you have any idea, feel free to discuss

fine,I think I understand this feature. Let me look at the logic about spark metrics system.

from incubator-uniffle.