nspyrison / cheem_paper Goto Github PK

(a) The writing of the paper is informal, lacks clarity and depth in many parts that are critical to understanding the method proposed, and requires significant improvement. The method of radial tour, which (apparently?) seems to be the most important contribution of the paper, is practically not explained at all. Up to Section 4, when the description of the interactive visualization tool starts, all we get for the explanation of the radial tour is one sentence, which is drastically not enough. This is observable throughout the text as a whole, and it is very hard to point exactly where and when it happens. In general, the paper's writing is informal, lacking the depth necessary to make it reproducible and auditable. Without clear, detailed explanations of the methods used, it becomes hard to accept it in a deeply technical journal such as JMLR.

• Di will clean up the language, and add explanation of the radial tour and the importance.

I offer some more detailed notes below, but they are not comprehensive.

• Abstract: While it is not wrong, not all XAI techniques use local explanations; some definitely do. The wording here could be a bit more accurate.

• check wording

• The introduction seems to end a bit abruptly. Maybe a couple of sentences about the results and outcomes of the research?

• Add segue

• Page 3: "Consider a highly nonlinear model." I feel like this (and the reasoning that follows) is a bit of an informal statement. Although intuitively I agree with the authors, is there any objective (or at least semi-formal) way to explain this problem? What is highly non-linear (vs. just linear)? Why, exactly, is it that it is so hard to interpret nonlinear models? That would improve the argumentation here quite a bit.

• Clarify

• "The attribution of feature importance depends on the sequence of the included features." This sentence feels out of place.

• Maybe remove or move

• "modified breakdown plots" -- Since this is taking an important place in the paper, with a figure attached to it, I think it needs to be better explained. What are exactly modified breakdown plots?

• Add explanation

• "The magnitude of the contributions depends on the sequence in which they appear." -- This is unclear. The figure has no label for the horizontal axes, which makes it very hard to interpret. And, in case, the sequence alters the magnitude of the contributions, then it should be explained how the sequence is determined. Right now, this explanation does not help to make the visualization or the method presented in Figure 1 easier to understand.

• Add clarification and modify figure

• "The attribution space corresponds to the local explanations for each instance; feature importance in the
  vicinity of the instance." -- I think this needs to be better explained. The concept of the attribution space remains obscure to me. It seems to be a very interesting concept and very important to the paper.

• Add explanation

• "The projection attribution of the primary instance (PI) is examined and typically viewed with an optional comparison instance (CI)." -- The concept of PI and CI remains confusing to me. Needs a clearer explanation.

• Clarify

• Section 4.2 as a whole -- This explanation is overly confusing and informal/subjective. It needs to be significantly revised to become more fluid and clear.

• Revise

• I don't believe that sections 4.7 and 4.8 are needed.

• Consider removing

• Should we use "Local variable attributions" instead of "local explanations" throughout?
• What do you mean "segments" in "whether outlier are present, whether segments can be identified."

I don't think you need the pages workflow to be active on this repo