[dcchairs2016] UbiComp/ISWC 2016 Doctoral School notification - #105

[dcchairs2016 at ubicomp.org]

Dear Pedro Garcia Garcia,

Please find enclosed the reviews for your submission for the Ubicomp/ISWC 2016 Doctoral School.

105: "Visual cue that change how people perceive and interact with autonomous systems"

Despite not being able to accept your submission at this year's Ubicomp/ISWC Doctoral School, committee members provided guidance and feedback on your submitted paper. We highly encourage you to follow the valuable advices that the committee member entered in their reviews towards improving on your doctoral work.

Thank you for submitting to the UbiComp 2014 Doctoral School.


Max Mühlhäuser
Nadir Weibel
Rene Mayrhofer

UbiComp 2016 Doctoral School Chairs


------------------------ Submission 105, Review 1 ------------------------

Title: Visual cue that change how people perceive and interact with autonomous systems


Confidence

   3  (Very confident - I am knowledgeable in the area)

Contribution to UbiComp

   The paper focusses on a CogSci/HCI related issue and has no existing or
   planned core Ubicomp contribution on its own.

Overall Rating

   1  (Definite reject: I would argue strongly for rejecting this paper.)

The Review

   "Management summary": (1) The status of the paper (in terms of quality of
   writing) is rather premature; (2) Both the past research and the planned
   research do not appear to be convincing in the sense of promising
   interesting scientific results or insights; (3) the relationship to
   UbiComp is extremely shallow.
   More details:
   The author (btw.: Why THREE blinded authors for ONE PhD?) seems to
   combine three incompatible things:
   1.	Quite far-fetching goals (“intelligibility of autonomous systems
   (AS)”, changing the way people perceive *and interact with* AS, etc.)
   2.	Extremely low-hanging-fruits as study setups/goals: visualization of
   the motion of (mobile) robots as a means for conveying their activities
   (ha! compare this to intentions/rationale etc. that are subject of
   intelligibility research! Remember the goal to “interact with” ASs in
   item 1)
   3.	Related work that is not properly related to the goal at hand –
   well, the related work is so different from the goal at hand that the
   relationship is very simple, repeatedly stated in the paper as “they do
   other things” (which is of course formulated more nicely)
   The bad writing quality makes it very difficult to imagine what
   “visualizing the motion of robots” actually means! Does the author
   compare a user’s understanding of the mobile-robot operation when she
   watches the robot move versus when she doesn’t (quite trivial!)? If
   not: where and how is the motion conveyed to the user? Similarly, the
   future work section – which is *very brief* talks about “different
   implementations of visual cues” – and remains totally unclear about
   what that means (does it rather mean “different visual cues”? If not,
   what is the role of IMPLEMENTATIONS in all this?). Since the paper fails
   to clearly articulate even such fundamental issues, there is no
   alternative to rejecting it.


------------------------ Submission 105, Review 2 ------------------------

Title: Visual cue that change how people perceive and interact with autonomous systems


Confidence

   2  (Somewhat confident - I have passing knowledge)

Contribution to UbiComp

   This draft suggests to study way to visualize the internal behavior of
   autonomous systems (AS) for better understanding by / interaction with
   human users. It proposes that visual cues may be an important aspect even
   when the activity performed by the AS is not primarily visual / motion.
   In principle, it is relevant to Ubicomp, but please see below for issues.

Overall Rating

   2  (Probably reject: I would argue for rejecting this paper.)

The Review

   As mentioned above, the topic itself is certainly interesting to Ubicomp,
   as interaction with AS is an important aspect of many of the core use
   cases. Unfortunately, it does not become clear how this proposed PhD will
   directly contribute to Ubicomp as a field. The comparison of visual cues
   with those given by musicians is both interesting and unhelpful, as the
   parallels between people playing instruments and robots performing simple
   tasks seem to end rather quickly. For musicians, I suggest that the
   visual cues are rather subtly interwoven with the main sense of sound,
   while a robot vacuum cleaner has exactly one simple task to perform, and
   the quality of its results is more easily measured. 

   This disconnect in the presented literature survey already points at the
   main issue in the current draft: that it is very vague and non-specific,
   and there is no clear plan on how to create a PhD thesis from these
   general research questions. What I miss most are specific ideas and
   concepts on how to move forward: a) which additional visual cues could
   aid which specific AS interactions/introspection (because I strongly
   suspect that they will not be easily generalizable across different use
   cases, much in the same way that a vacuum cleaner moving like a musician
   does not seem helpful); and b) how to evaluate their effectiveness (i.e.
   which metric will be used to quantify "performance")?

   I strongly suggest to formulate more specific hypothesis before external
   input from a DC panel will be helpful to the author.