[dcchairs2016] UbiComp/ISWC 2016 Doctoral School notification - #103

[dcchairs2016 at ubicomp.org]

Dear Jhim Kiel Verame,

Please find enclosed the reviews for your submission for the Ubicomp/ISWC 2016 Doctoral School.

103: "Investigating User Interaction with Autonomous Systems for Non-specialist Applications"

Despite not being able to accept your submission at this year's Ubicomp/ISWC Doctoral School, committee members provided guidance and feedback on your submitted paper. We highly encourage you to follow the valuable advices that the committee member entered in their reviews towards improving on your doctoral work.

Thank you for submitting to the UbiComp 2014 Doctoral School.


Max Mühlhäuser
Nadir Weibel
Rene Mayrhofer

UbiComp 2016 Doctoral School Chairs


------------------------ Submission 103, Review 1 ------------------------

Title: Investigating User Interaction with Autonomous Systems for Non-specialist Applications


Confidence

   3  (Very confident - I am knowledgeable in the area)

Contribution to UbiComp

   The paper looks at HCI (user interaction) for autonomous systems (AS),
   mentioning UbiComp related ASs (smart thermostat: autonomous temp
   control, smart vacuum cleaner etc.) but not supporting any reason why and
   how the PhD would be specific for Ubicomp. Up to now (1 year of PhD),
   there was no real Ubicomp system involved, but the author claims to take
   one into consideration in the future: autonomous food purchase. 

Overall Rating

   2  (Probably reject: I would argue for rejecting this paper.)

The Review

   In its outset, the paper provides a convincing rationale for focused
   research on UI (user interaction) for AS (autonomous systems): users need
   more trust in AS (trust can be greatly augmented if users understand why
   the system is doing what when, which in turn requires a pertinent UI) and
   due to the imperfection of AS, it can be very beneficial if users can
   influence the actions or get temporary control over them (which again
   requires a pertinent UI). 
   Apart from the nice general outset, the proposal appears to have very
   serious issues: (1) Year one of the PhD has passed and there is obviously
   just a literature review and a first user study (not based on a real
   system) – this seems “OKish but not overwhelming”; (2) The plans
   for the rest of the PhD are extremely shallow und blurry: (2a) it is
   claimed that a Ubicomp system shall be implemented that controls the food
   stock at home and autonomously orders replenishment: without providing
   more (and much more concrete) details, the author fails to convince the
   reader that he will be able to achieve this “heroic task” and build
   something that is even close to useful, but a useful system would be
   necessary for the proposed further user study to be of any meaning; (2b)
   based on this planned system, the author plans to “conduct a second
   user study” – it must be doubted that any university will bestow a
   highly valued PhD in an HCI field just for the outcome of two user
   studies; this being said, the proposal could be rescued if the author
   planned to draw genuine scientific results from the UI and from the stock
   replenishment AS; as of today, the planned system shall serve just as
   apparatus for HCI related hypothesis testing (given the great challenge
   that a decent food replenishment AS would represent, it would be a waste
   of energy to restrict its scientific value to this); (2c) Generally
   speaking, there is no evidence that the PhD will result in concrete
   interaction concepts for UIs-for-ASs: the paper just speaks of
   guidelines. 
   Summarizing, it is strongly recommended to follow one of the following
   paths:
   -	Either build the food replenishment system such that both the system
   and its interaction concepts lead to original scientific contributions or
   -	Forget about building such a challenging system, plan a broader variety
   of studies and plan early on which concrete UI-for-AS related questions
   each one should address
   Most disturbingly, the whole research seems to take UbiComp related ASs
   as the ASs-of-choice just by chance: no specific challenges of
   UbiComp-ASs are taken into account or even mentioned.


------------------------ Submission 103, Review 2 ------------------------

Title: Investigating User Interaction with Autonomous Systems for Non-specialist Applications


Confidence

   2  (Somewhat confident - I have passing knowledge)

Contribution to UbiComp

   This line of work addresses the issue of interaction with and trust in
   autonomous systems. It is highly relevant to Ubicomp, considering
   automatically triggered actions in many of the intended use cases.

Overall Rating

   2  (Probably reject: I would argue for rejecting this paper.)

The Review

   Although the topic is certainly very relevant to Ubicomp, the current
   draft does not make it sufficiently clear what the research issues
   tackled in the PhD thesis will be. It is clear that interaction with
   autonomous systems is an important area and that it remains unsolved to a
   certain extent, not only within Ubicomp core scenarios but also within
   other areas such as (semi-) autonomous vehicle control To better put this
   work into context, I suggest an even wider literature review on
   interaction with (semi-) autonomous / assistance / control systems.

   That said, I do not believe that the author would benefit significantly
   from attending the Ubicomp DC at this time, as the research questions are
   still very general and vague, and no clear idea/approach/hypothesis on
   _how_ to actually improve interaction with such systems is apparent in
   the submitted draft. That is, the author is probably a bit too early in
   their PhD process for targeted, external input. The suggested methodology
   is still on a meta level (asking questions if a system could be built,
   and not how it would look like), and I suggest to form more specific
   hypothesis before looking for external supervisor feedback.