Paper Reading #4 : Gestalt: Integrated Support for Implementation and Analysis in Machine Learning


Reference
Authors: Kayur Patel†, Naomi Bancroft†, Steven M. Drucker, Andrew J. Ko+, James A. Landay, James Fogarty*
Affiliations: DUB group, University Of Washington, Seattle, WA and Microsoft Research*, Redmond, WA respectively.
Presentation: UIST’10, October 3–6, 2010, New York, New York, USA.

Summary
Hypothesis
The paper presents Gestalt, a development environment designed to support the process of applying machine learning. It hypothesizes Gestalt to improve user's ability to find and fix errors in machine learning code by implementing a classification pipeline,analyzing data as it moves through that pipeline, and easily transitioning between implementation and analysis.

Methods and Results
In one of such pipeline implementation, a)data is transformed into discrete examples, (b) attributes† are computed over each example, (c) a learning algorithm is used to train a model, and (d) the accuracy of that model is evaluated. Gestalt can help developers locate execution errors within specific steps which is implemented using color schemes of red, yellow, green or grey.Gestalt uses a single unified table to store data at pipelines, hence developers are freed from managing data conversion or moving data between tools. The paper further presents different tool works for machine learning and how Gestalt can fit with those platforms.
User study was done among 8 computer science graduate students on gesture recognition and sentiment analysis problems with precoded bugs to compare the performance of Gestalt to baseline.Participants unanimously preferred Gestalt and were able to find and fix more bugs using Gestalt than using the baseline. The explicit structure provided by viewing and interacting with the classification pipeline was one of the most useful components of Gestalt to most of them.Their experiment shows Gestalt significantly improves the ability of developers to find and fix bugs in machine learning systems.

Discussion
Based on the quantitative results and as well the qualitative data presented by the paper, I was satisfied with the research and implementation of Gestalt. As I am not aware of machine learning algorithms, nor have I used one till now, I am not so sure why one would want a system like this. I would say the way it finds bugs and the way people were impressed about the progress in the pipeline is what made Gestalt a better tool. I am not sure if such techniques are desired in SDK's like Eclipse and Visual Studio, but may be if available, it can be beneficial.


Comments

Post a Comment

Popular posts from this blog

Paper Reading #20: The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures

Image
Reference Authors and Affiliations: Leah Findlater,Jacob O. Wobbrock ,The Information School DUB Group University of Washington Seattle, WA 98195 USA Darren Gergle, School of Communication Northwestern University Evanston, IL 60208 USA James J. Higgins,Department of Statistics Kansas State University Manhattan, KS 66506 USA Presentation: CHI 2011, May 7–12, 2011, Vancouver, BC, Canada. Summary Hypothesis: The paper present the Aligned Rank Transform (ART) for nonparametric factorial data analysis in HCI. They propose a preprocessing step that “aligns” data before applying averaged ranks, after which point, common ANOVA procedures can be used, making the ART accessible to anyone familiar with the F-test. They hypothesize that researchers only familiar with ANOVA can use, interpret, and report results from the ART. Contents Conover and Iman’s Rank Transform (RT) uses the parametric F-test on the ranks, resulting in a nonparametric factorial procedu...
Read more

Paper Reading #29 : Usable Gestures for Blind People: Understanding Preference and Performance

Image
Reference Authors and Affiliations: Shaun K. Kane, Jacob O. Wobbrock - The Information School, DUB Group University of Washington Seattle, WA 98195 USA Richard E. Ladner - Computer Science & Engineering, DUB Group University of Washington, Seattle, WA 98195 USA Presentation :CHI 2011, May 7–12, 2011, Vancouver, BC, Canada Summary Hypothesis The paper suggests that the blind people have different gesture preferences than sighted people, including preferences for edge-based gestures and gestures that involve tapping virtual keys on a keyboard. They hypothesize differences in the speed, size, and shape of gestures performed by blind people versus those performed by sighted people. Contents Accessible touch screens still present challenges to both users and designers. Users must be able to learn new touch screen applications quickly and effectively, while designers must be able to implement accessible touch screen interaction techniques for a diverse range of devices and application...
Read more

Paper Reading #27 : Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface

Image
Reference Authors and Affiliations: Erin Treacy Solovey,Francine Lalooses,Krysta Chauncey, Douglas Weaver,Margarita Parasi,Matthias Scheutz,Robert J.K. Jacob at Tufts University, Computer Science,161 College Ave., Medford, MA Angelo Sassaroli, Sergio Fantini, at Tufts University Biomedical Engineering 4 Colby St., Medford, MA Paul Schermerhorn,Indiana University Cognitive Science Bloomington, IN Audrey Girouard,Queens University School of Computing 25 Union St., Kingston, ON K7L 3N6 Presentation: CHI 2011, May 7–12, 2011, Vancouver, BC, Canada Summary Hypothesis The paper describes two experiments leading toward a system that detects cognitive multitasking processes and uses this information as input to an adaptive interface. Authors then present a human-robot system as a proof-of-concept that uses real-time cognitive state information as input and adapts in response. They hypothesize that the prototype system serves as ...
Read more