GUIsurfer: A Reverse Engineering Framework for User Interface Software

GUIsurfer: A Reverse Engineering Framework for User Interface Software

Interface virtual com sensor 3D para interação de pessoas acamadas

A interação homem-máquina tem evoluído significativamente nos últimos anos, a ponto de permitir desenvolver soluções adequadas para apoio a pessoas que possuem um certo tipo de limitação física ou cognitiva. O desenvolvimento de técnicas naturais e intuitivas de interação, as chamadas Natural User Interface (NUI), permitem, hoje, que pessoas que estejam acamadas e/ou com incapacidade motora possam executar um conjunto de ações por intermédio de gestos, aumentando assim a sua qualidade de vida. A solução implementada neste projecto é baseada em processamento de imagem e visão por computador através do sensor 3D Kinect e consiste numa interface natural para o desenvolvimento de uma aplicação que reconheça gestos efetuados por uma mão humana. Os gestos identificados pela aplicação acionam um conjunto de ações adequados a uma pessoa acamada, como, por exemplo, acionar a emergência, ligar ou desligar a TV ou controlar a inclinação da cama. O processo de desenvolvimento deste projeto implicou várias etapas. Inicialmente houve um trabalho intenso de investigação sobre as técnicas e tecnologias consideradas importantes para a realização do trabalho - a etapa de investigação, a qual acompanhou praticamente todo o processo. A segunda etapa consistiu na configuração do sistema ao nível do hardware e do software. Após a configuração do sistema, obtiveram-se os primeiros dados do sensor 3D Kinect, os quais foram convertidos num formato mais apropriado ao seu posterior tratamento. A segmentação da mão permitiu posteriormente o reconhecimento de gestos através da técnica de matching para os seis gestos implementados. Os resultados obtidos são satisfatórios, tendo-se contabilizado cerca de 96% de resultados válidos. A área da saúde e bem-estar tem necessidade de aplicações que melhorem a qualidade de vida de pessoas acamadas, nesse sentido, o protótipo desenvolvido faz todo o sentido na sociedade actual, onde se verifica o envelhecimento da população.

A mobile systems interface protocol

Recent progresses in the software development world has assisted a change in hardware from heavy mainframes and desktop machines to unimaginable small devices leading to the prophetic "third computing paradigm", Ubiquitous Computing. Still, this novel unnoticeable devices lack in various capabilities, like computing power, storage capacity and human interface. Connectivity associated to this devices is also considered an handicap which comes generally associated expensive and limited protocols like GSM and UMTS. Considering this scenario as background, this paper presents a minimal communication protocol introducing better interfaces for limited devices. Special attention has been paid to the limitations of connectivity, storage capacity and scalability of the developed software applications. Illustrating this new protocol, a case-study is presented addressing car sensors communicating with a central

A laparoscopic surgery training interface

Laparoscopy is a surgical procedure on which operations in the abdomen are performed through small incisions using several specialized instruments. The laparoscopic surgery success greatly depends on surgeon skills and training. To achieve these technical high-standards, different apprenticeship methods have been developed, many based on in vivo training, an approach that involves high costs and complex setup procedures. This paper explores Virtual Reality (VR) simulation as an alternative for novice surgeons training. Even though several simulators are available on the market claiming successful training experiences, their use is extremely limited due to the economic costs involved. In this work, we present a low-cost laparoscopy simulator able to monitor and assist the trainee’s surgical movements. The developed prototype consists of a set of inexpensive sensors, namely an accelerometer, a gyroscope, a magnetometer and a flex sensor, attached to specific laparoscopic instruments. Our approach allows repeated assisted training of an exercise, without time constraints or additional costs, since no human artificial model is needed. A case study of our simulator applied to instrument manipulation practice (hand-eye coordination) is also presented.

GUI Inspection from Source Code Analysis

Graphical user interfaces (GUIs) are critical components of todays software. Given their increased relevance, correctness and usability of GUIs are becoming essential. This paper describes the latest results in the development of our tool to reverse engineer the GUI layer of interactive computing systems. We use static analysis techniques to generate models of the user interface behaviour from source code. Models help in graphical user interface inspection by allowing designers to concentrate on its more important aspects. One particularly type of model that the tool is able to generate is state machines. The paper shows how graph theory can be useful when applied to these models. A number of metrics and algorithms are used in the analysis of aspects of the user interface's quality. The ultimate goal of the tool is to enable analysis of interactive system through GUIs source code inspection.

GUI Behavior from Source Code Analysis

When developing interactive applications, considering the correctness of graphical user interfaces (GUIs) code is essential. GUIs are critical components of today's software, and contemporary software tools do not provide enough support for ensuring GUIs' code quality. GUIsurfer, a GUI reverse engineering tool, enables evaluation of behavioral properties of user interfaces. It performs static analysis of GUI code, generating state machines that can help in the evaluation of interactive applications. This paper describes the design, software architecture, and the use of GUIsurfer through an example. The tool is easily re-targetable, and support is available to Java/Swing, and WxHaskell. The paper sets the ground for a generalization effort to consider rich internet applications. It explores the GWT web applications' user interface programming toolkit.

A Generic Library for GUI Reasoning and Testing

Graphical user interfaces (GUIs) make software easy to use by providing the user with visual controls. Therefore, correctness of GUI's code is essential to the correct execution of the overall software. Models can help in the evaluation of interactive applications by allowing designers to concentrate on its more important aspects. This paper presents a generic model for language-independent reverse engineering of graphical user interface based applications, and we explore the integration of model-based testing techniques in our approach, thus allowing us to perform fault detection. A prototype tool has been constructed, which is already capable of deriving and testing a user interface behavioral model of applications written in Java/Swing.

Combining Formal Methods and Functional Strategies Regarding the Reverse Engineering of Interactive Applications

Abstract. Graphical user interfaces (GUIs) make software easy to use by providing the user with visual controls. Therefore, correctness of GUI’s code is essential to the correct execution of the overall software. Models can help in the evaluation of interactive applications by allowing designers to concentrate on its more important aspects. This paper describes our approach to reverse engineer an abstract model of a user interface directly from the GUI’s legacy code. We also present results from a case study. These results are encouraging and give evidence that the goal of reverse engineering user interfaces can be met with more work on this technique.

Models for the Reverse Engineering of Java/Swing Applications

Abstract. Interest in design and development of graphical user interface (GUIs) is growing in the last few years. However, correctness of GUI's code is essential to the correct execution of the overall software. Models can help in the evaluation of interactive applications by allowing designers to concentrate on its more important aspects. This paper describes our approach to reverse engineering abstract GUI models directly from the Java/Swing code.

The role of coordination analysis in software integration projects

What sort of component coordination strategies emerge in a software integration process? How can such strategies be discovered and further analysed? How close are they to the coordination component of the envisaged architectural model which was supposed to guide the integration process? This paper introduces a framework in which such questions can be discussed and illustrates its use by describing part of a real case-study. The approach is based on a methodology which enables semi-automatic discovery of coordination patterns from source code, combining generalized slicing techniques and graph manipulation

Criminal Liability of Organizations, Corporations, Legal Persons, and Similar Entities on Law of Portuguese Cybercrime: A Brief Discussion on the Issue of Crimes of “False Information,” the “Damage on Other Programs or Computer Data,” the “Computer-Software Sabotage,” the “Illegitimate Access,” the “Unlawful Interception,” and “Illegitimate Reproduction of the Protected Program”

Abstract: in Portugal, and in much of the legal systems of Europe, «legal persons» are likely to be criminally responsibilities also for cybercrimes. Like for example the following crimes: «false information»; «damage on other programs or computer data»; «computer-software sabotage»; «illegitimate access»; «unlawful interception» and «illegitimate reproduction of protected program». However, in Portugal, have many exceptions. Exceptions to the «question of criminal liability» of «legal persons». Some «legal persons» can not be blamed for cybercrime. The legislature did not leave! These «legal persons» are v.g. the following («public entities»): legal persons under public law, which include the public business entities; entities utilities, regardless of ownership; or other legal persons exercising public powers. In other words, and again as an example, a Portuguese public university or a private concessionaire of a public service in Portugal, can not commit (in Portugal) any one of cybercrime pointed. Fair? Unfair. All laws should provide that all legal persons can commit cybercrimes. PS: resumo do artigo em inglês.

SHACC: a functional prototyper for a component calculus

Over the last decade component-based software development arose as a promising paradigm to deal with the ever increasing complexity in software design, evolution and reuse. SHACC is a prototyping tool for component-based systems in which components are modelled coinductively as generalized Mealy machines. The prototype is built as a HASKELL library endowed with a graphical user interface developed in Swing

A Wireless EEG Acquisition Platform based on Embedded Systems

This paper proposes a wireless EEG acquisition platform based on Open Multimedia Architecture Platform (OMAP) embedded system. A high-impedance active dry electrode was tested for improving the scalp- electrode interface. It was used the sigma-delta ADS1298 analog-to-digital converter, and developed a “kernelspace” character driver to manage the communications between the converter unit and the OMAP’s ARM core. The acquired EEG signal data is processed by a “userspace” application, which accesses the driver’s memory, saves the data to a SD-card and transmits them through a wireless TCP/IP-socket to a PC. The electrodes were tested through the alpha wave replacement phenomenon. The experimental results presented the expected alpha rhythm (8-13 Hz) reactiveness to the eyes opening task. The driver spends about 725 μs to acquire and store the data samples. The application takes about 244 μs to get the data from the driver and 1.4 ms to save it in the SD-card. A WiFi throughput of 12.8Mbps was measured which results in a transmission time of 5 ms for 512 kb of data. The embedded system consumes about 200 mAh when wireless off and 400 mAh when it is on. The system exhibits a reliable performance to record EEG signals and transmit them wirelessly. Besides the microcontroller-based architectures, the proposed platform demonstrates that powerful ARM processors running embedded operating systems can be programmed with real-time constrains at the kernel level in order to control hardware, while maintaining their parallel processing abilities in high level software applications.

Cognitive intervention protocol for age-related memory impairments

With the number of elderly people increasing tremendously worldwide, comes the need for effective methods to maintain or improve older adults' cognitive performance. Using continuous neurofeedback, through the use of EEG techniques, people can learn how to train and alter their brain electrical activity. A software platform that puts together the proposed rehabilitation methodology has been developed: a digital game protocol that supports neurofeedback training of alpha and theta rhythms, by reading the EEG activity and presenting it back to the subject, interleaved with neurocognitive tasks such as n-Back and Corsi Block-Tapping. This tool will be used as a potential rehabilitative platform for age-related memory impairments.

An approach for graphical user interface bad smells detection

In the context of an e ort to develop methodologies to support the evaluation of interactive system, this paper investigates an approach to detect graphical user interface bad smells. Our approach consists in detecting user interface bad smells through model-based reverse engineering from source code. Models are used to de ne which widgets are present in the interface, when can particular graphical user interface (GUI) events occur, under which conditions, which system actions are executed, and which GUI state is generated next.