crossmodal error correction of continuous handwriting recognition by speech

In recognition-based user interface, users’ satisfaction is determined not only by recognition accuracy but also by effort to correct recognition errors. In this paper, we introduce a crossmodal error correction technique, which allows users to correct errors of Chinese handwriting recognition by speech. The focus of the paper is a multimodal fusion algorithm supporting the crossmodal error correction. By fusing handwriting and speech recognition, the algorithm can correct errors in both character extraction and recognition of handwriting. The experimental result indicates that the algorithm is effective and efficient. Moreover, the evaluation also shows the correction technique can help users to correct errors in handwriting recognition more efficiently than the other two error correction techniques.

Theory of Handwriting

Handwriting production is viewed as a constrained modulation of an underlying oscillatory process. Coupled oscillations in horizontal and vertical directions produce letter forms, and when superimposed on a rightward constant velocity horizontal sweep result in spatially separated letters. Modulation of the vertical oscillation is responsible for control of letter height, either through altering the frequency or altering the acceleration amplitude. Modulation of the horizontal oscillation is responsible for control of corner shape through altering phase or amplitude. The vertical velocity zero crossing in the velocity space diagram is important from the standpoint of control. Changing the horizontal velocity value at this zero crossing controls corner shape, and such changes can be effected through modifying the horizontal oscillation amplitude and phase. Changing the slope at this zero crossing controls writing slant; this slope depends on the horizontal and vertical velocity zero amplitudes and on the relative phase difference. Letter height modulation is also best applied at the vertical velocity zero crossing to preserve an even baseline. The corner shape and slant constraints completely determine the amplitude and phase relations between the two oscillations. Under these constraints interletter separation is not an independent parameter. This theory applies generally to a number of acceleration oscillation patterns such as sinusoidal, rectangular and trapezoidal oscillations. The oscillation theory also provides an explanation for how handwriting might degenerate with speed. An implementation of the theory in the context of the spring muscle model is developed. Here sinusoidal oscillations arise from a purely mechanical sources; orthogonal antagonistic spring pairs generate particular cycloids depending on the initial conditions. Modulating between cycloids can be achieved by changing the spring zero settings at the appropriate times. Frequency can be modulated either by shifting between coactivation and alternating activation of the antagonistic springs or by presuming variable spring constant springs. An acceleration and position measuring apparatus was developed for measurements of human handwriting. Measurements of human writing are consistent with the oscillation theory. It is shown that the minimum energy movement for the spring muscle is bang-coast-bang. For certain parameter values a singular arc solution can be shown to be minimizing. Experimental measurements however indicate that handwriting is not a minimum energy movement.

Identifying Novice Difficulties in Object Orientated Design

Thomasson, B. Ratcliffe, M. Thomas, L. Identifying Novice Difficulties in Object Orientated Design. ACM SIGCSE Bulletin Volume 38 , Issue 3 (September 2006)

A Quantitative Evaluation of the AVITEWRITE Model of Handwriting Learning

Much sensory-motor behavior develops through imitation, as during the learning of handwriting by children. Such complex sequential acts are broken down into distinct motor control synergies, or muscle groups, whose activities overlap in time to generate continuous, curved movements that obey an intense relation between curvature and speed. The Adaptive Vector Integration to Endpoint (AVITEWRITE) model of Grossberg and Paine (2000) proposed how such complex movements may be learned through attentive imitation. The model suggest how frontal, parietal, and motor cortical mechanisms, such as difference vector encoding, under volitional control from the basal ganglia, interact with adaptively-timed, predictive cerebellar learning during movement imitation and predictive performance. Key psycophysical and neural data about learning to make curved movements were simulated, including a decrease in writing time as learning progresses; generation of unimodal, bell-shaped velocity profiles for each movement synergy; size scaling with isochrony, and speed scaling with preservation of the letter shape and the shapes of the velocity profiles; an inverse relation between curvature and tangential velocity; and a Two-Thirds Power Law relation between angular velocity and curvature. However, the model learned from letter trajectories of only one subject, and only qualitative kinematic comparisons were made with previously published human data. The present work describes a quantitative test of AVITEWRITE through direct comparison of a corpus of human handwriting data with the model's performance when it learns by tracing human trajectories. The results show that model performance was variable across subjects, with an average correlation between the model and human data of 89+/-10%. The present data from simulations using the AVITEWRITE model highlight some of its strengths while focusing attention on areas, such as novel shape learning in children, where all models of handwriting and learning of other complex sensory-motor skills would benefit from further research.

The VITEWRITE Model of Handwriting Production

This article describes the VITEWRITE model for generating handwriting movements. The model consists of a sequential controller, or motor program, that interacts with a trajectory generator to move a hand with redundant degrees of freedom. The neural trajectory generator is the Vector Integration to Endpoint (VITE) model for synchronous variable-speed control of multijoint movements. VITE properties enable a simple control strategy to generate complex handwritten script if the hand model contains redundant degrees of freedom. The controller launches transient directional commands to independent hand synergies at times when the hand begins to move, or when a velocity peak in the outflow command to a given synergy occurs. The VITE model translates these temporally disjoint synergy commands into smooth curvilinear trajectories among temporally overlapping synergetic movements. Each synergy exhibits a unimodal velocity profile during any stroke, generates letters that are invariant under speed and size rescaling, and enables effortless connection of letter shapes into words. Speed and size rescaling are achieved by scalar GO and GRO signals that express computationally simple volitional commands. Psychophysical data such as the isochrony principle, asymmetric velocity profiles, and the two-thirds power law relating movement curvature and velocity arise as emergent properties of model interactions.

Neural Models of Temporally Organized Behaviors: Handwriting Production and Working Memory

Advanced Research Projects Agency (ONR N00014-92-J-4015); Office of Naval Research (N00014-91-J-4100, N00014-92-J-1309)

Difficulties and errors in number reasoning development

The detailed study of difficulties and errors in young learner comprehension is a relevant and productive research field in Mathematics Education. Studies in the field are numerous although somewhat too varied. The present paper is suggesting methodological perspectives and principles applying to the field of research; we also show an example with school work. The use of figurate numbers as a representation system gives richer conceptual values, boosts visual reasoning and facilitates learner understanding.

Challenges and difficulties in assessing the Environmental Status under the requirements of the Ecosystem Approach in North-African countries, illustrated by eutrophication assessment

Marine ecosystems provide many ecosystem goods and services. However, these ecosystems and the benefits they create for humans are subject to competing uses and increasing pressures. As a consequence of the increasing threats to the marine environment, several regulations require applying an ecosystem-based approach for managing the marine environment. Within the Mediterranean Sea, in 2008, the Contracting Parties of the Mediterranean Action Plan decided to progressively apply the Ecosystem Approach (EcAp) with the objective of achieving Good Environmental Status (GES) for 2018. To assess the Environmental Status, the EcAp proposes 11 Ecological Objectives, each of which requires a set of relevant indicators to be integrated. Progress towards the EcAp entails a gradual and important challenge for North-African countries, and efforts have to be initiated to propose and discuss methods. Accordingly, to enhance the capacity of North-African countries to implement EcAp and particularly to propose and discuss indicators and methods to assess GES, the aim of this manuscript is to identify the practical problems and gaps found at each stage of the Environmental Status assessment process. For this purpose, a stepwise method has been proposed to assess the Environmental Status using Ecologic Objective 5-Eutrophication as example.

Challenges and difficulties in assessing the Environmental Status under the requirements of the Ecosystem Approach in North-African countries, illustrated by eutrophication assessment

Marine ecosystems provide many ecosystem goods and services. However, these ecosystems and the benefits they create for humans are subject to competing uses and increasing pressures. As a consequence of the increasing threats to the marine environment, several regulations require applying an ecosystem-based approach for managing the marine environment. Within the Mediterranean Sea, in 2008, the Contracting Parties of the Mediterranean Action Plan decided to progressively apply the Ecosystem Approach (EcAp) with the objective of achieving Good Environmental Status (GES) for 2018. To assess the Environmental Status, the EcAp proposes 11 Ecological Objectives, each of which requires a set of relevant indicators to be integrated. Progress towards the EcAp entails a gradual and important challenge for North-African countries, and efforts have to be initiated to propose and discuss methods. Accordingly, to enhance the capacity of North-African countries to implement EcAp and particularly to propose and discuss indicators and methods to assess GES, the aim of this manuscript is to identify the practical problems and gaps found at each stage of the Environmental Status assessment process. For this purpose, a stepwise method has been proposed to assess the Environmental Status using Ecologic Objective 5-Eutrophication as example.