Audio-Visual Information Processing across different age groups

People possess different sensory modalities to detect, interpret, and efficiently act upon various events in a complex and dynamic environment (Fetsch, DeAngelis, & Angelaki, 2013). Much empirical work has been done to understand the interplay of modalities (e.g. audio-visual interactions, see Calvert, Spence, & Stein, 2004). On the one hand, integration of multimodal input as a functional principle of the brain enables the versatile and coherent perception of the environment (Lewkowicz & Ghazanfar, 2009). On the other hand, sensory integration does not necessarily mean that input from modalities is always weighted equally (Ernst, 2008). Rather, when two or more modalities are stimulated concurrently, one often finds one modality dominating over another. Study 1 and 2 of the dissertation addressed the developmental trajectory of sensory dominance. In both studies, 6-year-olds, 9-year-olds, and adults were tested in order to examine sensory (audio-visual) dominance across different age groups. In Study 3, sensory dominance was put into an applied context by examining verbal and visual overshadowing effects among 4- to 6-year olds performing a face recognition task. The results of Study 1 and Study 2 support default auditory dominance in young children as proposed by Napolitano and Sloutsky (2004) that persists up to 6 years of age. For 9-year-olds, results on privileged modality processing were inconsistent. Whereas visual dominance was revealed in Study 1, privileged auditory processing was revealed in Study 2. Among adults, a visual dominance was observed in Study 1, which has also been demonstrated in preceding studies (see Spence, Parise, & Chen, 2012). No sensory dominance was revealed in Study 2 for adults. Potential explanations are discussed. Study 3 referred to verbal and visual overshadowing effects in 4- to 6-year-olds. The aim was to examine whether verbalization (i.e., verbally describing a previously seen face), or visualization (i.e., drawing the seen face) might affect later face recognition. No effect of visualization on recognition accuracy was revealed. As opposed to a verbal overshadowing effect, a verbal facilitation effect occurred. Moreover, verbal intelligence was a significant predictor for recognition accuracy in the verbalization group but not in the control group. This suggests that strengthening verbal intelligence in children can pay off in non-verbal domains as well, which might have educational implications.

Texture for Script identification

The problem of determining the script and language of a document image has a number of important applications in the field of document analysis, such as indexing and sorting of large collections of such images, or as a precursor to optical character recognition (OCR). In this paper, we investigate the use of texture as a tool for determining the script of a document image, based on the observation that text has a distinct visual texture. An experimental evaluation of a number of commonly used texture features is conducted on a newly created script database, providing a qualitative measure of which features are most appropriate for this task. Strategies for improving classification results in situations with limited training data and multiple font types are also proposed.

Making sense of an everyday science text : linguistic, visual and spatial design

In this article, we take a close look at the literacy demands of one task from the ‘Marvellous Micro-organisms Stage 3 Life and Living’ Primary Connections unit (Australian Academy of Science, 2005). One lesson from the unit, ‘Exploring Bread’, (pp 4-8) asks students to ‘use bread labels to locate ingredient information and synthesise understanding of bread ingredients’. We draw upon a framework offered by the New London Group (2000), that of linguistic, visual and spatial design, to consider in more detail three bread wrappers and from there the complex literacies that students need to interrelate to undertake the required task. Our findings are that although bread wrappers are an example of an everyday science text, their linguistic, visual and spatial designs and their interrelationship are not trivial. We conclude by reinforcing the need for teachers of science to also consider how the complex design elements of everyday science texts and their interrelated literacies are made visible through instructional practice.

Teacher-child relationship quality for young children with parent reported language concerns

Previous research has demonstrated the importance of the qualities of the teacher-child relationship on children’s development. Close teacher-child relationships are especially important for children at risk. Positive relationships have been shown to have beneficial effects on children’s social and academic development (Birch & Ladd, 1997; Pianta & Stuhlman, 2004). Children with language difficulties are likely to face increased risks with regard to long term social and academic outcomes. The purpose of the current research was to gain greater understanding of the qualities of teacher-child relationships for young children with parent reported language concerns. The research analyses completed for this thesis involved the use of data from the public-access database of Growing Up in Australia: The Longitudinal Study of Australian Children (LSAC). LSAC is a longitudinal study involving a nationally representative sample of 10,000 Australian children. Data are being collected biennially from 2004 (Wave 1 data collection) until 2010 (Wave 4 data collection). LSAC has a cross-sequential research design involving two cohorts, an infant cohort (0-1 year at age of recruitment) and a kindergarten cohort (4-5 years at age of recruitment). Two studies are reported in this thesis using data for the LSAC Kindergarten Cohort which had 4983 child participants at recruitment. Study 1 used Wave 1 data to identify the differences between teacher-child relationship qualities for children with parent reported language concerns and their peers. Children identified by parents for whom concerns were held about their receptive and expressive language, as measured by items from the Parents’ Evaluation of Developmental Status (PEDS) (Glascoe, 2000) were the target (at risk) group in the study (n = 210). A matched case control group of peers (n = 210), matched on the child characteristics of sex, age, cultural and linguistic differences (CALD), and socio-economic positioning (SEP), were the comparison group for this analysis. Teacher-child relationship quality was measured by teacher reports on the Closeness and Conflict scales from the short version of the Student-Teacher Relationship Scale (STRS) (Pianta, 2001). There were statistically significant differences in the levels of closeness and conflict between the two groups. The target group had relationships with their teachers that had lower levels of closeness and higher levels of conflict than the control group. Study 2 reports analyses that examined the stability of the qualities of the teacher-child relationships at Wave 1 (4-5 years) and the qualities of the teacher-child relationships at Wave 2 (6-7 years). This time frame crosses the period of the children’s transition to school. The study examined whether early patterns in the qualities of the teacher-child relationship for children with parent reported language concerns at Wave 1 predicted the qualities of the teacher-child relationship outcomes in the early years of formal school. The sample for this study consisted of the group of children identified with PEDS language concerns at Wave 1 who also had teacher report data at Wave 2 (n = 145). Teacher-child relationship quality at Wave 1 and Wave 2 was again measured by the STRS scales of Closeness and Conflict. Results from multiple regression models indicated that teacher-child relationship quality at Wave 1 significantly contributed to the prediction of the quality of the teacher-child relationship at Wave 2, beyond other predictor variables included in the regression models. Specifically, Wave 1 STRS Closeness scores were the most significant predictor for STRS Closeness scores at Wave 2, while Wave 1 STRS Conflict scores were the only significant predictor for Wave 2 STRS Conflict outcomes. These results indicate that the qualities of the teacher-child relationship experienced prior to school by children with parent reported language concerns remained stable across transitions into formal schooling at which time the child had a different teacher. The results of these studies provide valuable insight into the nature of teacher-child relationship quality for young children with parent reported language concerns. These children experienced teacher-child relationships of a lower quality when compared with peers and, additionally, the qualities of these relationships prior to formal schooling were predictive of the qualities of the relationships in the early years of formal schooling. This raises concerns, given the increased risks of poorer social and academic outcomes already faced by children with language difficulties, that these early teacher-child relationships have an impact on future teacher-child relationships. Results of these studies are discussed with these considerations in mind and also discussed in terms of the implications for educational theory, policy and practice.

Mid-level concept learning with visual contextual ontologies and probabilistic inference for image annotation

To date, automatic recognition of semantic information such as salient objects and mid-level concepts from images is a challenging task. Since real-world objects tend to exist in a context within their environment, the computer vision researchers have increasingly incorporated contextual information for improving object recognition. In this paper, we present a method to build a visual contextual ontology from salient objects descriptions for image annotation. The ontologies include not only partOf/kindOf relations, but also spatial and co-occurrence relations. A two-step image annotation algorithm is also proposed based on ontology relations and probabilistic inference. Different from most of the existing work, we specially exploit how to combine representation of ontology, contextual knowledge and probabilistic inference. The experiments show that image annotation results are improved in the LabelMe dataset.

Justifications in young children’s language in an early childhood classroom

This paper examines the interactional phenomenon of justification as it is produced in young children’s language. A justification provides a reason for one’s position and can be produced in children’s language at an early age. There are various pragmatic reasons for justifications. For example, justifications may be drawn upon by members to compensate for the disruption of the existing social order or to explain something that is possibly questionable. Justifications are also drawn upon to extend or close disputes. This study uses the analytical techniques of conversation analysis and membership categorisation to analyse video-recorded and transcribed interactions of young children (aged 4-6 years) in a preparatory classroom in a primary school in Australia. The focus is an episode that occurred within the block play area of the classroom that involved a dispute of ownership relating to a small, wooden plank. In analysing this dispute, justifications were frequent occurrences and the young participants drew upon justificatory devices in their everyday arguments. As the turns surrounding the justificatory language were examined, a pattern emerged: in each excerpt observed, a justification arose in response to a challenge. This pattern provided the basis for developing a model that helped to discern where, why and what type of justifications occurred in the interaction. To depict this interactional phenomenon, the model of ‘if x, then y’ was used, ‘x’ referring to the challenge or prompt, and ‘y’ referring to the justificatory response. Justifications related to the concepts of ownership and were used as devices by those engaged in disputes to support their positions and provide reasons for their actions. The children drew upon these child-constructed rules as resources to use in disputes with their peers, in order to construct and maintain the social order of the block area in the classroom.

Seeing is understanding : the effect of visualisation in understanding programming concepts

How and why visualisations support learning was the subject of this qualitative instrumental collective case study. Five computer programming languages (PHP, Visual Basic, Alice, GameMaker, and RoboLab) supporting differing degrees of visualisation were used as cases to explore the effectiveness of software visualisation to develop fundamental computer programming concepts (sequence, iteration, selection, and modularity). Cognitive theories of visual and auditory processing, cognitive load, and mental models provided a framework in which student cognitive development was tracked and measured by thirty-one 15-17 year old students drawn from a Queensland metropolitan secondary private girls’ school, as active participants in the research. Seventeen findings in three sections increase our understanding of the effects of visualisation on the learning process. The study extended the use of mental model theory to track the learning process, and demonstrated application of student research based metacognitive analysis on individual and peer cognitive development as a means to support research and as an approach to teaching. The findings also forward an explanation for failures in previous software visualisation studies, in particular the study has demonstrated that for the cases examined, where complex concepts are being developed, the mixing of auditory (or text) and visual elements can result in excessive cognitive load and impede learning. This finding provides a framework for selecting the most appropriate instructional programming language based on the cognitive complexity of the concepts under study.

The effect of language models on phonetic decoding for spoken term detection

Spoken term detection (STD) popularly involves performing word or sub-word level speech recognition and indexing the result. This work challenges the assumption that improved speech recognition accuracy implies better indexing for STD. Using an index derived from phone lattices, this paper examines the effect of language model selection on the relationship between phone recognition accuracy and STD accuracy. Results suggest that language models usually improve phone recognition accuracy but their inclusion does not always translate to improved STD accuracy. The findings suggest that using phone recognition accuracy to measure the quality of an STD index can be problematic, and highlight the need for an alternative that is more closely aligned with the goals of the specific detection task.

The language of neofeudal corporatism and the war on Iraq

In this paper we argue that the term “capitalism” is no longer useful for understanding the current system of political economic relations in which we live. Rather, we argue that the system can be more usefully characterised as neofeudal corporatism. Using examples drawn from a 300,000 word corpus of public utterances by three political leaders from the “coalition of the willing”— George W. Bush, Tony Blair, and John Howard—we show some defining characteristics of this relatively new system and how they are manifest in political language about the invasion of Iraq.

An old language for a new landscape

Because aesthetics can have a profound effect upon the human relationship to the non-human environment the importance of aesthetics to ecologically sustainable designed landscapes has been acknowledged. However, in recognition that the physical forms of designed landscapes are an expression of the social values of the time, some design professionals have called for a new aesthetic ― one that reflects these current ecological concerns. To address this, some authors have suggested various theoretical design frameworks upon which such an aesthetic could be based. Within these frameworks there is an underlying theme that the patterns and processes of natural systems have the potential to form a new aesthetic for landscape design —an aesthetic based on fractal rather than Euclidean geometry. Perry, Reeves and Sim (2008) have shown that it is possible to differentiate between different landscape forms by fractal analysis. However, this research also shows that individual scenes from within very different landscape forms can possess the same fractal properties. Early data, revealed by transforming landscape images from the spatial to the frequency domain, using the fast Fourier transform, suggest that fractal patterning can have a significant effect within the landscape. In fact, it may be argued that any landscape design that includes living processes will include some design element whose ultimate form can only be expressed through the mathematics of fractal geometry. This paper will present ongoing research into the potential role of fractal geometry as a basis for a new form language – a language that may articulate an aesthetic for landscape design that echoes our ecological awakening.

Automatic tracking, super-resolution and recognition of human faces from surveillance video

Identifying an individual from surveillance video is a difficult, time consuming and labour intensive process. The proposed system aims to streamline this process by filtering out unwanted scenes and enhancing an individual's face through super-resolution. An automatic face recognition system is then used to identify the subject or present the human operator with likely matches from a database. A person tracker is used to speed up the subject detection and super-resolution process by tracking moving subjects and cropping a region of interest around the subject's face to reduce the number and size of the image frames to be super-resolved respectively. In this paper, experiments have been conducted to demonstrate how the optical flow super-resolution method used improves surveillance imagery for visual inspection as well as automatic face recognition on an Eigenface and Elastic Bunch Graph Matching system. The optical flow based method has also been benchmarked against the ``hallucination'' algorithm, interpolation methods and the original low-resolution images. Results show that both super-resolution algorithms improved recognition rates significantly. Although the hallucination method resulted in slightly higher recognition rates, the optical flow method produced less artifacts and more visually correct images suitable for human consumption.

Contributions of vision-proprioception interactions to the estimation of time-varying hand and target locations

We investigated the relative importance of vision and proprioception in estimating target and hand locations in a dynamic environment. Subjects performed a position estimation task in which a target moved horizontally on a screen at a constant velocity and then disappeared. They were asked to estimate the position of the invisible target under two conditions: passively observing and manually tracking. The tracking trials included three visual conditions with a cursor representing the hand position: always visible, disappearing simultaneously with target disappearance, and always invisible. The target’s invisible displacement was systematically underestimated during passive observation. In active conditions, tracking with the visible cursor significantly decreased the extent of underestimation. Tracking of the invisible target became much more accurate under this condition and was not affected by cursor disappearance. In a second experiment, subjects were asked to judge the position of their unseen hand instead of the target during tracking movements. Invisible hand displacements were also underestimated when compared with the actual displacement. Continuous or brief presentation of the cursor reduced the extent of underestimation. These results suggest that vision–proprioception interactions are critical for representing exact target–hand spatial relationships, and that such sensorimotor representation of hand kinematics serves a cognitive function in predicting target position. We propose a hypothesis that the central nervous system can utilize information derived from proprioception and/or efference copy for sensorimotor prediction of dynamic target and hand positions, but that effective use of this information for conscious estimation requires that it be presented in a form that corresponds to that used for the estimations.