Semi-automatic knowledge extraction, representation and context-sensitive intelligent retrieval of video content using collateral context modelling with scalable ontological networks

Automatic indexing and retrieval of digital data poses major challenges. The main problem arises from the ever increasing mass of digital media and the lack of efficient methods for indexing and retrieval of such data based on the semantic content rather than keywords. To enable intelligent web interactions, or even web filtering, we need to be capable of interpreting the information base in an intelligent manner. For a number of years research has been ongoing in the field of ontological engineering with the aim of using ontologies to add such (meta) knowledge to information. In this paper, we describe the architecture of a system (Dynamic REtrieval Analysis and semantic metadata Management (DREAM)) designed to automatically and intelligently index huge repositories of special effects video clips, based on their semantic content, using a network of scalable ontologies to enable intelligent retrieval. The DREAM Demonstrator has been evaluated as deployed in the film post-production phase to support the process of storage, indexing and retrieval of large data sets of special effects video clips as an exemplar application domain. This paper provides its performance and usability results and highlights the scope for future enhancements of the DREAM architecture which has proven successful in its first and possibly most challenging proving ground, namely film production, where it is already in routine use within our test bed Partners' creative processes. (C) 2009 Published by Elsevier B.V.

Information system evolvability, feedback and pattern languages

Information systems for business are frequently heavily reliant on software. Two important feedback-related effects of embedding software in a business process are identified. First, the system dynamics of the software maintenance process can become complex, particularly in the number and scope of the feedback loops. Secondly, responsiveness to feedback can have a big effect on the evolvability of the information system. Ways have been explored to provide an effective mechanism for improving the quality of feedback between stakeholders during software maintenance. Understanding can be improved by using representations of information systems that are both service-based and architectural in scope. The conflicting forces that encourage change or stability can be resolved using patterns and pattern languages. A morphology of information systems pattern languages has been described to facilitate the identification and reuse of patterns and pattern languages. The kind of planning process needed to achieve consensus on a system's evolution is also considered.

Characterization and classification of tumor lesions using computerized fractal-based texture analysis and support vector machines in digital mammograms

Objective: This paper presents a detailed study of fractal-based methods for texture characterization of mammographic mass lesions and architectural distortion. The purpose of this study is to explore the use of fractal and lacunarity analysis for the characterization and classification of both tumor lesions and normal breast parenchyma in mammography. Materials and methods: We conducted comparative evaluations of five popular fractal dimension estimation methods for the characterization of the texture of mass lesions and architectural distortion. We applied the concept of lacunarity to the description of the spatial distribution of the pixel intensities in mammographic images. These methods were tested with a set of 57 breast masses and 60 normal breast parenchyma (dataset1), and with another set of 19 architectural distortions and 41 normal breast parenchyma (dataset2). Support vector machines (SVM) were used as a pattern classification method for tumor classification. Results: Experimental results showed that the fractal dimension of region of interest (ROIs) depicting mass lesions and architectural distortion was statistically significantly lower than that of normal breast parenchyma for all five methods. Receiver operating characteristic (ROC) analysis showed that fractional Brownian motion (FBM) method generated the highest area under ROC curve (A z = 0.839 for dataset1, 0.828 for dataset2, respectively) among five methods for both datasets. Lacunarity analysis showed that the ROIs depicting mass lesions and architectural distortion had higher lacunarities than those of ROIs depicting normal breast parenchyma. The combination of FBM fractal dimension and lacunarity yielded the highest A z value (0.903 and 0.875, respectively) than those based on single feature alone for both given datasets. The application of the SVM improved the performance of the fractal-based features in differentiating tumor lesions from normal breast parenchyma by generating higher A z value. Conclusion: FBM texture model is the most appropriate model for characterizing mammographic images due to self-affinity assumption of the method being a better approximation. Lacunarity is an effective counterpart measure of the fractal dimension in texture feature extraction in mammographic images. The classification results obtained in this work suggest that the SVM is an effective method with great potential for classification in mammographic image analysis.

A neurofuzzy network knowledge extraction and extended gram-Schmidt algorithm for model subspace decomposition

This paper introduces a new neurofuzzy model construction and parameter estimation algorithm from observed finite data sets, based on a Takagi and Sugeno (T-S) inference mechanism and a new extended Gram-Schmidt orthogonal decomposition algorithm, for the modeling of a priori unknown dynamical systems in the form of a set of fuzzy rules. The first contribution of the paper is the introduction of a one to one mapping between a fuzzy rule-base and a model matrix feature subspace using the T-S inference mechanism. This link enables the numerical properties associated with a rule-based matrix subspace, the relationships amongst these matrix subspaces, and the correlation between the output vector and a rule-base matrix subspace, to be investigated and extracted as rule-based knowledge to enhance model transparency. The matrix subspace spanned by a fuzzy rule is initially derived as the input regression matrix multiplied by a weighting matrix that consists of the corresponding fuzzy membership functions over the training data set. Model transparency is explored by the derivation of an equivalence between an A-optimality experimental design criterion of the weighting matrix and the average model output sensitivity to the fuzzy rule, so that rule-bases can be effectively measured by their identifiability via the A-optimality experimental design criterion. The A-optimality experimental design criterion of the weighting matrices of fuzzy rules is used to construct an initial model rule-base. An extended Gram-Schmidt algorithm is then developed to estimate the parameter vector for each rule. This new algorithm decomposes the model rule-bases via an orthogonal subspace decomposition approach, so as to enhance model transparency with the capability of interpreting the derived rule-base energy level. This new approach is computationally simpler than the conventional Gram-Schmidt algorithm for resolving high dimensional regression problems, whereby it is computationally desirable to decompose complex models into a few submodels rather than a single model with large number of input variables and the associated curse of dimensionality problem. Numerical examples are included to demonstrate the effectiveness of the proposed new algorithm.

Robust neurofuzzy rule base knowledge extraction and estimation using subspace decomposition combined with regularization and D-optimality

A new robust neurofuzzy model construction algorithm has been introduced for the modeling of a priori unknown dynamical systems from observed finite data sets in the form of a set of fuzzy rules. Based on a Takagi-Sugeno (T-S) inference mechanism a one to one mapping between a fuzzy rule base and a model matrix feature subspace is established. This link enables rule based knowledge to be extracted from matrix subspace to enhance model transparency. In order to achieve maximized model robustness and sparsity, a new robust extended Gram-Schmidt (G-S) method has been introduced via two effective and complementary approaches of regularization and D-optimality experimental design. Model rule bases are decomposed into orthogonal subspaces, so as to enhance model transparency with the capability of interpreting the derived rule base energy level. A locally regularized orthogonal least squares algorithm, combined with a D-optimality used for subspace based rule selection, has been extended for fuzzy rule regularization and subspace based information extraction. By using a weighting for the D-optimality cost function, the entire model construction procedure becomes automatic. Numerical examples are included to demonstrate the effectiveness of the proposed new algorithm.

A review and implementation of swarm pattern formation and transformation models

Purpose: The purpose of this paper is to address a classic problem – pattern formation identified by researchers in the area of swarm robotic systems – and is also motivated by the need for mathematical foundations in swarm systems. Design/methodology/approach: The work is separated out as inspirations, applications, definitions, challenges and classifications of pattern formation in swarm systems based on recent literature. Further, the work proposes a mathematical model for swarm pattern formation and transformation. Findings: A swarm pattern formation model based on mathematical foundations and macroscopic primitives is proposed. A formal definition for swarm pattern transformation and four special cases of transformation are introduced. Two general methods for transforming patterns are investigated and a comparison of the two methods is presented. The validity of the proposed models, and the feasibility of the methods investigated are confirmed on the Traer Physics and Processing environment. Originality/value: This paper helps in understanding the limitations of existing research in pattern formation and the lack of mathematical foundations for swarm systems. The mathematical model and transformation methods introduce two key concepts, namely macroscopic primitives and a mathematical model. The exercise of implementing the proposed models on physics simulator is novel.

Comprehension of reflexives and pronouns in sequential bilingual children: do they pattern similarly to L1 children, L2 adults, or children with specific language impairment?

This paper investigates how sequential bilingual (L2) Turkish-English children comprehend English reflexives and pronouns and tests whether they pattern similarly to monolingual (L1) children, L2 adults, or children with Specific Language Impairment (SLI). Thirty nine 6- to 9-year-old L2 children with an age of onset of 30-48 months and exposure to English of 30-72 months and 33 L1 age-matched control children completed the Advanced Syntactic Test of Pronominal Reference-Revised (van der Lely, 1997). The L2 children’s performance was compared to L2 adults from Demirci (2001) and children with SLI from van der Lely & Stollwerck (1997). The L2 children’s performance in the comprehension of reflexives was almost identical to their age-matched controls, and differed from L2 adults and children with SLI. In the comprehension of pronouns, L2 children showed an asymmetry between referential and quantificational NPs, a pattern attested in younger L1 children and children with SLI. Our study provides evidence that the development of comprehension of reflexives and pronouns in these children resembles monolingual L1 acquisition and not adult L2 acquisition or acquisition of children with SLI.

Production of relative clauses in monolingual Turkish children

Research on the production of relative clauses (RCs) has shown that in English, although children start using intransitive RCs at an earlier age, more complex, bi-propositional object RCs appear later (Hamburger & Crain, 1982; Diessel and Tomasello, 2005), and children use resumptive pronouns both in acceptable and unacceptable ways (McKee, McDaniel, & Snedeker, 1998; McKee & McDaniel, 2001). To date, it is unclear whether or not the same picture emerges in Turkish, a language with an SOV word-order and overt case marking. Some studies suggested that subject RCs are more frequent in adults and children (Slobin, 1986) and yield a better performance than object RCs (Özcan, 1996), but others reported the opposite pattern (Ekmekçi, 1990). Our study addresses this issue in Turkish children and adults, and uses participants’ errors to account for the emerging asymmetry between subject and object RCs. 37 5-to-8 year old monolingual Turkish children and 23 adult controls participated in a novel elicitation task involving cards, each consisting of four different pictures (see Figure 1). There were two sets of cards, one for the participant and one for the researcher. The former had animals with accessories (e.g., a hat) whereas the latter had no accessories. Participants were instructed to hold their card without showing it to the researcher and describe the animals with particular accessories. This prompted the use of subject and object RCs. The researcher had to identify the animals in her card (see Figure 2). A preliminary repeated measures ANOVA with the factor Group (pre-school, primary-school children) showed no differences between the groups in the use of RCs (p>.1), who were therefore collapsed into one for further analyses. A repeated measures ANOVA with the factors Group (children, adults) and RC-Type (Subject, Object) showed that children used fewer RCs than adults (F(1,58)=7.54, p<.01), and both groups used fewer object than subject RCs (F(1,58)=22.46, p<.001), but there was no Group by RC-Type interaction (see Figure 3). A similar ANOVA on the rate of grammatical RCs showed a main effect of Group (F(1,58)=77.25, p<.001), a main effect of RC-Type (F(1,58)=66.33, p<.001), and an interaction of Group by RC-Type (F(1,58)=64.6, p<.001) (see Figure 4). Children made more errors than adults in object RCs (F(1,58)=87.01, p<.001), and children made more errors in object compared to subject RCs (F(1,36)=106.35, p<.001), but adults did not show this asymmetry. The error analysis revealed that children systematically avoided the object-relativizing morpheme –DIK, which requires possessive agreement with the genitive-marked subject. They also used resumptive pronouns and resumptive full-DPs in the extraction site similarly to English children (see Figure 5). These findings are in line with Slobin (1986) and Özcan (1996). Children’s errors suggest that they avoid morphosyntactic complexity of object RCs and try to preserve the canonical word order by inserting resumptive pronouns in the extraction site. Finally, cross-linguistic similarity in the acquisition of RCs in typologically different languages suggests a higher accessibility of subject RCs both at the structural (Keenan and Comrie, 1977) and conceptual level (Bock and Warren, 1986).

Video activity extraction and reporting with incremental unsupervised learning

The present work presents a new method for activity extraction and reporting from video based on the aggregation of fuzzy relations. Trajectory clustering is first employed mainly to discover the points of entry and exit of mobiles appearing in the scene. In a second step, proximity relations between resulting clusters of detected mobiles and contextual elements from the scene are modeled employing fuzzy relations. These can then be aggregated employing typical soft-computing algebra. A clustering algorithm based on the transitive closure calculation of the fuzzy relations allows building the structure of the scene and characterises the ongoing different activities of the scene. Discovered activity zones can be reported as activity maps with different granularities thanks to the analysis of the transitive closure matrix. Taking advantage of the soft relation properties, activity zones and related activities can be labeled in a more human-like language. We present results obtained on real videos corresponding to apron monitoring in the Toulouse airport in France.

Perturbation, extraction and refinement of invariant pairs for matrix polynomials

Generalizing the notion of an eigenvector, invariant subspaces are frequently used in the context of linear eigenvalue problems, leading to conceptually elegant and numerically stable formulations in applications that require the computation of several eigenvalues and/or eigenvectors. Similar benefits can be expected for polynomial eigenvalue problems, for which the concept of an invariant subspace needs to be replaced by the concept of an invariant pair. Little has been known so far about numerical aspects of such invariant pairs. The aim of this paper is to fill this gap. The behavior of invariant pairs under perturbations of the matrix polynomial is studied and a first-order perturbation expansion is given. From a computational point of view, we investigate how to best extract invariant pairs from a linearization of the matrix polynomial. Moreover, we describe efficient refinement procedures directly based on the polynomial formulation. Numerical experiments with matrix polynomials from a number of applications demonstrate the effectiveness of our extraction and refinement procedures.