Mechanisms for multimodality : taking fiction to another dimension

We present methods for automatically constructing representations of fiction books in a range of modalities: audibly, graphically and as 3D virtual environments. The correspondence between the sequential ordering of events against the order of events presented in the text is used to correctly resolve the dynamic interactions for each representation. Synthesised audio created from the fiction text is used to calibrate the base time-line against which the other forms of media are correctly aligned. The audio stream is based on speech synthesis using the text of the book, and is enhanced using distinct voices for the different characters in a book. Sound effects are included automatically. The graphical representation represents the text (as subtitles), identifies active characters and provides visual feedback of the content of the story. Dynamic virtual environments conform to the constraints implied by the story, and are used as a source of further visual content. These representations are all aligned to a common time-line, and combined using sequencing facilities to provide a multimodal version of the original text.

Automating the creation of 3D animation from annotated fiction text

This paper describes a strategy for automatically converting fiction text into 3D animations. It assumes the existence of fiction text annotated with avatar, object, setting, transition and relation annotations, and presents a transformation process that converts annotated text into quantified constraint systems, the solutions to which are used in the population of 3D environments. Constraint solutions are valid over temporal intervals, ensuring that consistent dynamic behaviour is produced. A substantial level of automation is achieved, while providing opportunities for creative manual intervention in animation process. The process is demonstrated using annotated examples drawn from popular fiction text that are converted into animation sequences, confirming that the desired results can be achieved with only high-level human direction.

A naïve, salience-based method for speaker identification in fiction books

This paper presents a salience-based technique for the annotation of directly quoted speech from fiction text. In particular, this paper determines to what extent a naïve (without the use of complex machine learning or knowledge-based techniques) scoring technique can be used for the identification of the speaker of speech quotes. The presented technique makes use of a scoring technique, similar to that commonly found in knowledge-poor anaphora resolution research, as well as a set of hand-coded rules for the final identification of the speaker of each quote in the text. Speaker identification is shown to be achieved using three tasks: the identification of a speech-verb associated with a quote with a recall of 94.41%; the identification of the actor associated with a quote with a recall of 88.22%; and the selection of a speaker with an accuracy of 79.40%.

Constraint-based conversion of fiction text to a time-based graphical representation

This paper presents a method for converting unrestricted fiction text into a time-based graphical form. Key concepts extracted from the text are used to formulate constraints describing the interaction of entities in a scene. The solution of these constraints over their respective time intervals provides the trajectories for these entities in a graphical representation.

Three types of entity are extracted from fiction books to describe the scene, namely Avatars, Areas and Objects. We present a novel method for modeling the temporal aspect of a fiction story using multiple time-line representations after which the information extracted regarding entities and time-lines is used to formulate constraints. A constraint solving technique based on interval arithmetic is used to ensure that the behaviour of the entities satisfies the constraints over multiple universally quantified time intervals. This approach is demonstrated by finding solutions to multiple time-based constraints, and represents a new contribution to the field of Text-to-Scene conversion. An example of the automatically produced graphical output is provided in support of our constraint-based conversion scheme.

Hierarchical rule generalisation for speaker identification in fiction books

This paper presents a hierarchical pattern matching and generalisation technique which is applied to the problem of locating the correct speaker of quoted speech found in fiction books. Patterns from a training set are generalised to create a small number of rules, which can be used to identify items of interest within the text. The pattern matching technique is applied to finding the Speech-Verb, Actor and Speaker of quotes found in ction books. The technique performs well over the training data, resulting in rule-sets many times smaller than the training set, but providing very high accuracy. While the rule-set generalised from one book is less effective when applied to different books than an approach based on hand coded heuristics, performance is comparable when testing on data closely related to the training set.

Identifying annotations for adventure game generation from fiction text

Recent advancements in Text-to-Scene research have lead to the development of systems which automatically extract key concepts from the text of a fiction book and generate computer animated movies depicting the story. Extracting such annotations from raw fiction text is a laborious process and so in this work we evaluate appropriate candidates to serve as the basis for the required annotations for generating interactive virtual worlds.

We validate our choice by generating adventure games: inter-active virtual worlds which create a stylized representation of the environment described in the text, populate it with characters related to the story and define game goals related to the plot of the fiction story. Our prototype produces a fully playable game, making use of an existing open-source game engine.

The process is evaluated using user tests in which participants are asked to measure the accuracy with which the game represents the events, characters and goals described in the story. The response indicates that the chosen annotation set is sufficient to define a game that is a plausibly acceptable representation of the text.

A method for automatically creating 3D animated scenes from annotated fiction text

This paper describes a strategy for automatically converting fiction text into 3D animations. It assumes the existence of fiction text annotated with avatar, object, setting, transition and relation annotations, and presents a transformation process that converts annotated text into quantified constraint systems, the solutions to which are used in the population of 3D environments. Constraint solutions are valid over temporal intervals, ensuring that consistent dynamic behaviour is produced. A substantial level of automation is achieved, while providing opportunities for creative manual intervention in animation process. The process is demonstrated using annotated examples drawn from popular fiction text that are converted into animation sequences, confirming that the desired results can be achieved with only highlevel human direction.

Discriminating DDoS attack traffic from flash crowd through packet arrival patterns

Current DDoS attacks are carried out by attack tools, worms and botnets using different packet-transmission strategies and various forms of attack packets to beat defense systems. These problems lead to defense systems requiring various detection methods in order to identify attacks. Moreover, DDoS attacks can mix their traffics during flash crowds. By doing this, the complex defense system cannot detect the attack traffic in time. In this paper, we propose a behavior based detection that can discriminate DDoS attack traffic from traffic generated by real users. By using Pearson's correlation coefficient, our comparable detection methods can extract the repeatable features of the packet arrivals. The extensive simulations were tested for the accuracy of detection. We then performed experiments with several datasets and our results affirm that the proposed method can differentiate traffic of an attack source from legitimate traffic with a quick response. We also discuss approaches to improve our proposed methods at the conclusion of this paper.

Ten science fiction transitions SFX

Synthesised vocals for science fiction styled projects in mind. 10 different types of transitions that could be used as laser guns, vehicle transitions and whooshes.

Alarm for horror and science fiction SFX

Alarm for horror and science fiction SFX

The science of our own fiction : affective experiments enacted through creative research

This paper will explore the ways in which art may be understood as an ongoing experiment that interacts with the plasticity of the body to prompt change and affect the body-environment relationship. The arts offer an approach to research that recognizes the importance of the affect in studies of perception and action, self-organization and selection. An affective approach to experimentation would connect cognitive activity to the material processes of the environment in a science of our own fiction. This connection becomes the basis of affective experiments, which aim to yield new insights by merging the creative researcher with self-affecting-experimenter. To this end, I will discuss the scientific objectives of the “rubber hand”, and the ‘mirror-box” experiments are contrasted with work by artists-turned-architects Arakawa and Gins and three of my creative projects to suggest how creative research might enact embodied change. Throughout the paper I will argue that cognitive processes such as attention, selection, decision and judgment are ripe for re-entry and experimentation through an embodied approach to acquiring knowledge that is particular to the arts.