Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks

Cette th��se ��tudie des mod��les de s��quences de haute dimension bas��s sur des r��seaux de neurones r��currents (RNN) et leur application �� la musique et �� la parole. Bien qu'en principe les RNN puissent repr��senter les d��pendances �� long terme et la dynamique temporelle complexe propres aux s��quences d'int��r��t comme la vid��o, l'audio et la langue naturelle, ceux-ci n'ont pas ��t�� utilis��s �� leur plein potentiel depuis leur introduction par Rumelhart et al. (1986a) en raison de la difficult�� de les entra��ner efficacement par descente de gradient. R��cemment, l'application fructueuse de l'optimisation Hessian-free et d'autres techniques d'entra��nement avanc��es ont entra��n�� la recrudescence de leur utilisation dans plusieurs syst��mes de l'��tat de l'art. Le travail de cette th��se prend part �� ce d��veloppement. L'id��e centrale consiste �� exploiter la flexibilit�� des RNN pour apprendre une description probabiliste de s��quences de symboles, c'est-��-dire une information de haut niveau associ��e aux signaux observ��s, qui en retour pourra servir d'�� priori pour am��liorer la pr��cision de la recherche d'information. Par exemple, en mod��lisant l'��volution de groupes de notes dans la musique polyphonique, d'accords dans une progression harmonique, de phon��mes dans un ��nonc�� oral ou encore de sources individuelles dans un m��lange audio, nous pouvons am��liorer significativement les m��thodes de transcription polyphonique, de reconnaissance d'accords, de reconnaissance de la parole et de s��paration de sources audio respectivement. L'application pratique de nos mod��les �� ces t��ches est d��taill��e dans les quatre derniers articles pr��sent��s dans cette th��se. Dans le premier article, nous rempla��ons la couche de sortie d'un RNN par des machines de Boltzmann restreintes conditionnelles pour d��crire des distributions de sortie multimodales beaucoup plus riches. Dans le deuxi��me article, nous ��valuons et proposons des m��thodes avanc��es pour entra��ner les RNN. Dans les quatre derniers articles, nous examinons diff��rentes fa��ons de combiner nos mod��les symboliques �� des r��seaux profonds et �� la factorisation matricielle non-n��gative, notamment par des produits d'experts, des architectures entr��e/sortie et des cadres g��n��ratifs g��n��ralisant les mod��les de Markov cach��s. Nous proposons et analysons ��galement des m��thodes d'inf��rence efficaces pour ces mod��les, telles la recherche vorace chronologique, la recherche en faisceau �� haute dimension, la recherche en faisceau ��lagu�� et la descente de gradient. Finalement, nous abordons les questions de l'��tiquette biais��e, du ma��tre imposant, du lissage temporel, de la r��gularisation et du pr��-entra��nement.

Una aproximaci��n evolucionista para la generaci��n autom��tica de sentencias SQL a partir de ejemplos

En la actualidad, el uso de las tecnolog��as ha sido primordial para el avance de las sociedades, estas han permitido que personas sin conocimientos inform��ticos o usuarios llamados ���no expertos��� se interesen en su uso, raz��n por la cual los investigadores cient��ficos se han visto en la necesidad de producir estudios que permitan la adaptaci��n de sistemas, a la problem��tica existente dentro del ��mbito inform��tico. Una necesidad recurrente de todo usuario de un sistema es la gesti��n de la informaci��n, la cual se puede administrar por medio de una base de datos y lenguaje espec��fico, como lo es el SQL (Structured Query Language), pero esto obliga al usuario sin conocimientos a acudir a un especialista para su dise��o y construcci��n, lo cual se ve reflejado en costos y m��todos complejos, entonces se plantea una pregunta ��qu�� hacer cuando los proyectos son peque��as y los recursos y procesos son limitados? Teniendo como base la investigaci��n realizada por la universidad de Washington[39], donde sintetizan sentencias SQL a partir de ejemplos de entrada y salida, se pretende con esta memoria automatizar el proceso y aplicar una t��cnica diferente de aprendizaje, para lo cual utiliza una aproximaci��n evolucionista, donde la aplicaci��n de un algoritmo gen��tico adaptado origina sentencias SQL v��lidas que responden a las condiciones establecidas por los ejemplos de entrada y salida dados por el usuario. Se obtuvo como resultado de la aproximaci��n, una herramienta denominada EvoSQL que fue validada en este estudio. Sobre los 28 ejercicios empleados por la investigaci��n [39], 23 de los cuales se obtuvieron resultados perfectos y 5 ejercicios sin ��xito, esto representa un 82.1% de efectividad. Esta efectividad es superior en un 10.7% al establecido por la herramienta desarrollada en [39] SQLSynthesizer y 75% m��s alto que la herramienta siguiente m��s pr��xima Query by Output QBO[31]. El promedio obtenido en la ejecuci��n de cada ejercicio fue de 3 minutos y 11 segundos, este tiempo es superior al establecido por SQLSynthesizer; sin embargo, en la medida un algoritmo gen��tico supone la existencia de fases que ampl��an los rangos de tiempos, por lo cual el tiempo obtenido es aceptable con relaci��n a las aplicaciones de este tipo. En conclusi��n y seg��n lo anteriormente expuesto, se obtuvo una herramienta autom��tica con una aproximaci��n evolucionista, con buenos resultados y un proceso simple para el usuario ���no experto���.

Genetic Programming meets Model-Driven Development

Genetic programming is known to provide good solutions for many problems like the evolution of network protocols and distributed algorithms. In such cases it is most likely a hardwired module of a design framework that assists the engineer to optimize specific aspects of the system to be developed. It provides its results in a fixed format through an internal interface. In this paper we show how the utility of genetic programming can be increased remarkably by isolating it as a component and integrating it into the model-driven software development process. Our genetic programming framework produces XMI-encoded UML models that can easily be loaded into widely available modeling tools which in turn posses code generation as well as additional analysis and test capabilities. We use the evolution of a distributed election algorithm as an example to illustrate how genetic programming can be combined with model-driven development. This example clearly illustrates the advantages of our approach ��� the generation of source code in different programming languages.

A User-Centric Perspective on Parallel Programming with Focus on OpenMP

The process of developing software that takes advantage of multiple processors is commonly referred to as parallel programming. For various reasons, this process is much harder than the sequential case. For decades, parallel programming has been a problem for a small niche only: engineers working on parallelizing mostly numerical applications in High Performance Computing. This has changed with the advent of multi-core processors in mainstream computer architectures. Parallel programming in our days becomes a problem for a much larger group of developers. The main objective of this thesis was to find ways to make parallel programming easier for them. Different aims were identified in order to reach the objective: research the state of the art of parallel programming today, improve the education of software developers about the topic, and provide programmers with powerful abstractions to make their work easier. To reach these aims, several key steps were taken. To start with, a survey was conducted among parallel programmers to find out about the state of the art. More than 250 people participated, yielding results about the parallel programming systems and languages in use, as well as about common problems with these systems. Furthermore, a study was conducted in university classes on parallel programming. It resulted in a list of frequently made mistakes that were analyzed and used to create a programmers' checklist to avoid them in the future. For programmers' education, an online resource was setup to collect experiences and knowledge in the field of parallel programming - called the Parawiki. Another key step in this direction was the creation of the Thinking Parallel weblog, where more than 50.000 readers to date have read essays on the topic. For the third aim (powerful abstractions), it was decided to concentrate on one parallel programming system: OpenMP. Its ease of use and high level of abstraction were the most important reasons for this decision. Two different research directions were pursued. The first one resulted in a parallel library called AthenaMP. It contains so-called generic components, derived from design patterns for parallel programming. These include functionality to enhance the locks provided by OpenMP, to perform operations on large amounts of data (data-parallel programming), and to enable the implementation of irregular algorithms using task pools. AthenaMP itself serves a triple role: the components are well-documented and can be used directly in programs, it enables developers to study the source code and learn from it, and it is possible for compiler writers to use it as a testing ground for their OpenMP compilers. The second research direction was targeted at changing the OpenMP specification to make the system more powerful. The main contributions here were a proposal to enable thread-cancellation and a proposal to avoid busy waiting. Both were implemented in a research compiler, shown to be useful in example applications, and proposed to the OpenMP Language Committee.

A Virtual Machine for a Type-omega Denotational Proof Language

In this thesis, I designed and implemented a virtual machine (VM) for a monomorphic variant of Athena, a type-omega denotational proof language (DPL). This machine attempts to maintain the minimum state required to evaluate Athena phrases. This thesis also includes the design and implementation of a compiler for monomorphic Athena that compiles to the VM. Finally, it includes details on my implementation of a read-eval-print loop that glues together the VM core and the compiler to provide a full, user-accessible interface to monomorphic Athena. The Athena VM provides the same basis for DPLs that the SECD machine does for pure, functional programming and the Warren Abstract Machine does for Prolog.

Behavioral Measures and their Correlation with IPM Iteration Counts on Semi-Definite Programming Problems

We study four measures of problem instance behavior that might account for the observed differences in interior-point method (IPM) iterations when these methods are used to solve semidefinite programming (SDP) problem instances: (i) an aggregate geometry measure related to the primal and dual feasible regions (aspect ratios) and norms of the optimal solutions, (ii) the (Renegar-) condition measure C(d) of the data instance, (iii) a measure of the near-absence of strict complementarity of the optimal solution, and (iv) the level of degeneracy of the optimal solution. We compute these measures for the SDPLIB suite problem instances and measure the correlation between these measures and IPM iteration counts (solved using the software SDPT3) when the measures have finite values. Our conclusions are roughly as follows: the aggregate geometry measure is highly correlated with IPM iterations (CORR = 0.896), and is a very good predictor of IPM iterations, particularly for problem instances with solutions of small norm and aspect ratio. The condition measure C(d) is also correlated with IPM iterations, but less so than the aggregate geometry measure (CORR = 0.630). The near-absence of strict complementarity is weakly correlated with IPM iterations (CORR = 0.423). The level of degeneracy of the optimal solution is essentially uncorrelated with IPM iterations.

Forms of ethnic prejudice : Assessing the dimensionality of a Spanish-language version of the Blatant and Subtle Prejudice Scale

Resumen tomado de la publicaci??n

Response to instruction, English language learners and disproportionate representation : The role of assessment

Resumen tomado de la publicaci??n

A Self-Scoring Exercise On APA Style, Research Language, and Referencing for Undergraduate Students

Based on examples provided by 27 graduate psychology faculty, this self-test incorporates many of the more common errors in style, language, and referencing found in student papers. Taking this self-test helps students to recognize common errors and encourages them to refer the APA Publication Manual on a regular basis. In addition, students begin to think about how to use correctly the language of psychological research. This self-test should take about 30 minutes to complete and score. It is composed of three parts: a) a mock Discussion section, where students are asked to act as editors and find the errors, p. 2 (10 minutes). b) a corrected Discussion section, where students find the errors they missed, p. 3 (5 minutes) and, c) a full description of each error with illustrations of correct usage, pp. 4-7 (15 minutes). This exercise assumes some knowledge of APA style. Thus, it is best-suited for advanced undergraduates who need to write research reports and all levels of graduate students. It may be taken at home or in class. Although the self-test is designed to be fully self-directed, instructors may wish to use it at the beginning or end of a classroom discussion on APA style. It could also be used in a pre-test-post-test fashion to evaluate students learning over the course of a term.

Programming Principles: Loops and Arrays

In this session we look at the different types of loop in the Java language, and see how they can be used to iterate over Arrays.

COMP1004 Programming Principles

These are the resources used for the Computer Science course Programming Principles, designed to teach students the fundamentals of computer programming and object orientation via learning the Java language. We also touch on some software engineering basics, such as patterns, software design and testing. The course assumes no previous knowledge of programming, but there is a fairly steep learning curve, and students are encouraged to practice, practice, practice!

Language acquisition through topical syllabus.

El presente proyecto viene a ser una labor ya experimentada desde hace cuatro a��os. Se trata de una aproximaci��n a la Lengua Inglesa de COU de una forma totalmente innovadora. A partir de una serie de t��picos o temas de actualidad o considerados de inter��s por los alumnos, se elaboran unas tareas o actividades a realizar por el alumnado. Estos t��picos se trabajan a partir de material aut��ntico, tanto escrito como audiovisual, atendiendo tanto a razones de complejidad ling����stica como de complejidad conceptual. Su contenido oscilar�� desde lo puramente morfosint��ctico-l��xico hasta el m��s puramente comunicativo. El proyecto afecta a todos los grupos de COU de los siguientes centros de Bachillerato: Anaga, Tom��s de Iriarte, Padre Anchieta, Tejina, Cabrera Pinto, y de Formaci��n Profesional Los Realejos, todos de la isla de Tenerife. Se llev�� a cabo una evaluaci��n continua a trav��s de observaci��n directa, participaci��n y trabajo en clase, pruebas orales y escritas, etc. No constan resultados ni valoraciones al respecto.

Second lives and newer islands : language, culture and identity through migration literature. 'Second lives and newer islands : lengua, cultura e identidad a trav??s de la literatura de migraci??n'

Resumen en espa??ol

The role of triggering strategies in the foreing language class. 'El papel de las estrategias de provocaci��n en la clase de lengua extranjera'.

Se analizan los procesos de aprendizaje y las estrategias para la adquisici��n de la lengua extranjera.

Towards an interface of language and literature. 'Hacia una interfaz de la Lengua y la Literatura'.

Se analiza la conveniencia de usar textos simplificados o en su formato original para propositos pedag��gicos dentro de la ense��anza de la segunda lengua. El autor considera que ambas clases de textos (simplificados u originales) pueden formar el nivel intermedio de aprendizaje de lenguas extranjeras.