Building an Interactive, Three-dimensional Virtual World

The movement of graphics and audio programming towards three dimensions is to better simulate the way we experience our world. In this project I looked to use methods for coming closer to such simulation via realistic graphics and sound combined with a natural interface. I did most of my work on a Dell OptiPlex with an 800 MHz Pentium III processor and an NVIDlA GeForce 256 AGP Plus graphics accelerator -high end products in the consumer market as of April 2000. For graphics, I used OpenGL [1], an open·source, multi-platform set of graphics libraries that is relatively easy to use, coded in C . The basic engine I first put together was a system to place objects in a scene and to navigate around the scene in real time. Once I accomplished this, I was able to investigate specific techniques for making parts of a scene more appealing.

Promoting Programming Learning. Engagement, Automatic Assessment with Immediate Feedback in Visualizations

The skill of programming is a key asset for every computer science student. Many studies have shown that this is a hard skill to learn and the outcomes of programming courses have often been substandard. Thus, a range of methods and tools have been developed to assist students’ learning processes. One of the biggest fields in computer science education is the use of visualizations as a learning aid and many visualization based tools have been developed to aid the learning process during last few decades. Studies conducted in this thesis focus on two different visualizationbased tools TRAKLA2 and ViLLE. This thesis includes results from multiple empirical studies about what kind of effects the introduction and usage of these tools have on students’ opinions and performance, and what kind of implications there are from a teacher’s point of view. The results from studies in this thesis show that students preferred to do web-based exercises, and felt that those exercises contributed to their learning. The usage of the tool motivated students to work harder during their course, which was shown in overall course performance and drop-out statistics. We have also shown that visualization-based tools can be used to enhance the learning process, and one of the key factors is the higher and active level of engagement (see. Engagement Taxonomy by Naps et al., 2002). The automatic grading accompanied with immediate feedback helps students to overcome obstacles during the learning process, and to grasp the key element in the learning task. These kinds of tools can help us to cope with the fact that many programming courses are overcrowded with limited teaching resources. These tools allows us to tackle this problem by utilizing automatic assessment in exercises that are most suitable to be done in the web (like tracing and simulation) since its supports students’ independent learning regardless of time and place. In summary, we can use our course’s resources more efficiently to increase the quality of the learning experience of the students and the teaching experience of the teacher, and even increase performance of the students. There are also methodological results from this thesis which contribute to developing insight into the conduct of empirical evaluations of new tools or techniques. When we evaluate a new tool, especially one accompanied with visualization, we need to give a proper introduction to it and to the graphical notation used by tool. The standard procedure should also include capturing the screen with audio to confirm that the participants of the experiment are doing what they are supposed to do. By taken such measures in the study of the learning impact of visualization support for learning, we can avoid drawing false conclusion from our experiments. As computer science educators, we face two important challenges. Firstly, we need to start to deliver the message in our own institution and all over the world about the new – scientifically proven – innovations in teaching like TRAKLA2 and ViLLE. Secondly, we have the relevant experience of conducting teaching related experiment, and thus we can support our colleagues to learn essential know-how of the research based improvement of their teaching. This change can transform academic teaching into publications and by utilizing this approach we can significantly increase the adoption of the new tools and techniques, and overall increase the knowledge of best-practices. In future, we need to combine our forces and tackle these universal and common problems together by creating multi-national and multiinstitutional research projects. We need to create a community and a platform in which we can share these best practices and at the same time conduct multi-national research projects easily.

Modeling High-Dimensional Audio Sequences with Recurrent Neural Networks

Cette thèse étudie des modèles de séquences de haute dimension basés sur des réseaux de neurones récurrents (RNN) et leur application à la musique et à la parole. Bien qu'en principe les RNN puissent représenter les dépendances à long terme et la dynamique temporelle complexe propres aux séquences d'intérêt comme la vidéo, l'audio et la langue naturelle, ceux-ci n'ont pas été utilisés à leur plein potentiel depuis leur introduction par Rumelhart et al. (1986a) en raison de la difficulté de les entraîner efficacement par descente de gradient. Récemment, l'application fructueuse de l'optimisation Hessian-free et d'autres techniques d'entraînement avancées ont entraîné la recrudescence de leur utilisation dans plusieurs systèmes de l'état de l'art. Le travail de cette thèse prend part à ce développement. L'idée centrale consiste à exploiter la flexibilité des RNN pour apprendre une description probabiliste de séquences de symboles, c'est-à-dire une information de haut niveau associée aux signaux observés, qui en retour pourra servir d'à priori pour améliorer la précision de la recherche d'information. Par exemple, en modélisant l'évolution de groupes de notes dans la musique polyphonique, d'accords dans une progression harmonique, de phonèmes dans un énoncé oral ou encore de sources individuelles dans un mélange audio, nous pouvons améliorer significativement les méthodes de transcription polyphonique, de reconnaissance d'accords, de reconnaissance de la parole et de séparation de sources audio respectivement. L'application pratique de nos modèles à ces tâches est détaillée dans les quatre derniers articles présentés dans cette thèse. Dans le premier article, nous remplaçons la couche de sortie d'un RNN par des machines de Boltzmann restreintes conditionnelles pour décrire des distributions de sortie multimodales beaucoup plus riches. Dans le deuxième article, nous évaluons et proposons des méthodes avancées pour entraîner les RNN. Dans les quatre derniers articles, nous examinons différentes façons de combiner nos modèles symboliques à des réseaux profonds et à la factorisation matricielle non-négative, notamment par des produits d'experts, des architectures entrée/sortie et des cadres génératifs généralisant les modèles de Markov cachés. Nous proposons et analysons également des méthodes d'inférence efficaces pour ces modèles, telles la recherche vorace chronologique, la recherche en faisceau à haute dimension, la recherche en faisceau élagué et la descente de gradient. Finalement, nous abordons les questions de l'étiquette biaisée, du maître imposant, du lissage temporel, de la régularisation et du pré-entraînement.

Correlación entre señales de audio multi-microfónicas

Este proyecto pretende mostrar los desfases existentes entre señales de audio obtenidas de la misma fuente en distintos puntos distanciados entre sí. Para ello nos basamos en el análisis de la correlación de las señales de audio multi-microfónicas, para determinar los retrasos entre dichas señales. Durante las de tres partes diferentes que conforman este proyecto, explicaremos el dónde, cómo y por qué se produce este efecto en este tipo de señales. En la primera se presentan algunos de los conceptos teóricos necesarios para entender el desarrollo posterior, tales como la coherencia y correlación entre señales, los retardos de fase y la importancia del micro-tiempo. Además se explican diversas técnicas microfónicas que se utilizarán en la tercera parte. A lo largo de la segunda, se presenta el software desarrollado para determinar y corregir el retraso entre las señales que se deseen analizar. Para ello se ha escogido la herramienta de programación Matlab, ya que ha sido la más utilizada en la mayoría de las asignaturas que componen la titulación y por ello se posee el suficiente dominio de la misma. Además de presentar el propio software, al final de esta parte hay un manual de usuario del mismo, en el que se explica el manejo para posibles usos futuros por parte de otras personas interesadas. En la última parte se demuestra en varios casos reales, el estudio de la alineación de tomas multi-microfónicas en las cuales se produce en efecto que se intenta detectar y corregir. Aquí se realizan tres estudios de dicho fenómeno. En el primero se emplean señales digitales internas, concretamente ruido blanco, retrasando algunas muestras dichas señales unas de otras, para luego analizarlas con el software desarrollado y comprobar la eficacia del mismo. En el segundo se analizan la señales de audio obtenidas en el estudio de grabación de varios grupos de música moderna, mostrando los resultados del empleo del software en algunas de ellas, tales como las tomas de batería, bajo y guitarra. En el tercero se analizan las señales de audio obtenidas fuera del estudio de grabación, en donde no se dispone de las supuestas condiciones ideales que se tienen en el entorno que rodea a un estudio de grabación (acústicamente hablando). Se utilizan algunas de las técnicas microfónicas explicadas en el último apartado de la parte dedicada a los conceptos teóricos, para la grabación de una orquesta sinfónica, para luego analizar el efecto buscado mediante nuestro software, presentando los resultados obtenidos. De igual manera se realiza en el estudio con una agrupación coral de cuatro voces dentro de una Iglesia. ABSTRACT This project aims to show delays between audio signals obtained from the same source at diferent points spaced apart. To do this we rely on the analysis of the correlation of multi-microphonic audio signals, to determine the delay between these signals. During three diferent parts that make up this project, we will explain where, how and why this effect occurs in this type of signals. At the first part we present some of the theoretical concepts necessary to understand the subsequent development, such as coherence and correlation between signals, phase delays and the importance of micro-time. Also explains several microphone techniques to be used in the third part. During the second, it presents the software developed to determine and correct the delay between the signals that are desired to analyze. For this we have chosen the programming software Matlab , as it has been the most used in the majority of the subjects in the degree and therefore has suficient command of it. Besides presenting the software at the end of this part there is a user manual of it , which explains the handling for future use by other interested people. The last part is shown in several real cases, the study of aligning multi- microphonic sockets in which it is produced in effect trying to detect and correct. This includes three studies of this phenomenon. In the first internal digital signals are used, basically white noise, delaying some samples the signals from each other, then with software developed analyzing and verifying its efectiveness. In the second analyzes the audio signals obtained in the recording studio several contemporary bands, showing the results of using the software in some of them, such as the taking of drums, bass and guitar. In the third analyzes audio signals obtained outside the recording studio, where there are no ideal conditions alleged to have on the environment surrounding a recording studio (acoustically speaking). We use some of the microphone techniques explained in the last paragraph of the section on theoretical concepts, for the recording of a symphony orchestra, and then analyze the effect sought by our software, presenting the results. Similarly, in the study performed with a four-voice choir in a church.

Unidad de efectos de audio en Simulink

La tecnología moderna de computación ha permitido cambiar radicalmente la investigación tecnológica en todos los ámbitos. El proceso general utilizado previamente consistía en el desarrollo de prototipos analógicos, creando múltiples versiones del mismo hasta llegar al resultado adecuado. Este es un proceso costoso a nivel económico y de carga de trabajo. Es por ello por lo que el proceso de investigación actual aprovecha las nuevas tecnologías para lograr el objetivo final mediante la simulación. Gracias al desarrollo de software para la simulación de distintas áreas se ha incrementado el ritmo de crecimiento de los avances tecnológicos y reducido el coste de los proyectos en investigación y desarrollo. La simulación, por tanto, permite desarrollar previamente prototipos simulados con un coste mucho menor para así lograr un producto final, el cual será llevado a cabo en su ámbito correspondiente. Este proceso no sólo se aplica en el caso de productos con circuitería, si bien es utilizado también en productos programados. Muchos de los programas actuales trabajan con algoritmos concretos cuyo funcionamiento debe ser comprobado previamente, para después centrarse en la codificación del mismo. Es en este punto donde se encuentra el objetivo de este proyecto, simular algoritmos de procesado digital de la señal antes de la codificación del programa final. Los sistemas de audio están basados en su totalidad en algoritmos de procesado de la señal, tanto analógicos como digitales, siendo estos últimos los que están sustituyendo al mundo analógico mediante los procesadores y los ordenadores. Estos algoritmos son la parte más compleja del sistema, y es la creación de nuevos algoritmos la base para lograr sistemas de audio novedosos y funcionales. Se debe destacar que los grupos de desarrollo de sistemas de audio presentan un amplio número de miembros con cometidos diferentes, separando las funciones de programadores e ingenieros de la señal de audio. Es por ello por lo que la simulación de estos algoritmos es fundamental a la hora de desarrollar nuevos y más potentes sistemas de audio. Matlab es una de las herramientas fundamentales para la simulación por ordenador, la cual presenta utilidades para desarrollar proyectos en distintos ámbitos. Sin embargo, en creciente uso actualmente se encuentra el software Simulink, herramienta especializada en la simulación de alto nivel que simplifica la dificultad de la programación en Matlab y permite desarrollar modelos de forma más rápida. Simulink presenta una completa funcionalidad para el desarrollo de algoritmos de procesado digital de audio. Por ello, el objetivo de este proyecto es el estudio de las capacidades de Simulink para generar sistemas de audio funcionales. A su vez, este proyecto pretende profundizar en los métodos de procesado digital de la señal de audio, logrando al final un paquete de sistemas de audio compatible con los programas de edición de audio actuales. ABSTRACT. Modern computer technology has dramatically changed the technological research in multiple areas. The overall process previously used consisted of the development of analog prototypes, creating multiple versions to reach the proper result. This is an expensive process in terms of an economically level and workload. For this reason actual investigation process take advantage of the new technologies to achieve the final objective through simulation. Thanks to the software development for simulation in different areas the growth rate of technological progress has been increased and the cost of research and development projects has been decreased. Hence, simulation allows previously the development of simulated protoypes with a much lower cost to obtain a final product, which will be held in its respective field. This process is not only applied in the case of circuitry products, but is also used in programmed products. Many current programs work with specific algorithms whose performance should be tested beforehand, which allows focusing on the codification of the program. This is the main point of this project, to simulate digital signal processing algorithms before the codification of the final program. Audio systems are entirely based on signal processing, both analog and digital systems, being the digital systems which are replacing the analog world thanks to the processors and computers. This algorithms are the most complex part of every system, and the creation of new algorithms is the most important step to achieve innovative and functional new audio systems. It should be noted that development groups of audio systems have a large number of members with different roles, separating them into programmers and audio signal engineers. For this reason, the simulation of this algorithms is essential when developing new and more powerful audio systems. Matlab is one of the most important tools for computer simulation, which has utilities to develop projects in different areas. However, the use of the Simulink software is constantly growing. It is a simulation tool specialized in high-level simulations which simplifies the difficulty of programming in Matlab and allows the developing of models faster. Simulink presents a full functionality for the development of algorithms for digital audio processing. Therefore, the objective of this project is to study the posibilities of Simulink to generate funcional audio systems. In turn, this projects aims to get deeper into the methods of digital audio signal processing, making at the end a software package of audio systems compatible with the current audio editing software.

Implementación de efectos de audio en la tarjeta Raspberry Pi B+

Este proyecto consiste en el diseño e implementación de un procesador digital de efectos de audio en tiempo real orientado a instrumentos eléctricos tales como guitarras, bajos, teclados, etc. El procesador está basado en la tarjeta Raspberry Pi B+, ordenador de placa reducida de bajo coste, desarrollado en Reino unido y cuyo lanzamiento tuvo lugar en el año 2012. En primer lugar, ha sido necesario lograr que la tarjeta asuma la funcionalidad de un procesador de audio en tiempo real. Para ello se ha instalado un sistema operativo Linux orientado a Raspberry (Raspbian) y se ha hecho uso de Pure Data (Pd): lenguaje de programación gráfico que fue desarrollado en los años 90 por Miller Puckette con intención de ser enfocado a la creación de eventos multimedia y de música por computador. El papel que desempeña Pd es de capa intermedia entre el hardware y el software ya que se encarga de tomar bloques de N muestras del convertidor analógico/digital y encaminarlas a través del flujo de señal diseñado gráficamente. En segundo lugar, se han implementado diferentes efectos de audio de distintas características. Así pues, se encuentran efectos basados en retardos, filtros digitales y procesadores de dinámica. Concretamente, los efectos implementados son los siguientes: delay, flanger, vibrato, reverberador de Schroeder, filtros (paso bajo, paso alto y paso banda), ecualizador paramétrico y compresor y expansor de dinámica. Estos efectos han sido implementados en lenguaje C de acuerdo con la API de Pd. Con esto se ha conseguido obtener un objeto por cada efecto, el cual es “instanciado” en Pd pudiendo ejecutarlo en tiempo real. En este proyecto se expone la problemática que supone cada paso del diseño proponiendo soluciones válidas. Además se incluye una guía paso a paso para configurar la tarjeta y lograr realizar un bypass de señal y un efecto simple partiendo desde cero. ABSTRACT. This project involves the design and implementation of a digital real-time audio processor for electrical instruments (guitars, basses, keyboards, etc.). The processor is based on the Raspberry Pi B + card: low cost computer, developed in UK in 2012. First, it was necessary to make the cards assume the functionality of a real time audio processor. A Linux operating system called Raspberry (Raspbian) was installed. In this Project is used Pure Data (Pd): a graphical programming language developed in the 90s by Miller Puckette intending to be focused on creating multimedia and computer music events. The role of Pd is an intermediate layer between the hardware and the software. It is responsible for taking blocks of N samples of the analog/digital converter and route it through the signal flow. Secondly, it is necessary to implemented the different audio effects. There are delays based effects, digital filter and dynamics effects. Specifically, the implemented effects are: delay, flanger, vibrato, Schroeder reverb, filters (lowpass, highpass and bandpass), parametric equalizer and compressor and expander dynamics. These effects have been implemented in C language according to the Pd API. As a result, it has been obtained an object for each effect, which is instantiated in Pd. In this Project, the problems of every step are exposed with his corresponding solution. It is inlcuded a step-by-step guide to configure the card and achieve perform a bypass signal process and a simple effect.

Evolution of classifiers for pitch estimation of piano music using cartesian genetic programming

Pitch Estimation, also known as Fundamental Frequency (F0) estimation, has been a popular research topic for many years, and is still investigated nowadays. The goal of Pitch Estimation is to find the pitch or fundamental frequency of a digital recording of a speech or musical notes. It plays an important role, because it is the key to identify which notes are being played and at what time. Pitch Estimation of real instruments is a very hard task to address. Each instrument has its own physical characteristics, which reflects in different spectral characteristics. Furthermore, the recording conditions can vary from studio to studio and background noises must be considered. This dissertation presents a novel approach to the problem of Pitch Estimation, using Cartesian Genetic Programming (CGP).We take advantage of evolutionary algorithms, in particular CGP, to explore and evolve complex mathematical functions that act as classifiers. These classifiers are used to identify piano notes pitches in an audio signal. To help us with the codification of the problem, we built a highly flexible CGP Toolbox, generic enough to encode different kind of programs. The encoded evolutionary algorithm is the one known as 1 + , and we can choose the value for . The toolbox is very simple to use. Settings such as the mutation probability, number of runs and generations are configurable. The cartesian representation of CGP can take multiple forms and it is able to encode function parameters. It is prepared to handle with different type of fitness functions: minimization of f(x) and maximization of f(x) and has a useful system of callbacks. We trained 61 classifiers corresponding to 61 piano notes. A training set of audio signals was used for each of the classifiers: half were signals with the same pitch as the classifier (true positive signals) and the other half were signals with different pitches (true negative signals). F-measure was used for the fitness function. Signals with the same pitch of the classifier that were correctly identified by the classifier, count as a true positives. Signals with the same pitch of the classifier that were not correctly identified by the classifier, count as a false negatives. Signals with different pitch of the classifier that were not identified by the classifier, count as a true negatives. Signals with different pitch of the classifier that were identified by the classifier, count as a false positives. Our first approach was to evolve classifiers for identifying artifical signals, created by mathematical functions: sine, sawtooth and square waves. Our function set is basically composed by filtering operations on vectors and by arithmetic operations with constants and vectors. All the classifiers correctly identified true positive signals and did not identify true negative signals. We then moved to real audio recordings. For testing the classifiers, we picked different audio signals from the ones used during the training phase. For a first approach, the obtained results were very promising, but could be improved. We have made slight changes to our approach and the number of false positives reduced 33%, compared to the first approach. We then applied the evolved classifiers to polyphonic audio signals, and the results indicate that our approach is a good starting point for addressing the problem of Pitch Estimation.

Metodo de sincronização de cameras de video utilizando a banda de audio

Universidade Estadual de Campinas . Faculdade de Educação Física

Medidas de audição de pais de indivíduos com deficiência auditiva de herança autossômica recessiva

TEMA: avaliação audiológica de pais de indivíduos com perda auditiva de herança autossômica recessiva. OBJETIVO: estudar o perfil audiológico de pais de indivíduos com perda auditiva, de herança autossômica recessiva, inferida pela história familial ou por testes moleculares que detectaram mutação no gene GJB2, responsável por codificar a Conexina 26. MÉTODO: 36 indivíduos entre 30 e 60 anos foram avaliados e divididos em dois grupos: grupo controle, sem queixas auditivas e sem história familiar de deficiência auditiva, e grupo de estudos composto por pais heterozigotos em relação a genes de surdez de herança autossômica recessiva inespecífica ou portadores heterozigotos de mutação no gene da Conexina 26. Todos foram submetidos à audiometria tonal liminar (0,25kHz a 8), audiometria de altas freqüências (9kHz a 20) e emissões otoacústicas produtos de distorção (EOAPD). RESULTADOS: houve diferenças significativas na amplitude das EOAPD nas freqüências 1001 e 1501Hz entre os grupos, sendo maior a amplitude no grupo controle. Não houve diferença significativa entre os grupos para os limiares tonais de 0,25 a 20KHz. CONCLUSÃO: as EOAPD foram mais eficazes, em comparação com a audiometria tonal liminar, para detectar diferenças auditivas entre os grupos. Mais pesquisas são necessárias para verificar a confiabilidade destes dados.

Fatores relacionados à autopercepção da audição entre idosos do município de São Paulo - Projeto SABE

Este artigo relata a influência de fatores sociodemográficos e de saúde na autopercepção da audição entre os idosos do projeto " Saúde, Bem-Estar e Envelhecimento" (Projeto SABE) no município de São Paulo. O estudo incluiu 2.143 indivíduos de 60 anos e mais. Um modelo de regressão logística ordinal, considerando o desenho da amostra, foi usado na análise multivariável. O aumento da idade; o sexo masculino; morar acompanhado; relatar tontura; memória regular/ ruim e saúde regular ou ruim aumentaram a chance de autopercepção ruim da audição. O conhecimento da autopercepção da audição e dos seus fatores relacionados é importante para avaliar a qualidade de vida dos idosos e a necessidade de reabilitação auditiva

Minimizing total tardiness in a stochastic single machine scheduling problem using approximate dynamic programming

This paper addresses the non-preemptive single machine scheduling problem to minimize total tardiness. We are interested in the online version of this problem, where orders arrive at the system at random times. Jobs have to be scheduled without knowledge of what jobs will come afterwards. The processing times and the due dates become known when the order is placed. The order release date occurs only at the beginning of periodic intervals. A customized approximate dynamic programming method is introduced for this problem. The authors also present numerical experiments that assess the reliability of the new approach and show that it performs better than a myopic policy.

ECONOMICAL OCCUPATION OF AGRICULTURAL AREA OF PIRACICABA (SAO PAULO STATE, BRAZIL), INCLUDING RISK THROUGH LINEAR PROGRAMMING

The economic occupation of an area of 500 ha for Piracicaba was studied with the irrigated cultures of maize, tomato, sugarcane and beans, having used models of deterministic linear programming and linear programming including risk for the Target-Motad model, where two situations had been analyzed. In the deterministic model the area was the restrictive factor and the water was not restrictive for none of the tested situations. For the first situation the gotten maximum income was of R$ 1,883,372.87 and for the second situation it was of R$ 1,821,772.40. In the model including risk a producer that accepts risk can in the first situation get the maximum income of R$ 1,883,372. 87 with a minimum risk of R$ 350 year(-1), and in the second situation R$ 1,821,772.40 with a minimum risk of R$ 40 year(-1). Already a producer averse to the risk can get in the first situation a maximum income of R$ 1,775,974.81 with null risk and for the second situation R$ 1.707.706, 26 with null risk, both without water restriction. These results stand out the importance of the inclusion of the risk in supplying alternative occupations to the producer, allowing to a producer taking of decision considered the risk aversion and the pretension of income.

Rough Notes on Programming AVR Microcontrollers in C

These notes follow on from the material that you studied in CSSE1000 Introduction to Computer Systems. There you studied details of logic gates, binary numbers and instruction set architectures using the Atmel AVR microcontroller family as an example. In your present course (METR2800 Team Project I), you need to get on to designing and building an application which will include such a microcontroller. These notes focus on programming an AVR microcontroller in C and provide a number of example programs to illustrate the use of some of the AVR peripheral devices.