Towards an Integrative Information Society: Studies on Individuality in Speech and Sign

The flow of information within modern information society has increased rapidly over the last decade. The major part of this information flow relies on the individual’s abilities to handle text or speech input. For the majority of us it presents no problems, but there are some individuals who would benefit from other means of conveying information, e.g. signed information flow. During the last decades the new results from various disciplines have all suggested towards the common background and processing for sign and speech and this was one of the key issues that I wanted to investigate further in this thesis. The basis of this thesis is firmly within speech research and that is why I wanted to design analogous test batteries for widely used speech perception tests for signers – to find out whether the results for signers would be the same as in speakers’ perception tests. One of the key findings within biology – and more precisely its effects on speech and communication research – is the mirror neuron system. That finding has enabled us to form new theories about evolution of communication, and it all seems to converge on the hypothesis that all communication has a common core within humans. In this thesis speech and sign are discussed as equal and analogical counterparts of communication and all research methods used in speech are modified for sign. Both speech and sign are thus investigated using similar test batteries. Furthermore, both production and perception of speech and sign are studied separately. An additional framework for studying production is given by gesture research using cry sounds. Results of cry sound research are then compared to results from children acquiring sign language. These results show that individuality manifests itself from very early on in human development. Articulation in adults, both in speech and sign, is studied from two perspectives: normal production and re-learning production when the apparatus has been changed. Normal production is studied both in speech and sign and the effects of changed articulation are studied with regards to speech. Both these studies are done by using carrier sentences. Furthermore, sign production is studied giving the informants possibility for spontaneous speech. The production data from the signing informants is also used as the basis for input in the sign synthesis stimuli used in sign perception test battery. Speech and sign perception were studied using the informants’ answers to questions using forced choice in identification and discrimination tasks. These answers were then compared across language modalities. Three different informant groups participated in the sign perception tests: native signers, sign language interpreters and Finnish adults with no knowledge of any signed language. This gave a chance to investigate which of the characteristics found in the results were due to the language per se and which were due to the changes in modality itself. As the analogous test batteries yielded similar results over different informant groups, some common threads of results could be observed. Starting from very early on in acquiring speech and sign the results were highly individual. However, the results were the same within one individual when the same test was repeated. This individuality of results represented along same patterns across different language modalities and - in some occasions - across language groups. As both modalities yield similar answers to analogous study questions, this has lead us to providing methods for basic input for sign language applications, i.e. signing avatars. This has also given us answers to questions on precision of the animation and intelligibility for the users – what are the parameters that govern intelligibility of synthesised speech or sign and how precise must the animation or synthetic speech be in order for it to be intelligible. The results also give additional support to the well-known fact that intelligibility in fact is not the same as naturalness. In some cases, as shown within the sign perception test battery design, naturalness decreases intelligibility. This also has to be taken into consideration when designing applications. All in all, results from each of the test batteries, be they for signers or speakers, yield strikingly similar patterns, which would indicate yet further support for the common core for all human communication. Thus, we can modify and deepen the phonetic framework models for human communication based on the knowledge obtained from the results of the test batteries within this thesis.

Making the impact on research and society : a case study: crowdsourcing solutions developed for the linguistic research and citizen science

Can crowdsourcing solutions serve many masters? Can they be beneficial for both, for the layman or native speakers of minority languages on the one hand and serious linguistic research on the other? How did an infrastructure that was designed to support linguistics turn out to be a solution for raising awareness of native languages? Since 2012 the National Library of Finland has been developing the Digitisation Project for Kindred Languages, in which the key objective is to support a culture of openness and interaction in linguistic research, but also to promote crowdsourcing as a tool for participation of the language community in research. In the course of the project, over 1,200 monographs and nearly 111,000 pages of newspapers in Finno-Ugric languages will be digitised and made available in the Fenno-Ugrica digital collection. This material was published in the Soviet Union in the 1920s and 1930s, and users have had only sporadic access to the material. The publication of open-access and searchable materials from this period is a goldmine for researchers. Historians, social scientists and laymen with an interest in specific local publications can now find text materials pertinent to their studies. The linguistically-oriented population can also find writings to delight them: (1) lexical items specific to a given publication, and (2) orthographically-documented specifics of phonetics. In addition to the open access collection, we developed an open source code OCR editor that enables the editing of machine-encoded text for the benefit of linguistic research. This tool was necessary since these rare and peripheral prints often include already archaic characters, which are neglected by modern OCR software developers but belong to the historical context of kindred languages, and are thus an essential part of the linguistic heritage. When modelling the OCR editor, it was essential to consider both the needs of researchers and the capabilities of lay citizens, and to have them participate in the planning and execution of the project from the very beginning. By implementing the feedback iteratively from both groups, it was possible to transform the requested changes as tools for research that not only supported the work of linguistics but also encouraged the citizen scientists to face the challenge and work with the crowdsourcing tools for the benefit of research. This presentation will not only deal with the technical aspects, developments and achievements of the infrastructure but will highlight the way in which user groups, researchers and lay citizens were engaged in a process as an active and communicative group of users and how their contributions were made to mutual benefit.

Theme of "swinging" in shamanism and folk culture

V. E. Sharapov & D. A. Nesanelis

A bridge between engineering and language learning : automation and German in an online course

Artikkeli luettavissa osassa: Part 2. - ISBN 9789522163172(PDF). - Liitteenä työpaperi

Making the impact on research and society : nichesourcing of Uralic language material for the benefit of linguistic research and native-speakers

Presentation at the 12th Bibliotheca Baltica Symposium at Södertörn University Library

Portfolio as an Indicator of Young Learners’ English Proficiency in Mainstream Language Instruction (EFL) and Bilingual Content Instruction (CLIL)

Tämä soveltavan kielitieteen ja kielitaidon arvioinnin toimintatutkimus tarkasteli kieliportfolion ominaisuuksia ja mahdollisuuksia nuorten oppijoiden englannin kielen arvioinnissa kahdessa eri oppimiskontekstissa: englanti oppiaineena (EFL) ja kaksikielinen sisällönopetus (CLIL). Tutkielman itsenäiset, kahteen eri englannin kielen rekisteriin (arkikieli ja akateeminen kieli) kohdistuneet portfoliokokeilut olivat erillisiä tapaustutkimuksia. Molemmat portfoliot perustuivat väljästi Eurooppalaiseen kielisalkkumalliin, ja ne olivat osa tutkielmantekijän luokkaopetusta ja -toimintaa. EFL -portfoliokokeilu 9-10-vuotiaille kolmasluokkalaisille toteutettiin marraskuun 2011 ja toukokuun 2012 välisenä aikana, kun CLIL -portfoliokokeilu n. 7-9-vuotiallle ensimmäisen ja toisen luokan oppilaille kesti kaksi lukuvuotta 2012–2014. Molemmissa kokeiluissa myös oppilaiden vanhemmat kuuluivat tutkimusjoukkoon, samoin CLIL -portfolion toteutuksessa avustaneet ja opettajanäkökulmaa edustaneet opettajaopiskelijat. Portfoliokokeilun aloitti myös kaksi muuta CLIL -opettajaa, mutta kumpikin kokeilu päättyi alkuvaiheeseensa. Tarkemman tarkastelun kohteina olivat tutkimuksen osallistujien kokemukset ja mielipiteet portfoliokokeiluista. Erityisesti tavoitteena oli selvittää, miten informatiivisena englannin kielitaidon indikaattorina kieliportfoliota pidettiin. Myös kehitysehdotuksia kerättiin. Trianguloitu aineisto koottiin sekä puolistrukturoiduin kyselyin että vapaaehtoisin teemahaastatteluin, jotka äänitettiin. EFL -aineisto koostui 18 oppilaskyselystä, 17 huoltajakyselystä ja 7 oppilashaastattelusta. CLIL -aineistoon sisältyi 19 oppilaskyselyä, 18 huoltajakyselyä, 7 oppilashaastattelua ja yksi opettajaopiskelijoiden (N=3) ryhmähaastattelu. Aineisto analysoitiin pääosin kvalitatiivisin menetelmin temaattisen sisältöanalyysin keinoin, mutta myös laskien frekvenssejä ja prosenttisosuuksia. Osallistujien mielipiteet ja kokemukset olivat hyvin samankaltaiset ja positiiviset kummassakin portfoliokokeilussa. Merkittävä enemmistö sekä oppilaista että huoltajista koki, että portfolion avulla on mahdollista osoittaa englannin kielitaitoa ja sen kehittymistä. Oppilaat kuvailivat portfoliotyötä hauskaksi ja kivaksi, ja heidän mielestään portfoliotehtävien pitäisi olla tarpeeksi haastavia, sisältää taiteellisia ja luovia elementtejä sekä kohdistua tuttuihin, mielenkiintoisiin aiheisiin. He totesivat, että portfolion avulla voi oppia lisää kieltä. Vanhempien mielestä portfolio kertoo koulun vieraiksi jääneistä oppisisällöistä, auttaa ymmärtämään lapsen ajatusmaailmaa ja motivaatiotasoa sekä paljastaa heidän kielitaidostaan uusia ulottuvuuksia. Opettajaopiskelijat havaitsivat, että portfolion avulla voi tutustua oppilaiden kieli- ja kulttuuritaustoihin sekä kartoittaa heidän kielellisiä tarpeitaan. Tämän tutkielman teoreettisen tarkastelun ja tulosten mukaan kieliportfolio tukee erinomaisesti uuden Perusopetuksen Opetussuunnitelman (NCC 2014) tavoitteita ja arvioinnin uudistuspyrkimyksiä sekä lainsäädännön arvioinnille asettamia edellytyksiä. Portfolio on erittäin suositeltava nuorten oppijoiden kielitaidon arviointimenetelmä perinteisten rinnalle.