Web-makrot ja käyttäjäautomaatio web-sovelluksessa: prototyypin suunnittelu ja toteutus

Tämän kandidaatintyön aiheena on web-sovelluksen käyttäjäautomaatio web-makrojen avulla. Työssä esitellään prototyyppitoteutus SaaS-sovellukseen kohdistetusta web-makrojärjestelmästä. Työn teoriaosa esittelee web-automaation ja web-ympäristön keskeisen teknologian. Ratkaisuosa pohjustaa ratkaisuun käytetyn lähestymistavan, esittelee työn prototyyppiratkaisun ja arvioi ratkaisun lopputulosta. Lopputuloksena havaitaan, että prototyyppiratkaisu onnistuu selvittämään makrotoiminnallisuuden toteuttamiseen liittyvät tekniset haasteet. Ratkaisun merkittävimmäksi tekniseksi haasteeksi arvioidaan selainyhteensopivuuden saavutus ja ylläpito. Ratkaisun hyödynnettävyys koko web-sovelluksen mittakaavassa todetaan kuitenkin heikoksi, koska sovelluksen liiketoimintalogiikka rajoittaa makroparadigman käyttöä.

Search Interfaces on the Web: Querying and Characterizing

Current-day web search engines (e.g., Google) do not crawl and index a significant portion of theWeb and, hence, web users relying on search engines only are unable to discover and access a large amount of information from the non-indexable part of the Web. Specifically, dynamic pages generated based on parameters provided by a user via web search forms (or search interfaces) are not indexed by search engines and cannot be found in searchers’ results. Such search interfaces provide web users with an online access to myriads of databases on the Web. In order to obtain some information from a web database of interest, a user issues his/her query by specifying query terms in a search form and receives the query results, a set of dynamic pages that embed required information from a database. At the same time, issuing a query via an arbitrary search interface is an extremely complex task for any kind of automatic agents including web crawlers, which, at least up to the present day, do not even attempt to pass through web forms on a large scale. In this thesis, our primary and key object of study is a huge portion of the Web (hereafter referred as the deep Web) hidden behind web search interfaces. We concentrate on three classes of problems around the deep Web: characterization of deep Web, finding and classifying deep web resources, and querying web databases. Characterizing deep Web: Though the term deep Web was coined in 2000, which is sufficiently long ago for any web-related concept/technology, we still do not know many important characteristics of the deep Web. Another matter of concern is that surveys of the deep Web existing so far are predominantly based on study of deep web sites in English. One can then expect that findings from these surveys may be biased, especially owing to a steady increase in non-English web content. In this way, surveying of national segments of the deep Web is of interest not only to national communities but to the whole web community as well. In this thesis, we propose two new methods for estimating the main parameters of deep Web. We use the suggested methods to estimate the scale of one specific national segment of the Web and report our findings. We also build and make publicly available a dataset describing more than 200 web databases from the national segment of the Web. Finding deep web resources: The deep Web has been growing at a very fast pace. It has been estimated that there are hundred thousands of deep web sites. Due to the huge volume of information in the deep Web, there has been a significant interest to approaches that allow users and computer applications to leverage this information. Most approaches assumed that search interfaces to web databases of interest are already discovered and known to query systems. However, such assumptions do not hold true mostly because of the large scale of the deep Web – indeed, for any given domain of interest there are too many web databases with relevant content. Thus, the ability to locate search interfaces to web databases becomes a key requirement for any application accessing the deep Web. In this thesis, we describe the architecture of the I-Crawler, a system for finding and classifying search interfaces. Specifically, the I-Crawler is intentionally designed to be used in deepWeb characterization studies and for constructing directories of deep web resources. Unlike almost all other approaches to the deep Web existing so far, the I-Crawler is able to recognize and analyze JavaScript-rich and non-HTML searchable forms. Querying web databases: Retrieving information by filling out web search forms is a typical task for a web user. This is all the more so as interfaces of conventional search engines are also web forms. At present, a user needs to manually provide input values to search interfaces and then extract required data from the pages with results. The manual filling out forms is not feasible and cumbersome in cases of complex queries but such kind of queries are essential for many web searches especially in the area of e-commerce. In this way, the automation of querying and retrieving data behind search interfaces is desirable and essential for such tasks as building domain-independent deep web crawlers and automated web agents, searching for domain-specific information (vertical search engines), and for extraction and integration of information from various deep web resources. We present a data model for representing search interfaces and discuss techniques for extracting field labels, client-side scripts and structured data from HTML pages. We also describe a representation of result pages and discuss how to extract and store results of form queries. Besides, we present a user-friendly and expressive form query language that allows one to retrieve information behind search interfaces and extract useful data from the result pages based on specified conditions. We implement a prototype system for querying web databases and describe its architecture and components design.

HTML-teknologian hyödyntäminen Web-sovelluksen prototypoinnissa

Käyttöliittymä on rajapinta käyttäjän ja järjestelmän tarjoamien toimintojen välillä ja sen toimivuus vaikuttaa toimintojen suorittamiseen joko positiivisesti tai negatiivisesti. Täten sovelluksen suunnitteluvaiheessa on hyvä arvioida käyttöliittymän ja sen toimintojen laatua ja kokeilla ideoiden toimivuutta rakentamalla asiasta prototyyppejä. Prototypoinnilla voidaan tunnistaa ja korjata mahdolliset ongelmat jo suunnittelupöydällä. Tämä diplomityö käsittelee Web-sovelluksen kehityksen aikana toteutettua käyttöliittymän ja sen toimintojen prototypointia. Käyttöliittymien mallintamista voidaan toteuttaa erilaisilla menetelmillä, joita työssä käydään läpi teknologisista näkökulmista eli miten prototypointimenetelmiä voidaan soveltaa projektin eri vaiheissa. Prototypoinnin apuna käytettäviin työkaluihin luodaan lyhyt katsaus esitellen yleisellä tasolla muutamia eri sovelluskategorian ohjelmistoja ja lisäksi käsitellään suunnittelumallien hyödyntämistä. Työ osoittaa, että yleisiä prototypointimenetelmiä ja -periaatteita voidaan soveltaa Web-sovellusten prototypoinnissa. Prototypointi on hyödyllistä aloittaa luonnostelemalla ja jatkaa aikaisessa vaiheessa HTML-malleihin, joilla päästään lähelle toteutuksen teknologioita ja mallintamaan sovelluksen luonnetta, ilmettä, tuntumaa ja vuorovaikutusta. HTML-prototyypeistä voidaan jalostaa sekoitetun tarkkuuden malleja ja ne toimivat toteutuksen perustana. Jatkokehityksessä ideoita voidaan esittää useilla eri tarkkuuden tekniikoilla.

Web Services In Symbian OS

This thesis presents different aspects of Web Services usage in Symbian OS that is an operating system for handheld devices. The practical part of the work was to develop Symbian OS client application for Web Services. It produced four reusable software components. XML enables platform and programming language independent services. Web Services use XML to create standardized message oriented services that are accessed through HTTP. Web Services are moving towards dynamic B2B interaction. Web Services increases the amount of transferred data, which is not desirable in mobile networks where transfer speed is slower than in the traditional networks. However the modern mobile networks are able to transfer the extra payload with reasonable time. XML processing is not a big problem. Web Services can be accessed from the modern mobile devices and they can cut down the development costs.

Sulautetun järjestelmän Web-käyttö

Diplomityössä on tutkittu sulautetun järjestelmän liittämistä Ethernet-verkkoon sekä TCP/IP-protokollapinoon kuuluvien tavallisimpien protokollien toimintaa. Työn tuloksien perusteella on suunniteltu harjoitustyö, jota voidaan käyttää sähkötekniikan osaston opetuksessa. Työssä hankittiin Atmelin sulautettu Web-serverikortti (EWS) ja STK500-kortti serverin ohjelmointiin. Serverin mukana tuli Internet-yhteyden mahdollistava TCP/IP-pinon lähdekoodi. Työssä selvitettiin TCP/IP:hen kuuluvien protokollien toimintaa teoriassa ja käytännön toteutusta EWS:n avulla. Lähdekoodiin lisättiin ominaisuudet, joiden avulla laite hakee kellonaikatiedot aikapalvelimelta time-protokollaa käyttäen ja lähettää sähköpostia määriteltyyn osoitteeseen sähköpostipalvelimen kautta. Laitetta käytettiin sekä palvelimena että asiakkaana. Työssä perehdyttiin sulautettujen järjestelmien yleisiin ominaisuuksiin ja erilaisiin verkonhallinnan apuohjelmiin. Työssä tutkittiin palvelin- ja asiakaskoneen välillä Ethernetissä kulkevaa verkkoliikennettä. Työssä minimoitiin protokollatiedostojen viemä tila prosessorin muistista ja tutkittiin Internet-yhteyden kuluttamien kellojaksojen määrää. Työssä selvitettiin tietoturva-kysymysten merkitystä ja toteutusta sulautetuissa järjestelmissä.

Ajax: Web-sovellusten uudet mahdollisuudet

Tässä työssä selvitettiin Ajax-tekniikan tilannetta web-sovellusten kehityksessä. Sitä varten kehitettiin demosovellus, jonka avulla tekniikoiden käyttökelpoisuutta voitiin arvioida. Samalla työssä on esitelty eri tekniikoita, jotka liittyvät kiinteästi Ajax-sovellusten toteuttamiseen. Demosovellus tehtiin vapaalla LAMP (Linux, Apache, MySQL and PHP) -alustalla. Työssä on arvioitu Ajax-tekniikan käyttökelpoisuutta ja ongelmia nykyisen webin, web-kehittäjien, käytössä olevien selainten ja käyttäjien kannalta. Lopussa on myös pohdittu hieman webin tulevaisuutta ja Ajaxin osaa siinä.

Web Application Performance Testing

Web application performance testing is an emerging and important field of software engineering. As web applications become more commonplace and complex, the need for performance testing will only increase. This paper discusses common concepts, practices and tools that lie at the heart of web application performance testing. A pragmatic, hands-on approach is assumed where applicable; real-life examples of test tooling, execution and analysis are presented right next to the underpinning theory. At the client-side, web application performance is primarily driven by the amount of data transmitted over the wire. At the server-side, selection of programming language and platform, implementation complexity and configuration are the primary contributors to web application performance. Web application performance testing is an activity that requires delicate coordination between project stakeholders, developers, system administrators and testers in order to produce reliable and useful results. Proper test definition, execution, reporting and repeatable test results are of utmost importance. Open-source performance analysis tools such as Apache JMeter, Firebug and YSlow can be used to realise effective web application performance tests. A sample case study using these tools is presented in this paper. The sample application was found to perform poorly even under the moderate load incurred by the sample tests.

The case of web based information environment in web based weight management services

Abstrakti

Design and evaluation of web interface signs to improve web usability : a semiotic framework

Technological innovations, the development of the internet, and globalization have increased the number and complexity of web applications. As a result, keeping web user interfaces understandable and usable (in terms of ease-of-use, effectiveness, and satisfaction) is a challenge. As part of this, designing userintuitive interface signs (i.e., the small elements of web user interface, e.g., navigational link, command buttons, icons, small images, thumbnails, etc.) is an issue for designers. Interface signs are key elements of web user interfaces because ‘interface signs’ act as a communication artefact to convey web content and system functionality, and because users interact with systems by means of interface signs. In the light of the above, applying semiotic (i.e., the study of signs) concepts on web interface signs will contribute to discover new and important perspectives on web user interface design and evaluation. The thesis mainly focuses on web interface signs and uses the theory of semiotic as a background theory. The underlying aim of this thesis is to provide valuable insights to design and evaluate web user interfaces from a semiotic perspective in order to improve overall web usability. The fundamental research question is formulated as What do practitioners and researchers need to be aware of from a semiotic perspective when designing or evaluating web user interfaces to improve web usability? From a methodological perspective, the thesis follows a design science research (DSR) approach. A systematic literature review and six empirical studies are carried out in this thesis. The empirical studies are carried out with a total of 74 participants in Finland. The steps of a design science research process are followed while the studies were designed and conducted; that includes (a) problem identification and motivation, (b) definition of objectives of a solution, (c) design and development, (d) demonstration, (e) evaluation, and (f) communication. The data is collected using observations in a usability testing lab, by analytical (expert) inspection, with questionnaires, and in structured and semi-structured interviews. User behaviour analysis, qualitative analysis and statistics are used to analyze the study data. The results are summarized as follows and have lead to the following contributions. Firstly, the results present the current status of semiotic research in UI design and evaluation and highlight the importance of considering semiotic concepts in UI design and evaluation. Secondly, the thesis explores interface sign ontologies (i.e., sets of concepts and skills that a user should know to interpret the meaning of interface signs) by providing a set of ontologies used to interpret the meaning of interface signs, and by providing a set of features related to ontology mapping in interpreting the meaning of interface signs. Thirdly, the thesis explores the value of integrating semiotic concepts in usability testing. Fourthly, the thesis proposes a semiotic framework (Semiotic Interface sign Design and Evaluation – SIDE) for interface sign design and evaluation in order to make them intuitive for end users and to improve web usability. The SIDE framework includes a set of determinants and attributes of user-intuitive interface signs, and a set of semiotic heuristics to design and evaluate interface signs. Finally, the thesis assesses (a) the quality of the SIDE framework in terms of performance metrics (e.g., thoroughness, validity, effectiveness, reliability, etc.) and (b) the contributions of the SIDE framework from the evaluators’ perspective.

Adopting the Vaadin Java framework to rebuild VTT’s Web Interfaces and Database Applications

With the growth in new technologies, using online tools have become an everyday lifestyle. It has a greater impact on researchers as the data obtained from various experiments needs to be analyzed and knowledge of programming has become mandatory even for pure biologists. Hence, VTT came up with a new tool, R Executables (REX) which is a web application designed to provide a graphical interface for biological data functions like Image analysis, Gene expression data analysis, plotting, disease and control studies etc., which employs R functions to provide results. REX provides a user interactive application for the biologists to directly enter the values and run the required analysis with a single click. The program processes the given data in the background and prints results rapidly. Due to growth of data and load on server, the interface has gained problems concerning time consumption, poor GUI, data storage issues, security, minimal user interactive experience and crashes with large amount of data. This thesis handles the methods by which these problems were resolved and made REX a better application for the future. The old REX was developed using Python Django and now, a new programming language, Vaadin has been implemented. Vaadin is a Java framework for developing web applications and the programming language is extremely similar to Java with new rich components. Vaadin provides better security, better speed, good and interactive interface. In this thesis, subset functionalities of REX was selected which includes IST bulk plotting and image segmentation and implemented those using Vaadin. A code of 662 lines was programmed by me which included Vaadin as the front-end handler while R language was used for back-end data retrieval, computing and plotting. The application is optimized to allow further functionalities to be migrated with ease from old REX. Future development is focused on including Hight throughput screening functions along with gene expression database handling