Design And Development Of A Named Entity Based Question Answering System For Malayalam Language

This is a Named Entity Based Question Answering System for Malayalam Language. Although a vast amount of information is available today in digital form, no effective information access mechanism exists to provide humans with convenient information access. Information Retrieval and Question Answering systems are the two mechanisms available now for information access. Information systems typically return a long list of documents in response to a user’s query which are to be skimmed by the user to determine whether they contain an answer. But a Question Answering System allows the user to state his/her information need as a natural language question and receives most appropriate answer in a word or a sentence or a paragraph. This system is based on Named Entity Tagging and Question Classification. Document tagging extracts useful information from the documents which will be used in finding the answer to the question. Question Classification extracts useful information from the question to determine the type of the question and the way in which the question is to be answered. Various Machine Learning methods are used to tag the documents. Rule-Based Approach is used for Question Classification. Malayalam belongs to the Dravidian family of languages and is one of the four major languages of this family. It is one of the 22 Scheduled Languages of India with official language status in the state of Kerala. It is spoken by 40 million people. Malayalam is a morphologically rich agglutinative language and relatively of free word order. Also Malayalam has a productive morphology that allows the creation of complex words which are often highly ambiguous. Document tagging tools such as Parts-of-Speech Tagger, Phrase Chunker, Named Entity Tagger, and Compound Word Splitter are developed as a part of this research work. No such tools were available for Malayalam language. Finite State Transducer, High Order Conditional Random Field, Artificial Immunity System Principles, and Support Vector Machines are the techniques used for the design of these document preprocessing tools. This research work describes how the Named Entity is used to represent the documents. Single sentence questions are used to test the system. Overall Precision and Recall obtained are 88.5% and 85.9% respectively. This work can be extended in several directions. The coverage of non-factoid questions can be increased and also it can be extended to include open domain applications. Reference Resolution and Word Sense Disambiguation techniques are suggested as the future enhancements

Studies on a syntax based approach for translation between structurally different languages and the development of a prototype for Malayalam to English translation

This thesis summarizes the results on the studies on a syntax based approach for translation between Malayalam, one of Dravidian languages and English and also on the development of the major modules in building a prototype machine translation system from Malayalam to English. The development of the system is a pioneering effort in Malayalam language unattempted by previous researchers. The computational models chosen for the system is first of its kind for Malayalam language. An in depth study has been carried out in the design of the computational models and data structures needed for different modules: morphological analyzer , a parser, a syntactic structure transfer module and target language sentence generator required for the prototype system. The generation of list of part of speech tags, chunk tags and the hierarchical dependencies among the chunks required for the translation process also has been done. In the development process, the major goals are: (a) accuracy of translation (b) speed and (c) space. Accuracy-wise, smart tools for handling transfer grammar and translation standards including equivalent words, expressions, phrases and styles in the target language are to be developed. The grammar should be optimized with a view to obtaining a single correct parse and hence a single translated output. Speed-wise, innovative use of corpus analysis, efficient parsing algorithm, design of efficient Data Structure and run-time frequency-based rearrangement of the grammar which substantially reduces the parsing and generation time are required. The space requirement also has to be minimised

Syntactic Based Machine Translation from English to Malayalam

Due to the emergence of multiple language support on the Internet, machine translation (MT) technologies are indispensable to the communication between speakers using different languages. Recent research works have started to explore tree-based machine translation systems with syntactical and morphological information. This work aims the development of Syntactic Based Machine Translation from English to Malayalam by adding different case information during translation. The system identifies general rules for various sentence patterns in English. These rules are generated using the Parts Of Speech (POS) tag information of the texts. Word Reordering based on the Syntax Tree is used to improve the translation quality of the system. The system used Bilingual English –Malayalam dictionary for translation.

A Framework of Statistical Machine Translator from English to Malayalam

In this paper we describe the methodology and the structural design of a system that translates English into Malayalam using statistical models. A monolingual Malayalam corpus and a bilingual English/Malayalam corpus are the main resource in building this Statistical Machine Translator. Training strategy adopted has been enhanced by PoS tagging which helps to get rid of the insignificant alignments. Moreover, incorporating units like suffix separator and the stop word eliminator has proven to be effective in bringing about better training results. In the decoder, order conversion rules are applied to reduce the structural difference between the language pair. The quality of statistical outcome of the decoder is further improved by applying mending rules. Experiments conducted on a sample corpus have generated reasonably good Malayalam translations and the results are verified with F measure, BLEU and WER evaluation metrics

Handling OOV Words in Phrase-Based Statistical Machine Translation for Malayalam

Statistical Machine Translation (SMT) is one of the potential applications in the field of Natural Language Processing. The translation process in SMT is carried out by acquiring translation rules automatically from the parallel corpora. However, for many language pairs (e.g. Malayalam- English), they are available only in very limited quantities. Therefore, for these language pairs a huge portion of phrases encountered at run-time will be unknown. This paper focuses on methods for handling such out-of-vocabulary (OOV) words in Malayalam that cannot be translated to English using conventional phrase-based statistical machine translation systems. The OOV words in the source sentence are pre-processed to obtain the root word and its suffix. Different inflected forms of the OOV root are generated and a match is looked up for the word variants in the phrase translation table of the translation model. A Vocabulary filter is used to choose the best among the translations of these word variants by finding the unigram count. A match for the OOV suffix is also looked up in the phrase entries and the target translations are filtered out. Structuring of the filtered phrases is done and SMT translation model is extended by adding OOV with its new phrase translations. By the results of the manual evaluation done it is observed that amount of OOV words in the input has been reduced considerably

Extension schemes for the Alignment Model of English-Malayalam Statistical Machine Translator

In Statistical Machine Translation from English to Malayalam, an unseen English sentence is translated into its equivalent Malayalam sentence using statistical models. A parallel corpus of English-Malayalam is used in the training phase. Word to word alignments has to be set among the sentence pairs of the source and target language before subjecting them for training. This paper deals with certain techniques which can be adopted for improving the alignment model of SMT. Methods to incorporate the parts of speech information into the bilingual corpus has resulted in eliminating many of the insignificant alignments. Also identifying the name entities and cognates present in the sentence pairs has proved to be advantageous while setting up the alignments. Presence of Malayalam words with predictable translations has also contributed in reducing the insignificant alignments. Moreover, reduction of the unwanted alignments has brought in better training results. Experiments conducted on a sample corpus have generated reasonably good Malayalam translations and the results are verified with F measure, BLEU and WER evaluation metrics.

ICT Tools for Teaching & Learning

Page 1. ICT Tools for Teaching & Learning G Santhosh Kumar Cochin University Page 2. Agenda • What is ICT? • Why integrate ICT in Education? • What are the Challenges? • What are the good resources? Page 3. Questions # “Despite the increasing use of ICT, the need for teachers is as great as ever” # “Placing ICT in schools will automatically improve the quality of education that children receive” #”The Internet is unsafe for children to use because there is so much dangerous material available on it” Page 4. What is ICT? • ICT is short for ..

Techniques to Improve the word alignments in Statistical Machine Translation from English to Malayalam

In Statistical Machine Translation from English to Malayalam, an unseen English sentence is translated into its equivalent Malayalam translation using statistical models like translation model, language model and a decoder. A parallel corpus of English-Malayalam is used in the training phase. Word to word alignments has to be set up among the sentence pairs of the source and target language before subjecting them for training. This paper is deals with the techniques which can be adopted for improving the alignment model of SMT. Incorporating the parts of speech information into the bilingual corpus has eliminated many of the insignificant alignments. Also identifying the name entities and cognates present in the sentence pairs has proved to be advantageous while setting up the alignments. Moreover, reduction of the unwanted alignments has brought in better training results. Experiments conducted on a sample corpus have generated reasonably good Malayalam translations and the results are verified with F measure, BLEU and WER evaluation metrics

Unconstrained Handwritten Malayalam Character Recognition using Wavelet Transform and Support vector Machine Classifier

This paper presents the application of wavelet processing in the domain of handwritten character recognition. To attain high recognition rate, robust feature extractors and powerful classifiers that are invariant to degree of variability of human writing are needed. The proposed scheme consists of two stages: a feature extraction stage, which is based on Haar wavelet transform and a classification stage that uses support vector machine classifier. Experimental results show that the proposed method is effective

Using Neural Network Classifier Support Vector Machine Regression for the prediction of Melting Point of Drug – like compounds

In our study we use a kernel based classification technique, Support Vector Machine Regression for predicting the Melting Point of Drug – like compounds in terms of Topological Descriptors, Topological Charge Indices, Connectivity Indices and 2D Auto Correlations. The Machine Learning model was designed, trained and tested using a dataset of 100 compounds and it was found that an SVMReg model with RBF Kernel could predict the Melting Point with a mean absolute error 15.5854 and Root Mean Squared Error 19.7576

Exploration tools in formal concept analysis

The development of conceptual knowledge systems specifically requests knowledge acquisition tools within the framework of formal concept analysis. In this paper, the existing tools are presented, and furhter developments are discussed.

Fliesslochformen und Gewindefurchen in dünne Bleche und Rohre aus Kupfer und Kupferlegierungen

This thesis in Thermal Flow Drilling and Flowtap in thin metal sheet and pipes of copper and copper alloys had as objectives to know the comportment of copper and copper alloys sheet metal during the Thermal Flow Drill processes with normal tools, to know the best Speed and Feed machine data for the best bushing quality, to known the best Speed for Form Tapping processes and to know the best bush long in pure copper pipes for water solar interchange equipment. Thermal Flow Drilling (TFD) and Form Tapping (FT) is one of the research lines of the Institute of Production and Logistics (IPL) at University of Kassel. At December 1995, a work meeting of IPL, Santa Catarina University, Brazil, Buenos Aires University, Argentine, Tarapacá University (UTA), Chile members and the CEO of Flowdrill B.V. was held in Brazil. The group decided that the Manufacturing Laboratory (ML) of UTA would work with pure copper and brass alloys sheet metal and pure copper pipes in order to develop a water interchange solar heater. The Flowdrill BV Company sent tools to Tarapacá University in 1996. In 1999 IPL and the ML carried out an ALECHILE research project promoted by the DAAD and CONICyT in copper sheet metal and copper pipes and sheet metal a-brass alloys. The normal tools are lobed, conical tungsten carbide tool. When rotated at high speed and pressed with high axial force into sheet metal or thin walled tube generated heat softens the metal and allows the drill to feed forward produce a hole and simultaneously form a bushing from the displacement material. In the market exist many features but in this thesis is used short and longs normal tools of TFD. For reach the objectives it was takes as references four qualities of the frayed end bushing, where the best one is the quality class I. It was used pure copper and a-brass alloys sheet metals, with different thickness. It was used different TFD drills diameter for four thread type, from M-5 to M10. Similar to the Aluminium sheet metals studies it was used the predrilling processes with HSS drills around 30% of the TFD diameter (1,5 – 3,0 mm D). In the next step is used only 2,0 mm thick metal sheet, and 9,2 mm TFD diameter for M-10 thread. For the case of pure commercial copper pipes is used for ¾” inch diameter and 12, 8 mm (3/8”) TFD drill for holes for 3/8” pipes and different normal HSS drills for predrilling processes. The chemical sheet metal characteristics were takes as reference for the material behaviour. The Chilean pure copper have 99,35% of Cu and 0,163% of Zinc and the Chilean a-brass alloys have 75,6% of Cu and 24,0% of Zinc. It is used two German a-brass alloys; Nº1 have 61,6% of Cu, 36,03 % of Zinc and 2,2% of Pb and the German a-brass alloys Nº2 have 63,1% of Cu, 36,7% of Zinc and 0% of Pb. The equipments used were a HAAS CNC milling machine centre, a Kistler dynamometer, PC Pentium II, Acquisition card, TESTPOINT and XAct software, 3D measurement machine, micro hardness, universal test machine, and metallographic microscope. During the test is obtained the feed force and momentum curves that shows the material behaviour with TFD processes. In general it is take three phases. It was possible obtain the best machining data for the different sheet of copper and a-brass alloys thick of Chilean materials and bush quality class I. In the case of a-brass alloys, the chemical components and the TFD processes temperature have big influence. The temperature reach to 400º Celsius during the TFD processes and the a-brass alloys have some percents of Zinc the bush quality is class I. But when the a-brass alloys have some percents of Lead who have 200º C melting point is not possible to obtain a bush, because the Lead gasify and the metallographic net broke. During the TFD processes the recrystallization structures occur around the Copper and a-brass alloy bush, who gives more hardness in these zones. When the threads were produce with Form Tapping processes with Flowtap tools, this hardness amount gives a high limit load of the thread when hey are tested in a special support that was developed for it. For eliminated the predrilling processes with normal HSS drills it was developed a compound tool. With this new tool it was possible obtain the best machining data for quality class I bush. For the copper pipes it is made bush without predrilling and the quality class IV was obtained. When it is was used predrilling processes, quality classes I bush were obtained. Then with different HSS drill diameter were obtained different long bush, where were soldering with four types soldering materials between pipes with 3/8” in a big one as ¾”. Those soldering unions were tested by traction test and all the 3/8” pipes broken, and the soldering zone doesn’t have any problem. Finally were developed different solar water interchange heaters and tested. As conclusions, the present Thesis shows that the Thermal Flow Drilling in thinner metal sheets of cooper and cooper alloys needs a predrilling process for frayed end quality class I bushings, similar to thinner sheets of aluminium bushes. The compound tool developed could obtain quality class I bushings and excludes predrilling processes. The bush recrystalization, product of the friction between the tool and the material, the hardness grows and it is advantageous for the Form Tapping. The methodology developed for commercial copper pipes permits to built water solar interchange heaters.

Adaptive Real-time Anomaly-based Intrusion Detection using Data Mining and Machine Learning Techniques

Die zunehmende Vernetzung der Informations- und Kommunikationssysteme führt zu einer weiteren Erhöhung der Komplexität und damit auch zu einer weiteren Zunahme von Sicherheitslücken. Klassische Schutzmechanismen wie Firewall-Systeme und Anti-Malware-Lösungen bieten schon lange keinen Schutz mehr vor Eindringversuchen in IT-Infrastrukturen. Als ein sehr wirkungsvolles Instrument zum Schutz gegenüber Cyber-Attacken haben sich hierbei die Intrusion Detection Systeme (IDS) etabliert. Solche Systeme sammeln und analysieren Informationen von Netzwerkkomponenten und Rechnern, um ungewöhnliches Verhalten und Sicherheitsverletzungen automatisiert festzustellen. Während signatur-basierte Ansätze nur bereits bekannte Angriffsmuster detektieren können, sind anomalie-basierte IDS auch in der Lage, neue bisher unbekannte Angriffe (Zero-Day-Attacks) frühzeitig zu erkennen. Das Kernproblem von Intrusion Detection Systeme besteht jedoch in der optimalen Verarbeitung der gewaltigen Netzdaten und der Entwicklung eines in Echtzeit arbeitenden adaptiven Erkennungsmodells. Um diese Herausforderungen lösen zu können, stellt diese Dissertation ein Framework bereit, das aus zwei Hauptteilen besteht. Der erste Teil, OptiFilter genannt, verwendet ein dynamisches "Queuing Concept", um die zahlreich anfallenden Netzdaten weiter zu verarbeiten, baut fortlaufend Netzverbindungen auf, und exportiert strukturierte Input-Daten für das IDS. Den zweiten Teil stellt ein adaptiver Klassifikator dar, der ein Klassifikator-Modell basierend auf "Enhanced Growing Hierarchical Self Organizing Map" (EGHSOM), ein Modell für Netzwerk Normalzustand (NNB) und ein "Update Model" umfasst. In dem OptiFilter werden Tcpdump und SNMP traps benutzt, um die Netzwerkpakete und Hostereignisse fortlaufend zu aggregieren. Diese aggregierten Netzwerkpackete und Hostereignisse werden weiter analysiert und in Verbindungsvektoren umgewandelt. Zur Verbesserung der Erkennungsrate des adaptiven Klassifikators wird das künstliche neuronale Netz GHSOM intensiv untersucht und wesentlich weiterentwickelt. In dieser Dissertation werden unterschiedliche Ansätze vorgeschlagen und diskutiert. So wird eine classification-confidence margin threshold definiert, um die unbekannten bösartigen Verbindungen aufzudecken, die Stabilität der Wachstumstopologie durch neuartige Ansätze für die Initialisierung der Gewichtvektoren und durch die Stärkung der Winner Neuronen erhöht, und ein selbst-adaptives Verfahren eingeführt, um das Modell ständig aktualisieren zu können. Darüber hinaus besteht die Hauptaufgabe des NNB-Modells in der weiteren Untersuchung der erkannten unbekannten Verbindungen von der EGHSOM und der Überprüfung, ob sie normal sind. Jedoch, ändern sich die Netzverkehrsdaten wegen des Concept drif Phänomens ständig, was in Echtzeit zur Erzeugung nicht stationärer Netzdaten führt. Dieses Phänomen wird von dem Update-Modell besser kontrolliert. Das EGHSOM-Modell kann die neuen Anomalien effektiv erkennen und das NNB-Model passt die Änderungen in Netzdaten optimal an. Bei den experimentellen Untersuchungen hat das Framework erfolgversprechende Ergebnisse gezeigt. Im ersten Experiment wurde das Framework in Offline-Betriebsmodus evaluiert. Der OptiFilter wurde mit offline-, synthetischen- und realistischen Daten ausgewertet. Der adaptive Klassifikator wurde mit dem 10-Fold Cross Validation Verfahren evaluiert, um dessen Genauigkeit abzuschätzen. Im zweiten Experiment wurde das Framework auf einer 1 bis 10 GB Netzwerkstrecke installiert und im Online-Betriebsmodus in Echtzeit ausgewertet. Der OptiFilter hat erfolgreich die gewaltige Menge von Netzdaten in die strukturierten Verbindungsvektoren umgewandelt und der adaptive Klassifikator hat sie präzise klassifiziert. Die Vergleichsstudie zwischen dem entwickelten Framework und anderen bekannten IDS-Ansätzen zeigt, dass der vorgeschlagene IDSFramework alle anderen Ansätze übertrifft. Dies lässt sich auf folgende Kernpunkte zurückführen: Bearbeitung der gesammelten Netzdaten, Erreichung der besten Performanz (wie die Gesamtgenauigkeit), Detektieren unbekannter Verbindungen und Entwicklung des in Echtzeit arbeitenden Erkennungsmodells von Eindringversuchen.

UNITRAN: A Principle-Based Approach to Machine Translation

Machine translation has been a particularly difficult problem in the area of Natural Language Processing for over two decades. Early approaches to translation failed since interaction effects of complex phenomena in part made translation appear to be unmanageable. Later approaches to the problem have succeeded (although only bilingually), but are based on many language-specific rules of a context-free nature. This report presents an alternative approach to natural language translation that relies on principle-based descriptions of grammar rather than rule-oriented descriptions. The model that has been constructed is based on abstract principles as developed by Chomsky (1981) and several other researchers working within the "Government and Binding" (GB) framework. Thus, the grammar is viewed as a modular system of principles rather than a large set of ad hoc language-specific rules.