A Framework of Statistical Machine Translator from English to Malayalam

In this paper we describe the methodology and the structural design of a system that translates English into Malayalam using statistical models. A monolingual Malayalam corpus and a bilingual English/Malayalam corpus are the main resource in building this Statistical Machine Translator. Training strategy adopted has been enhanced by PoS tagging which helps to get rid of the insignificant alignments. Moreover, incorporating units like suffix separator and the stop word eliminator has proven to be effective in bringing about better training results. In the decoder, order conversion rules are applied to reduce the structural difference between the language pair. The quality of statistical outcome of the decoder is further improved by applying mending rules. Experiments conducted on a sample corpus have generated reasonably good Malayalam translations and the results are verified with F measure, BLEU and WER evaluation metrics

A framework for translating English text into Malayalam using statistical models

A methodology for translating text from English into the Dravidian language, Malayalam using statistical models is discussed in this paper. The translator utilizes a monolingual Malayalam corpus and a bilingual English/Malayalam corpus in the training phase and generates automatically the Malayalam translation of an unseen English sentence. Various techniques to improve the alignment model by incorporating the morphological inputs into the bilingual corpus are discussed. Removing the insignificant alignments from the sentence pairs by this approach has ensured better training results. Pre-processing techniques like suffix separation from the Malayalam corpus and stop word elimination from the bilingual corpus also proved to be effective in producing better alignments. Difficulties in translation process that arise due to the structural difference between the English Malayalam pair is resolved in the decoding phase by applying the order conversion rules. The handcrafted rules designed for the suffix separation process which can be used as a guideline in implementing suffix separation in Malayalam language are also presented in this paper. Experiments conducted on a sample corpus have generated reasonably good Malayalam translations and the results are verified with F measure, BLEU and WER evaluation metrics

Ontology based Framework for News Extraction in Visual Media

Anticipating the increase in video information in future, archiving of news is an important activity in the visual media industry. When the volume of archives increases, it will be difficult for journalists to find the appropriate content using current search tools. This paper provides the details of the study we conducted about the news extraction systems used in different news channels in Kerala. Semantic web technologies can be used effectively since news archiving share many of the characteristics and problems of WWW. Since visual news archives of different media resources follow different metadata standards, interoperability between the resources is also an issue. World Wide Web Consortium has proposed a draft for an ontology framework for media resource which addresses the intercompatiblity issues. In this paper, the w3c proposed framework and its drawbacks is also discussed

Developing a Theoretical Framework for a Study on the Impact of Advertising Credibility of Consumer Healthcare Products

The tough competition in the global and national markets and new trends in consumerism resulted in an increase in the volume of advertisements. Sometimes advertisers are successful in achieving their intended objectives with a particular advertisement and sometimes they are not .These factors contributed a lot towards the decision making problems of advertising agencies with regard to the selection of appropriate advertising strategies and tactics. The tough competition and large volume of advertising make the consumers confused and this even created doubts in the minds of consumers about the genuineness and reliability of manufacturers and products. These factors caused a query regarding the active role of credibility element in advertising. The proposed study examines the effects of advertising credibility in consumer health care non durable product advertising on communication effect, purchase behavior and ad skepticism. This paper examines the need for the study of advertising credibility and reviews the advertising- consumer behaviour- credibility – healthcare theories which form a basis for the study. It identifies the different components and dimensions of advertising credibility and the importance of communication effect, purchase behavior and ad skepticism. It also studies the relevance of credibility in the consumer healthcare products advertising and suggests a Theoretical Framework for the proposed study

Automatic Segmentation Framework for Primary Tumors From Brain MRIs Using Morphological Filtering Techniques

This paper describes a novel framework for automatic segmentation of primary tumors and its boundary from brain MRIs using morphological filtering techniques. This method uses T2 weighted and T1 FLAIR images. This approach is very simple, more accurate and less time consuming than existing methods. This method is tested by fifty patients of different tumor types, shapes, image intensities, sizes and produced better results. The results were validated with ground truth images by the radiologist. Segmentation of the tumor and boundary detection is important because it can be used for surgical planning, treatment planning, textural analysis, 3-Dimensional modeling and volumetric analysis

Framework for middleware executed on mobile devices

Die ubiquitäre Datenverarbeitung ist ein attraktives Forschungsgebiet des vergangenen und aktuellen Jahrzehnts. Es handelt von unaufdringlicher Unterstützung von Menschen in ihren alltäglichen Aufgaben durch Rechner. Diese Unterstützung wird durch die Allgegenwärtigkeit von Rechnern ermöglicht die sich spontan zu verteilten Kommunikationsnetzwerken zusammen finden, um Informationen auszutauschen und zu verarbeiten. Umgebende Intelligenz ist eine Anwendung der ubiquitären Datenverarbeitung und eine strategische Forschungsrichtung der Information Society Technology der Europäischen Union. Das Ziel der umbebenden Intelligenz ist komfortableres und sichereres Leben. Verteilte Kommunikationsnetzwerke für die ubiquitäre Datenverarbeitung charakterisieren sich durch Heterogenität der verwendeten Rechner. Diese reichen von Kleinstrechnern, eingebettet in Gegenstände des täglichen Gebrauchs, bis hin zu leistungsfähigen Großrechnern. Die Rechner verbinden sich spontan über kabellose Netzwerktechnologien wie wireless local area networks (WLAN), Bluetooth, oder UMTS. Die Heterogenität verkompliziert die Entwicklung und den Aufbau von verteilten Kommunikationsnetzwerken. Middleware ist eine Software Technologie um Komplexität durch Abstraktion zu einer homogenen Schicht zu reduzieren. Middleware bietet eine einheitliche Sicht auf die durch sie abstrahierten Ressourcen, Funktionalitäten, und Rechner. Verteilte Kommunikationsnetzwerke für die ubiquitäre Datenverarbeitung sind durch die spontane Verbindung von Rechnern gekennzeichnet. Klassische Middleware geht davon aus, dass Rechner dauerhaft miteinander in Kommunikationsbeziehungen stehen. Das Konzept der dienstorienterten Architektur ermöglicht die Entwicklung von Middleware die auch spontane Verbindungen zwischen Rechnern erlaubt. Die Funktionalität von Middleware ist dabei durch Dienste realisiert, die unabhängige Software-Einheiten darstellen. Das Wireless World Research Forum beschreibt Dienste die zukünftige Middleware beinhalten sollte. Diese Dienste werden von einer Ausführungsumgebung beherbergt. Jedoch gibt es noch keine Definitionen wie sich eine solche Ausführungsumgebung ausprägen und welchen Funktionsumfang sie haben muss. Diese Arbeit trägt zu Aspekten der Middleware-Entwicklung für verteilte Kommunikationsnetzwerke in der ubiquitären Datenverarbeitung bei. Der Schwerpunkt liegt auf Middleware und Grundlagentechnologien. Die Beiträge liegen als Konzepte und Ideen für die Entwicklung von Middleware vor. Sie decken die Bereiche Dienstfindung, Dienstaktualisierung, sowie Verträge zwischen Diensten ab. Sie sind in einem Rahmenwerk bereit gestellt, welches auf die Entwicklung von Middleware optimiert ist. Dieses Rahmenwerk, Framework for Applications in Mobile Environments (FAME²) genannt, beinhaltet Richtlinien, eine Definition einer Ausführungsumgebung, sowie Unterstützung für verschiedene Zugriffskontrollmechanismen um Middleware vor unerlaubter Benutzung zu schützen. Das Leistungsspektrum der Ausführungsumgebung von FAME² umfasst: • minimale Ressourcenbenutzung, um auch auf Rechnern mit wenigen Ressourcen, wie z.B. Mobiltelefone und Kleinstrechnern, nutzbar zu sein • Unterstützung für die Anpassung von Middleware durch Änderung der enthaltenen Dienste während die Middleware ausgeführt wird • eine offene Schnittstelle um praktisch jede existierende Lösung für das Finden von Diensten zu verwenden • und eine Möglichkeit der Aktualisierung von Diensten zu deren Laufzeit um damit Fehlerbereinigende, optimierende, und anpassende Wartungsarbeiten an Diensten durchführen zu können Eine begleitende Arbeit ist das Extensible Constraint Framework (ECF), welches Design by Contract (DbC) im Rahmen von FAME² nutzbar macht. DbC ist eine Technologie um Verträge zwischen Diensten zu formulieren und damit die Qualität von Software zu erhöhen. ECF erlaubt das aushandeln sowie die Optimierung von solchen Verträgen.

Design, implementation and application of a generic framework for integrated regional land-use modeling

Land use is a crucial link between human activities and the natural environment and one of the main driving forces of global environmental change. Large parts of the terrestrial land surface are used for agriculture, forestry, settlements and infrastructure. Given the importance of land use, it is essential to understand the multitude of influential factors and resulting land use patterns. An essential methodology to study and quantify such interactions is provided by the adoption of land-use models. By the application of land-use models, it is possible to analyze the complex structure of linkages and feedbacks and to also determine the relevance of driving forces. Modeling land use and land use changes has a long-term tradition. In particular on the regional scale, a variety of models for different regions and research questions has been created. Modeling capabilities grow with steady advances in computer technology, which on the one hand are driven by increasing computing power on the other hand by new methods in software development, e.g. object- and component-oriented architectures. In this thesis, SITE (Simulation of Terrestrial Environments), a novel framework for integrated regional sland-use modeling, will be introduced and discussed. Particular features of SITE are the notably extended capability to integrate models and the strict separation of application and implementation. These features enable efficient development, test and usage of integrated land-use models. On its system side, SITE provides generic data structures (grid, grid cells, attributes etc.) and takes over the responsibility for their administration. By means of a scripting language (Python) that has been extended by language features specific for land-use modeling, these data structures can be utilized and manipulated by modeling applications. The scripting language interpreter is embedded in SITE. The integration of sub models can be achieved via the scripting language or by usage of a generic interface provided by SITE. Furthermore, functionalities important for land-use modeling like model calibration, model tests and analysis support of simulation results have been integrated into the generic framework. During the implementation of SITE, specific emphasis was laid on expandability, maintainability and usability. Along with the modeling framework a land use model for the analysis of the stability of tropical rainforest margins was developed in the context of the collaborative research project STORMA (SFB 552). In a research area in Central Sulawesi, Indonesia, socio-environmental impacts of land-use changes were examined. SITE was used to simulate land-use dynamics in the historical period of 1981 to 2002. Analogous to that, a scenario that did not consider migration in the population dynamics, was analyzed. For the calculation of crop yields and trace gas emissions, the DAYCENT agro-ecosystem model was integrated. In this case study, it could be shown that land-use changes in the Indonesian research area could mainly be characterized by the expansion of agricultural areas at the expense of natural forest. For this reason, the situation had to be interpreted as unsustainable even though increased agricultural use implied economic improvements and higher farmers' incomes. Due to the importance of model calibration, it was explicitly addressed in the SITE architecture through the introduction of a specific component. The calibration functionality can be used by all SITE applications and enables largely automated model calibration. Calibration in SITE is understood as a process that finds an optimal or at least adequate solution for a set of arbitrarily selectable model parameters with respect to an objective function. In SITE, an objective function typically is a map comparison algorithm capable of comparing a simulation result to a reference map. Several map optimization and map comparison methodologies are available and can be combined. The STORMA land-use model was calibrated using a genetic algorithm for optimization and the figure of merit map comparison measure as objective function. The time period for the calibration ranged from 1981 to 2002. For this period, respective reference land-use maps were compiled. It could be shown, that an efficient automated model calibration with SITE is possible. Nevertheless, the selection of the calibration parameters required detailed knowledge about the underlying land-use model and cannot be automated. In another case study decreases in crop yields and resulting losses in income from coffee cultivation were analyzed and quantified under the assumption of four different deforestation scenarios. For this task, an empirical model, describing the dependence of bee pollination and resulting coffee fruit set from the distance to the closest natural forest, was integrated. Land-use simulations showed, that depending on the magnitude and location of ongoing forest conversion, pollination services are expected to decline continuously. This results in a reduction of coffee yields of up to 18% and a loss of net revenues per hectare of up to 14%. However, the study also showed that ecological and economic values can be preserved if patches of natural vegetation are conservated in the agricultural landscape. -----------------------------------------------------------------------

Empathie in der Psychotherapie aus neuer Perspektive

Die traditionellen Empathiekonzepte (z. B. Rogers, Kohut), die in der Psychotherapie bis heute maßgebend sind, werden einer gründlichen Überprüfung unterzogen. Dabei ergeben sich drei wesentliche Kritikpunkte: (1) Empathische Vorgänge in der Psychotherapie werden als einseitige Prozesse verstanden; der Therapeut fühlt sich in die Klientin ein, nicht auch umgekehrt. (2) Empathie wird in Cartesianischer Tradition schwerpunktmäßig als kognitive Leistung gesehen; ein körperloser Geist vergegenwärtigt sich die mentalen Inhalte eines anderen. (3) Das traditionelle Empathieverständnis ist individualistisch: Therapeutin und Klient halten sich demnach scheinbar im luftleeren Raum auf. Es sieht so aus, als existiere kein Kontext, der sie umgibt. So einseitig, wie ihre Beziehung gedacht wird, so abgetrennt, wie ihr Körper von ihrem Geist zu sein scheint, so unverbunden sind sie scheinbar mit dem Rest der Welt. Aus diesen drei Kritikpunkten folgt die Notwendigkeit, den Empathiebegriff der Psychotherapie zu erweitern, d. h. (a) Empathie als gegenseitigen Prozess der Beteiligten zu begreifen, (b) ihre tiefe Verwurzelung in der Leiblichkeit des Menschen zu berücksichtigen und (c) ihre Einbettung in die Dynamiken einer gemeinsamen Situation im Rahmen eines kulturellen Kontextes einzubeziehen. Mit Rückgriff auf neuere Untersuchungsergebnisse aus der Entwicklungspsychologie (z. B. Emde, Hobson, Meltzoff, Stern, Trevarthen), der Sozial- und Emotionspsychologie (z. B. Chartrand, Ekman, Goleman, Hatfield, Holodynski), der sozialen Neurowissenschaften (z. B. Damasio, Gallese, Iacoboni, LeDoux, Rizzolatti), aber auch mit Hilfe der Erkenntnisse aus der klassischen (Husserl, Merleau- Ponty, Edith Stein) und der Neuen Phänomenologie (Schmitz) sowie aus symbolischem Interaktionismus (Mead) und aus der kulturhistorischen Schule (Vygotskij) werden diese drei bislang wenig beleuchteten Dimensionen der Empathie betrachtet. ad a) Die Gegenseitigkeit empathischer Vorgänge in der Psychotherapie wird anhand des entwicklungspsychologischen Konzepts des social referencing erläutert und untersucht: Kleinkinder, die in eine unbekannte bzw. unsichere Situation geraten (z. B. im Experiment mit der "visuellen Klippe"), orientieren sich an den nonverbalen Signalen ihrer Bezugspersonen, um diese Situation zu bewältigen. Dabei erfasst die Mutter die Situation des Kindes, versucht ihm ihre Stellungnahme zu seiner Situation zu übermitteln, und das Kind begreift die Reaktion der Mutter als Stellungnahme zu seiner Situation. ad b) Die Körperlichkeit bzw. Leiblichkeit der Einfühlung manifestiert sich in vielfältigen Formen, wie sie von der Psychologie, der Phänomenologie und den Neurowissenschaften erforscht werden. Das kulturübergreifende Erkennen des Gesichtsausdrucks von Basisemotionen ist hier ebenso zu nennen wie die Verhaltensweisen des motor mimicry, bei dem Menschen Körperhaltungen und – bewegungen ihrer Bezugspersonen unwillkürlich imitieren; des Weiteren das unmittelbare Verstehen von Gesten sowie die Phänomene der „Einleibung“, bei denen die körperliche Situation des Anderen (z. B. eines stürzenden Radfahrers, den man beobachtet) am eigenen Leib mitgefühlt wird; und außerdem die Entdeckung der „Spiegelneurone“ und anderer neuronaler Strukturen, durch die Wahrgenommenes direkt in analoge motorische Aktivität übersetzt wird. ad c) Intersubjektivitätstheoretische Überlegungen, Konzepte wie „dyadisch erweiterter Bewusstseinszustand“ (Tronick) und „gemeinsame Situation“ (Gurwitsch, Schmitz) verweisen auf die Bedeutung überindividueller, ‚emergenter’ Dimensionen, die für die Verständigung zwischen Menschen wichtig sind. Sie folgen gestaltpsychologischen Prinzipien („Das Ganze ist mehr und anders als die Summe seiner Teile.“), die mit Hilfe von Gadamers Begriff des „Spiels“ analysiert werden. Am Ende der Arbeit stehen die Definition eines neuen Empathiebegriffs, wie er sich aus den vorangegangenen Überlegungen ergibt, sowie eine These über die psychotherapeutische Wirkweise menschlicher Einfühlung, die durch weitere Forschungen zu überprüfen wäre.

A Framework for TEI-Based Scholarly Text Editions

In this paper, we describe an interdisciplinary project in which visualization techniques were developed for and applied to scholarly work from literary studies. The aim was to bring Christof Schöch's electronic edition of Bérardier de Bataut's Essai sur le récit (1776) to the web. This edition is based on the Text Encoding Initiative's XML-based encoding scheme (TEI P5, subset TEI-Lite). This now de facto standard applies to machine-readable texts used chiefly in the humanities and social sciences. The intention of this edition is to make the edited text freely available on the web, to allow for alternative text views (here original and modern/corrected text), to ensure reader-friendly annotation and navigation, to permit on-line collaboration in encoding and annotation as well as user comments, all in an open source, generically usable, lightweight package. These aims were attained by relying on a GPL-based, public domain CMS (Drupal) and combining it with XSL-Stylesheets and Java Script.

Development of directly modulated high speed telecommunication lasers based on surface defined feedback gratings

High-speed semiconductor lasers are an integral part in the implemen- tation of high-bit-rate optical communications systems. They are com- pact, rugged, reliable, long-lived, and relatively inexpensive sources of coherent light. Due to the very low attenuation window that exists in the silica based optical fiber at 1.55 μm and the zero dispersion point at 1.3 μm, they have become the mainstay of optical fiber com- munication systems. For the fabrication of lasers with gratings such as, distributed bragg reflector or distributed feedback lasers, etching is the most critical step. Etching defines the lateral dimmensions of the structure which determines the performance of optoelectronic devices. In this thesis studies and experiments were carried out about the exist- ing etching processes for InP and a novel dry etching process was de- veloped. The newly developed process was based on Cl2/CH4/H2/Ar chemistry and resulted in very smooth surfaces and vertical side walls. With this process the grating definition was significantly improved as compared to other technological developments in the respective field. A surface defined grating definition approach is used in this thesis work which does not require any re-growth steps and makes the whole fabrication process simpler and cost effective. Moreover, this grating fabrication process is fully compatible with nano-imprint lithography and can be used for high throughput low-cost manufacturing. With usual etching techniques reported before it is not possible to etch very deep because of aspect ratio dependent etching phenomenon where with increasing etch depth the etch rate slows down resulting in non-vertical side walls and footing effects. Although with our de- veloped process quite vertical side walls were achieved but footing was still a problem. To overcome the challenges related to grating defini- tion and deep etching, a completely new three step gas chopping dry etching process was developed. This was the very first time that a time multiplexed etching process for an InP based material system was demonstrated. The developed gas chopping process showed extra ordinary results including high mask selectivity of 15, moderate etch- ing rate, very vertical side walls and a record high aspect ratio of 41. Both the developed etching processes are completely compatible with nano imprint lithography and can be used for low-cost high-throughput fabrication. A large number of broad area laser, ridge waveguide laser, distributed feedback laser, distributed bragg reflector laser and coupled cavity in- jection grating lasers were fabricated using the developed one step etch- ing process. Very extensive characterization was done to optimize all the important design and fabrication parameters. The devices devel- oped have shown excellent performance with a very high side mode suppression ratio of more than 52 dB, an output power of 17 mW per facet, high efficiency of 0.15 W/A, stable operation over temperature and injected currents and a threshold current as low as 30 mA for almost 1 mm long device. A record high modulation bandwidth of 15 GHz with electron-photon resonance and open eye diagrams for 10 Gbps data transmission were also shown.

Finding Spaces for Urban Food Production – Matching Spatial and Stakeholder Analysis with Urban Agriculture Approaches in the Urban Renewal Area of Dortmund-Hörde, Germany

Using the case of an economically declined neighbourhood in the post-industrial German Ruhr Area (sometimes characterized as Germany’s “Rust Belt”), we analyse, describe and conclude how urban agriculture can be used as a catalyst to stimulate and support urban renewal and regeneration, especially from a socio-cultural perspective. Using the methodological framework of participatory action research, and linking bottom-up and top-down planning approaches, a project path was developed to include the population affected and foster individual responsibility for their district, as well as to strengthen inhabitants and stakeholder groups in a permanent collective stewardship for the individual forms of urban agriculture developed and implemented. On a more abstract level, the research carried out can be characterized as a form of action research with an intended transgression of the boundaries between research, planning, design, and implementation. We conclude that by synchronously combining those four domains with intense feedback loops, synergies for the academic knowledge on the potential performance of urban agriculture in terms of sustainable development, as well as the benefits for the case-study area and the interests of individual urban gardeners can be achieved.

Efficient Characterisation and Optimal Control of Open Quantum Systems - Mathematical Foundations and Physical Applications

Since no physical system can ever be completely isolated from its environment, the study of open quantum systems is pivotal to reliably and accurately control complex quantum systems. In practice, reliability of the control field needs to be confirmed via certification of the target evolution while accuracy requires the derivation of high-fidelity control schemes in the presence of decoherence. In the first part of this thesis an algebraic framework is presented that allows to determine the minimal requirements on the unique characterisation of arbitrary unitary gates in open quantum systems, independent on the particular physical implementation of the employed quantum device. To this end, a set of theorems is devised that can be used to assess whether a given set of input states on a quantum channel is sufficient to judge whether a desired unitary gate is realised. This allows to determine the minimal input for such a task, which proves to be, quite remarkably, independent of system size. These results allow to elucidate the fundamental limits regarding certification and tomography of open quantum systems. The combination of these insights with state-of-the-art Monte Carlo process certification techniques permits a significant improvement of the scaling when certifying arbitrary unitary gates. This improvement is not only restricted to quantum information devices where the basic information carrier is the qubit but it also extends to systems where the fundamental informational entities can be of arbitary dimensionality, the so-called qudits. The second part of this thesis concerns the impact of these findings from the point of view of Optimal Control Theory (OCT). OCT for quantum systems utilises concepts from engineering such as feedback and optimisation to engineer constructive and destructive interferences in order to steer a physical process in a desired direction. It turns out that the aforementioned mathematical findings allow to deduce novel optimisation functionals that significantly reduce not only the required memory for numerical control algorithms but also the total CPU time required to obtain a certain fidelity for the optimised process. The thesis concludes by discussing two problems of fundamental interest in quantum information processing from the point of view of optimal control - the preparation of pure states and the implementation of unitary gates in open quantum systems. For both cases specific physical examples are considered: for the former the vibrational cooling of molecules via optical pumping and for the latter a superconducting phase qudit implementation. In particular, it is illustrated how features of the environment can be exploited to reach the desired targets.

Design and Control of an Anthropomorphic Robotic Finger with Multi-point Tactile Sensation

The goal of this research is to develop the prototype of a tactile sensing platform for anthropomorphic manipulation research. We investigate this problem through the fabrication and simple control of a planar 2-DOF robotic finger inspired by anatomic consistency, self-containment, and adaptability. The robot is equipped with a tactile sensor array based on optical transducer technology whereby localized changes in light intensity within an illuminated foam substrate correspond to the distribution and magnitude of forces applied to the sensor surface plane. The integration of tactile perception is a key component in realizing robotic systems which organically interact with the world. Such natural behavior is characterized by compliant performance that can initiate internal, and respond to external, force application in a dynamic environment. However, most of the current manipulators that support some form of haptic feedback either solely derive proprioceptive sensation or only limit tactile sensors to the mechanical fingertips. These constraints are due to the technological challenges involved in high resolution, multi-point tactile perception. In this work, however, we take the opposite approach, emphasizing the role of full-finger tactile feedback in the refinement of manual capabilities. To this end, we propose and implement a control framework for sensorimotor coordination analogous to infant-level grasping and fixturing reflexes. This thesis details the mechanisms used to achieve these sensory, actuation, and control objectives, along with the design philosophies and biological influences behind them. The results of behavioral experiments with a simple tactilely-modulated control scheme are also described. The hope is to integrate the modular finger into an %engineered analog of the human hand with a complete haptic system.

Reinforcement Learning by Policy Search

One objective of artificial intelligence is to model the behavior of an intelligent agent interacting with its environment. The environment's transformations can be modeled as a Markov chain, whose state is partially observable to the agent and affected by its actions; such processes are known as partially observable Markov decision processes (POMDPs). While the environment's dynamics are assumed to obey certain rules, the agent does not know them and must learn. In this dissertation we focus on the agent's adaptation as captured by the reinforcement learning framework. This means learning a policy---a mapping of observations into actions---based on feedback from the environment. The learning can be viewed as browsing a set of policies while evaluating them by trial through interaction with the environment. The set of policies is constrained by the architecture of the agent's controller. POMDPs require a controller to have a memory. We investigate controllers with memory, including controllers with external memory, finite state controllers and distributed controllers for multi-agent systems. For these various controllers we work out the details of the algorithms which learn by ascending the gradient of expected cumulative reinforcement. Building on statistical learning theory and experiment design theory, a policy evaluation algorithm is developed for the case of experience re-use. We address the question of sufficient experience for uniform convergence of policy evaluation and obtain sample complexity bounds for various estimators. Finally, we demonstrate the performance of the proposed algorithms on several domains, the most complex of which is simulated adaptive packet routing in a telecommunication network.