Integrating and evaluating neural word embeddings in information retrieval

Recent advances in neural language models have contributed new methods for learning distributed vector representations of words (also called word embeddings). Two such methods are the continuous bag-of-words model and the skipgram model. These methods have been shown to produce embeddings that capture higher order relationships between words that are highly effective in natural language processing tasks involving the use of word similarity and word analogy. Despite these promising results, there has been little analysis of the use of these word embeddings for retrieval. Motivated by these observations, in this paper, we set out to determine how these word embeddings can be used within a retrieval model and what the benefit might be. To this aim, we use neural word embeddings within the well known translation language model for information retrieval. This language model captures implicit semantic relations between the words in queries and those in relevant documents, thus producing more accurate estimations of document relevance. The word embeddings used to estimate neural language models produce translations that differ from previous translation language model approaches; differences that deliver improvements in retrieval effectiveness. The models are robust to choices made in building word embeddings and, even more so, our results show that embeddings do not even need to be produced from the same corpus being used for retrieval.

Neural stem cell characteristics affected by oncogenic pathways and in a human motoneuron disease

Neural stem cell characteristics affected by oncogenic pathways and in a human motoneuron disease Stem cells provide the self-renewing cell pool for developing or regenerating organs. The mechanisms underlying the decisions of a stem or progenitor cell to either self-renew and maintain multipotentiality or alternatively to differentiate are incompletely understood. In this thesis work, I have approached this question by investigating the role of the proto-oncogene Myc in the regulatory functions of neural progenitor cell (NPC) self-renewal, proliferation and differentiation. By using a retroviral transduction technique to create overexpression models in embryonic NPCs cultured as neurospheres, I show that activated levels of Myc increase NPC self-renewal. Furthermore, several mechanisms that regulate the activity of Myc were identified. Myc induced self-renewal is signalled through binding to the transcription factor Miz-1 as shown by the inhibited capacity of a Myc mutant (MycV394D), deficient in binding to Miz-1, to increase self-renewal in NPCs. Furthermore, overexpression of the newly identified proto-oncogene CIP2A recapitulates the effects of Myc overexpression in NPCs. Also the expression levels and in vivo expression patterns of Myc and CIP2A were linked together. CIP2A stabilizes Myc protein levels in several cancer types by inhibiting its degradation and our results suggest the same function for CIP2A in NPCs. Our results also support the conception of self-renewal and proliferation being two separately regulated cellular functions. Finally, I suggest that Myc regulates NPC self-renewal by influencing the way stem and progenitor cells react to the environmental cues that normally dictate the cellular identity of tissues containing self-renewing cells. Neurosphere cultures were also utilised in order to characterise functional defects in a human disease. Neural stem cell cultures obtained post-mortem from foetuses of lethal congenital contracture syndrome (LCCS) were used to reveal possible cell autonomous differentiation defects of patient NPCs. However, LCCS derived NPCs were able to differentiate normally in vitro although several transcriptional differences were identified by using microarray analysis. Proliferation rate of the patient NPCs was also increased as compared to NPCs of age-matched control foetuses.

Automatic Lameness Detection in a Milking Robot : Instrumentation, measurement software, algorithms for data analysis and a neural network model

The aim of this thesis is to develop a fully automatic lameness detection system that operates in a milking robot. The instrumentation, measurement software, algorithms for data analysis and a neural network model for lameness detection were developed. Automatic milking has become a common practice in dairy husbandry, and in the year 2006 about 4000 farms worldwide used over 6000 milking robots. There is a worldwide movement with the objective of fully automating every process from feeding to milking. Increase in automation is a consequence of increasing farm sizes, the demand for more efficient production and the growth of labour costs. As the level of automation increases, the time that the cattle keeper uses for monitoring animals often decreases. This has created a need for systems for automatically monitoring the health of farm animals. The popularity of milking robots also offers a new and unique possibility to monitor animals in a single confined space up to four times daily. Lameness is a crucial welfare issue in the modern dairy industry. Limb disorders cause serious welfare, health and economic problems especially in loose housing of cattle. Lameness causes losses in milk production and leads to early culling of animals. These costs could be reduced with early identification and treatment. At present, only a few methods for automatically detecting lameness have been developed, and the most common methods used for lameness detection and assessment are various visual locomotion scoring systems. The problem with locomotion scoring is that it needs experience to be conducted properly, it is labour intensive as an on-farm method and the results are subjective. A four balance system for measuring the leg load distribution of dairy cows during milking in order to detect lameness was developed and set up in the University of Helsinki Research farm Suitia. The leg weights of 73 cows were successfully recorded during almost 10,000 robotic milkings over a period of 5 months. The cows were locomotion scored weekly, and the lame cows were inspected clinically for hoof lesions. Unsuccessful measurements, caused by cows standing outside the balances, were removed from the data with a special algorithm, and the mean leg loads and the number of kicks during milking was calculated. In order to develop an expert system to automatically detect lameness cases, a model was needed. A probabilistic neural network (PNN) classifier model was chosen for the task. The data was divided in two parts and 5,074 measurements from 37 cows were used to train the model. The operation of the model was evaluated for its ability to detect lameness in the validating dataset, which had 4,868 measurements from 36 cows. The model was able to classify 96% of the measurements correctly as sound or lame cows, and 100% of the lameness cases in the validation data were identified. The number of measurements causing false alarms was 1.1%. The developed model has the potential to be used for on-farm decision support and can be used in a real-time lameness monitoring system.

N-syndecan and HB-GAM in neural migration and differentiation : Modulation of growth factor activity in brain

The juvenile sea squirt wanders through the sea searching for a suitable rock or hunk of coral to cling to and make its home for life. For this task it has a rudimentary nervous system. When it finds its spot and takes root, it doesn't need its brain any more so it eats it. It's rather like getting tenure. Daniel C. Dennett (from Consciousness Explained, 1991) The little sea squirt needs its brain for a task that is very simple and short. When the task is completed, the sea squirt starts a new life in a vegetative state, after having a nourishing meal. The little brain is more tightly structured than our massive primate brains. The number of neurons is exact, no leeway in neural proliferation is tolerated. Each neuroblast migrates exactly to the correct position, and only a certain number of connections with the right companions is allowed. In comparison, growth of a mammalian brain is a merry mess. The reason is obvious: Squirt brain needs to perform only a few, predictable functions, before becoming waste. The more mobile and complex mammals engage their brains in tasks requiring quick adaptation and plasticity in a constantly changing environment. Although the regulation of nervous system development varies between species, many regulatory elements remain the same. For example, all multicellular animals possess a collection of proteoglycans (PG); proteins with attached, complex sugar chains called glycosaminoglycans (GAG). In development, PGs participate in the organization of the animal body, like in the construction of parts of the nervous system. The PGs capture water with their GAG chains, forming a biochemically active gel at the surface of the cell, and in the extracellular matrix (ECM). In the nervous system, this gel traps inside it different molecules: growth factors and ECM-associated proteins. They regulate the proliferation of neural stem cells (NSC), guide the migration of neurons, and coordinate the formation of neuronal connections. In this work I have followed the role of two molecules contributing to the complexity of mammalian brain development. N-syndecan is a transmembrane heparan sulfate proteoglycan (HSPG) with cell signaling functions. Heparin-binding growth-associated molecule (HB-GAM) is an ECM-associated protein with high expression in the perinatal nervous system, and high affinity to HS and heparin. N-syndecan is a receptor for several growth factors and for HB-GAM. HB-GAM induces specific signaling via N-syndecan, activating c-Src, calcium/calmodulin-dependent serine protein kinase (CASK) and cortactin. By studying the gene knockouts of HB-GAM and N-syndecan in mice, I have found that HB-GAM and N-syndecan are involved as a receptor-ligand-pair in neural migration and differentiation. HB-GAM competes with the growth factors fibriblast growth factor (FGF)-2 and heparin-binding epidermal growth factor (HB-EGF) in HS-binding, causing NSCs to stop proliferation and to differentiate, and affects HB-EGF-induced EGF receptor (EGFR) signaling in neural cells during migration. N-syndecan signaling affects the motility of young neurons, by boosting EGFR-mediated cell migration. In addition, these two receptors form a complex at the surface of the neurons, probably creating a motility-regulating structure.

Differentiation of neural stem cells in fragile X syndrome

Multipotent stem cells can self-renew and give rise to multiple cell types. One type of mammalian multipotent stem cells are neural stem cells (NSC)s, which can generate neurons, astrocytes and oligodendrocytes. NSCs are likely involved in learning and memory, but their exact role in cognitive function in the developing and adult brain is unclear. We have studied properties of NSCs in fragile X syndrome (FXS), which is the most common form of inherited mental retardation. FXS is caused by the lack of functional fragile X mental retardation protein (FMRP). FMRP is involved in the regulation of postsynaptic protein synthesis in a group I metabotropic glutamate receptor 5 (mGluR5)-dependent manner. In the absence of functional FMRP, the formation of functional synapses is impaired in the forebrain which results in alterations in synaptic plasticity. In our studies, we found that FMRP-deficient NSCs generated more neurons and less glia than control NSCs. The newborn neurons derived from FMRP-deficient NSCs showed an abnormally immature morphology. Furthermore, FMRP-deficient NSCs exhibited aberrant oscillatory Ca2+ responses to glutamate, which were specifically abolished by an antagonist of the mGluR5 receptor. The data suggested alterations in glutamatergic differentiation of FMRP-deficient NSCs and were further supported by an accumulation of cells committed to glutamatergic lineage in the subventricular zone of the embryonic Fmr1-knockout (Fmr1-KO) neocortex. Postnatally, the aberrant cells likely contributed to abnormal formation of the neocortex. The findings suggested a defect in the differentiation of distinct glutamatergic mGluR5 responsive cells in the absence of functional FMRP. Furthermore, we found that in the early postnatal Fmr1-KO mouse brain, the expression of mRNA for regulator of G-protein signalling-4 (RGS4) was decreased which was in line with disturbed G-protein signalling in NSCs lacking FMRP. Brain derived neurotrophic factor (BDNF) promotes neuronal differentiation of NSCs as the absence of FMRP was shown to do. This led us to study the effect of impaired BDNF/TrkB receptor signaling on NSCs by overexpression of TrkB.T1 receptor isoform. We showed that changes in the relative expression levels of the full-length and truncated TrkB isoforms influenced the replication capacity of NSCs. After the differentiation, the overexpression of TrkB.T1 increased neuronal turnover. To summarize, FMRP and TrkB signaling are involved in normal differentiation of NSCs in the developing brain. Since NSCs might have potential for therapeutic interventions in a variety of neurological disorders, our findings may be useful in the design of pharmacological interventions in neurological disorders of learning and memory.

From neural stem cells to precursors : Molecular regulation of self-renewal and differentiation

Stem cells are responsible for tissue turnover throughout lifespan. Only highly controlled specific environment, the stem cell niche , can sustain undifferentiated stem cell-pool. The balance between maintenance and differentiation is crucial for individual s health: uncontrolled stem cell self-renewal or proliferation can lead to hyperplasia and mutations that further provoke malignant transformation of the cells. On the other hand, uninhibited differentiation may result in diminished stem cell population, which is unable to maintain tissue turnover. The mechanisms that control the switch from maintenance to differentiation in stem cells are not well known. The same mechanisms that direct the self-renewal and proliferation in normal stem cells are likely to be also involved in maintenance of cancer stem cell . Cancer stem cells exhibit stem cell like properties such as self-renewal- and differentiation capacity and they can also regenerate the tumor tissue. In this thesis, I have investigated the effect of classical oncogenes E6/E7 and c-Myc, tumor suppressors p53 and retinoblastoma (pRb) family, and vascular endothelial growth factor (VEGF) subfamily and glial cell line-derived neurothropic factor (GDNF) family ligands on behavior of embryonic neural stem cells (NSCs) and progenitors. The study includes also the characterization of cytoskeletal tumor suppressor neurofibromatosis 2 (NF2) protein merlin and ezrin-radixin-moesin (ERM) protein ezrin expression in neural progenitors cells and their progeny. This study reveals some potential mechanisms regarding to NSCs maintenance. In summary, the studied molecules are able to shift the balance either towards stem cell maintenance or differentiation; tumor suppressor p53 represses whereas E6/E7 oncogenes and c-Myc increase the proportion of self-renewing and proliferating NSCs or progenitors. The data suggests that active MEK-ERK signaling is critical for self-renewal of normal and oncogene expressing NSCs. In addition, the results indicate that expression of cytoskeletal tumor suppressor merlin and ERM protein ezrin in central nervous system (CNS) tissue and progenitors indicates their role in cell differentiation. Furthermore, the data suggests that VEGF-C a factor involved in lymphatic system development, angiogenesis, neovascularization and metastasis but also in maintenance of some neural populations in brain is a novel thropic factor for progenitors in early sympathetic nervous system (SNS). It seems that VEGF-C dose dependently through ERK-pathway supports the proliferation and survival of early sympathetic progenitor cells, and the effect is comparable to that of GDNF family ligands.

Human mesenchymal stem cells retain multilineage differentiation capacity including neural marker expression after extended in vitro expansion

The suitability of human mesenchymal stem cells (hMSCs) in regenerative medicine relies on retention of their proliferative expansion potential in conjunction with the ability to differentiate toward multiple lineages. Successful utilisation of these cells in clinical applications linked to tissue regeneration requires consideration of biomarker expression, time in culture and donor age, as well as their ability to differentiate towards mesenchymal (bone, cartilage, fat) or non-mesenchymal (e.g., neural) lineages. To identify potential therapeutic suitability we examined hMSCs after extended expansion including morphological changes, potency (stemness) and multilineage potential. Commercially available hMSC populations were expanded in vitro for > 20 passages, equating to > 60 days and > 50 population doublings. Distinct growth phases (A-C) were observed during serial passaging and cells were characterised for stemness and lineage markers at representative stages (Phase A: P+5, approximately 13 days in culture; Phase B: P+7, approximately 20 days in culture; and Phase C: P+13, approximately 43 days in culture). Cell surface markers, stem cell markers and lineage-specific markers were characterised by FACS, ICC and Q-PCR revealing MSCs maintained their multilineage potential, including neural lineages throughout expansion. Co-expression of multiple lineage markers along with continued CD45 expression in MSCs did not affect completion of osteogenic and adipogenic specification or the formation of neurospheres. Improved standardised isolation and characterisation of MSCs may facilitate the identification of biomarkers to improve therapeutic efficacy to ensure increased reproducibility and routine production of MSCs for therapeutic applications including neural repair.

Cell surface heparan sulfate proteoglycans as novel markers of human neural stem cell fate determination

Multipotent neural stem cells (NSCs) provide a model to investigate neurogenesis and develop mechanisms of cell transplantation. In order to define improved markers of stemness and lineage specificity, we examined self-renewal and multi-lineage markers during long-term expansion and under neuronal and astrocyte differentiating conditions in human ESC-derived NSCs (hNSC H9 cells). In addition, with proteoglycans ubiquitous to the neural niche, we also examined heparan sulfate proteoglycans (HSPGs) and their regulatory enzymes. Our results demonstrate that hNSC H9 cells maintain self-renewal and multipotent capacity during extended culture and express HS biosynthesis enzymes and several HSPG core protein syndecans (SDCs) and glypicans (GPCs) at a high level. In addition, hNSC H9 cells exhibit high neuronal and a restricted glial differentiative potential with lineage differentiation significantly increasing expression of many HS biosynthesis enzymes. Furthermore, neuronal differentiation of the cells upregulated SDC4, GPC1, GPC2, GPC3 and GPC6 expression with increased GPC4 expression observed under astrocyte culture conditions. Finally, downregulation of selected HSPG core proteins altered hNSC H9 cell lineage potential. These findings demonstrate an involvement for HSPGs in mediating hNSC maintenance and lineage commitment and their potential use as novel markers of hNSC and neural cell lineage specification.

An adaptive drug delivery design using neural networks for effective treatment of infectious diseases: A simulation study

An adaptive drug delivery design is presented in this paper using neural networks for effective treatment of infectious diseases. The generic mathematical model used describes the coupled evolution of concentration of pathogens, plasma cells, antibodies and a numerical value that indicates the relative characteristic of a damaged organ due to the disease under the influence of external drugs. From a system theoretic point of view, the external drugs can be interpreted as control inputs, which can be designed based on control theoretic concepts. In this study, assuming a set of nominal parameters in the mathematical model, first a nonlinear controller (drug administration) is designed based on the principle of dynamic inversion. This nominal drug administration plan was found to be effective in curing "nominal model patients" (patients whose immunological dynamics conform to the mathematical model used for the control design exactly. However, it was found to be ineffective in curing "realistic model patients" (patients whose immunological dynamics may have off-nominal parameter values and possibly unwanted inputs) in general. Hence, to make the drug delivery dosage design more effective for realistic model patients, a model-following adaptive control design is carried out next by taking the help of neural networks, that are trained online. Simulation studies indicate that the adaptive controller proposed in this paper holds promise in killing the invading pathogens and healing the damaged organ even in the presence of parameter uncertainties and continued pathogen attack. Note that the computational requirements for computing the control are very minimal and all associated computations (including the training of neural networks) can be carried out online. However it assumes that the required diagnosis process can be carried out at a sufficient faster rate so that all the states are available for control computation.