Learning from Geometry

Subspaces and manifolds are two powerful models for high dimensional signals. Subspaces model linear correlation and are a good fit to signals generated by physical systems, such as frontal images of human faces and multiple sources impinging at an antenna array. Manifolds model sources that are not linearly correlated, but where signals are determined by a small number of parameters. Examples are images of human faces under different poses or expressions, and handwritten digits with varying styles. However, there will always be some degree of model mismatch between the subspace or manifold model and the true statistics of the source. This dissertation exploits subspace and manifold models as prior information in various signal processing and machine learning tasks.

A near-low-rank Gaussian mixture model measures proximity to a union of linear or affine subspaces. This simple model can effectively capture the signal distribution when each class is near a subspace. This dissertation studies how the pairwise geometry between these subspaces affects classification performance. When model mismatch is vanishingly small, the probability of misclassification is determined by the product of the sines of the principal angles between subspaces. When the model mismatch is more significant, the probability of misclassification is determined by the sum of the squares of the sines of the principal angles. Reliability of classification is derived in terms of the distribution of signal energy across principal vectors. Larger principal angles lead to smaller classification error, motivating a linear transform that optimizes principal angles. This linear transformation, termed TRAIT, also preserves some specific features in each class, being complementary to a recently developed Low Rank Transform (LRT). Moreover, when the model mismatch is more significant, TRAIT shows superior performance compared to LRT.

The manifold model enforces a constraint on the freedom of data variation. Learning features that are robust to data variation is very important, especially when the size of the training set is small. A learning machine with large numbers of parameters, e.g., deep neural network, can well describe a very complicated data distribution. However, it is also more likely to be sensitive to small perturbations of the data, and to suffer from suffer from degraded performance when generalizing to unseen (test) data.

From the perspective of complexity of function classes, such a learning machine has a huge capacity (complexity), which tends to overfit. The manifold model provides us with a way of regularizing the learning machine, so as to reduce the generalization error, therefore mitigate overfiting. Two different overfiting-preventing approaches are proposed, one from the perspective of data variation, the other from capacity/complexity control. In the first approach, the learning machine is encouraged to make decisions that vary smoothly for data points in local neighborhoods on the manifold. In the second approach, a graph adjacency matrix is derived for the manifold, and the learned features are encouraged to be aligned with the principal components of this adjacency matrix. Experimental results on benchmark datasets are demonstrated, showing an obvious advantage of the proposed approaches when the training set is small.

Stochastic optimization makes it possible to track a slowly varying subspace underlying streaming data. By approximating local neighborhoods using affine subspaces, a slowly varying manifold can be efficiently tracked as well, even with corrupted and noisy data. The more the local neighborhoods, the better the approximation, but the higher the computational complexity. A multiscale approximation scheme is proposed, where the local approximating subspaces are organized in a tree structure. Splitting and merging of the tree nodes then allows efficient control of the number of neighbourhoods. Deviation (of each datum) from the learned model is estimated, yielding a series of statistics for anomaly detection. This framework extends the classical {\em changepoint detection} technique, which only works for one dimensional signals. Simulations and experiments highlight the robustness and efficacy of the proposed approach in detecting an abrupt change in an otherwise slowly varying low-dimensional manifold.

Chondrogenesis of adult stem cells from adipose tissue and bone marrow: induction by growth factors and cartilage-derived matrix.

OBJECTIVES: Adipose-derived stem cells (ASCs) and bone marrow-derived mesenchymal stem cells (MSCs) are multipotent adult stem cells with potential for use in cartilage tissue engineering. We hypothesized that these cells show distinct responses to different chondrogenic culture conditions and extracellular matrices, illustrating important differences between cell types. METHODS: Human ASCs and MSCs were chondrogenically differentiated in alginate beads or a novel scaffold of reconstituted native cartilage-derived matrix with a range of growth factors, including dexamethasone, transforming growth factor beta3, and bone morphogenetic protein 6. Constructs were analyzed for gene expression and matrix synthesis. RESULTS: Chondrogenic growth factors induced a chondrocytic phenotype in both ASCs and MSCs in alginate beads or cartilage-derived matrix. MSCs demonstrated enhanced type II collagen gene expression and matrix synthesis as well as a greater propensity for the hypertrophic chondrocyte phenotype. ASCs had higher upregulation of aggrecan gene expression in response to bone morphogenetic protein 6 (857-fold), while MSCs responded more favorably to transforming growth factor beta3 (573-fold increase). CONCLUSIONS: ASCs and MSCs are distinct cell types as illustrated by their unique responses to growth factor-based chondrogenic induction. This chondrogenic induction is affected by the composition of the scaffold and the presence of serum.

Ligament-derived matrix stimulates a ligamentous phenotype in human adipose-derived stem cells.

Human adipose stem cells (hASCs) can differentiate into a variety of phenotypes. Native extracellular matrix (e.g., demineralized bone matrix or small intestinal submucosa) can influence the growth and differentiation of stem cells. The hypothesis of this study was that a novel ligament-derived matrix (LDM) would enhance expression of a ligamentous phenotype in hASCs compared to collagen gel alone. LDM prepared using phosphate-buffered saline or 0.1% peracetic acid was mixed with collagen gel (COL) and was evaluated for its ability to induce proliferation, differentiation, and extracellular matrix synthesis in hASCs over 28 days in culture at different seeding densities (0, 0.25 x 10(6), 1 x 10(6), or 2 x 10(6) hASC/mL). Biochemical and gene expression data were analyzed using analysis of variance. Fisher's least significant difference test was used to determine differences between treatments following analysis of variance. hASCs in either LDM or COL demonstrated changes in gene expression consistent with ligament development. hASCs cultured with LDM demonstrated more dsDNA content, sulfated-glycosaminoglycan accumulation, and type I and III collagen synthesis, and released more sulfated-glycosaminoglycan and collagen into the medium compared to hASCs in COL (p

Assessment of LD matrix measures for the analysis of biological pathway association.

Complex diseases will have multiple functional sites, and it will be invaluable to understand the cross-locus interaction in terms of linkage disequilibrium (LD) between those sites (epistasis) in addition to the haplotype-LD effects. We investigated the statistical properties of a class of matrix-based statistics to assess this epistasis. These statistical methods include two LD contrast tests (Zaykin et al., 2006) and partial least squares regression (Wang et al., 2008). To estimate Type 1 error rates and power, we simulated multiple two-variant disease models using the SIMLA software package. SIMLA allows for the joint action of up to two disease genes in the simulated data with all possible multiplicative interaction effects between them. Our goal was to detect an interaction between multiple disease-causing variants by means of their linkage disequilibrium (LD) patterns with other markers. We measured the effects of marginal disease effect size, haplotype LD, disease prevalence and minor allele frequency have on cross-locus interaction (epistasis). In the setting of strong allele effects and strong interaction, the correlation between the two disease genes was weak (r=0.2). In a complex system with multiple correlations (both marginal and interaction), it was difficult to determine the source of a significant result. Despite these complications, the partial least squares and modified LD contrast methods maintained adequate power to detect the epistatic effects; however, for many of the analyses we often could not separate interaction from a strong marginal effect. While we did not exhaust the entire parameter space of possible models, we do provide guidance on the effects that population parameters have on cross-locus interaction.

Integrin-mediated interactions with extracellular matrix proteins for nucleus pulposus cells of the human intervertebral disc.

The extracellular matrix (ECM) of the human intervertebral disc is rich in molecules that interact with cells through integrin-mediated attachments. Porcine nucleus pulposus (NP) cells have been shown to interact with laminin (LM) isoforms LM-111 and LM-511 through select integrins that regulate biosynthesis and cell attachment. Since human NP cells lose many phenotypic characteristics with age, attachment and interaction with the ECM may be altered. Expression of LM-binding integrins was quantified for human NP cells using flow cytometry. The cell-ECM attachment mechanism was determined by quantifying cell attachment to LM-111, LM-511, or type II collagen after functionally blocking specific integrin subunits. Human NP cells express integrins β1, α3, and α5, with over 70% of cells positive for each subunit. Blocking subunit β1 inhibited NP cell attachment to all substrates. Blocking subunits α1, α2, α3, and α5 simultaneously, but not individually, inhibits NP cell attachment to laminins. While integrin α6β1 mediated porcine NP cell attachment to LM-111, we found integrins α3, α5, and β1 instead contributed to human NP cell attachment. These findings identify integrin subunits that may mediate interactions with the ECM for human NP cells and could be used to promote cell attachment, survival, and biosynthesis in cell-based therapeutics.

The role of extracellular matrix elasticity and composition in regulating the nucleus pulposus cell phenotype in the intervertebral disc: a narrative review.

Intervertebral disc (IVD) disorders are a major contributor to disability and societal health care costs. Nucleus pulposus (NP) cells of the IVD exhibit changes in both phenotype and morphology with aging-related IVD degeneration that may impact the onset and progression of IVD pathology. Studies have demonstrated that immature NP cell interactions with their extracellular matrix (ECM) may be key regulators of cellular phenotype, metabolism and morphology. The objective of this article is to review our recent experience with studies of NP cell-ECM interactions that reveal how ECM cues can be manipulated to promote an immature NP cell phenotype and morphology. Findings demonstrate the importance of a soft (<700 Pa), laminin-containing ECM in regulating healthy, immature NP cells. Knowledge of NP cell-ECM interactions can be used for development of tissue engineering or cell delivery strategies to treat IVD-related disorders.

Interstitial engraftment of adipose-derived stem cells into an acellular dermal matrix results in improved inward angiogenesis and tissue incorporation.

Acellular dermal matrices (ADM) are commonly used in reconstructive procedures and rely on host cell invasion to become incorporated into host tissues. We investigated different approaches to adipose-derived stem cells (ASCs) engraftment into ADM to enhance this process. Lewis rat adipose-derived stem cells were isolated and grafted (3.0 × 10(5) cells) to porcine ADM disks (1.5 mm thick × 6 mm diameter) using either passive onlay or interstitial injection seeding techniques. Following incubation, seeding efficiency and seeded cell viability were measured in vitro. In addition, Eighteen Lewis rats underwent subcutaneous placement of ADM disk either as control or seeded with PKH67 labeled ASCs. ADM disks were seeded with ASCs using either onlay or injection techniques. On day 7 and or 14, ADM disks were harvested and analyzed for host cell infiltration. Onlay and injection techniques resulted in unique seeding patterns; however cell seeding efficiency and cell viability were similar. In-vivo studies showed significantly increased host cell infiltration towards the ASCs foci following injection seeding in comparison to control group (p < 0.05). Moreover, regional endothelial cell invasion was significantly greater in ASCs injected grafts in comparison to onlay seeding (p < 0.05). ADM can successfully be engrafted with ASCs. Interstitial engraftment of ASCs into ADM via injection enhances regional infiltration of host cells and angiogenesis, whereas onlay seeding showed relatively broad and superficial cell infiltration. These findings may be applied to improve the incorporation of avascular engineered constructs.