Supervised Gaussian Process Latent Variable Model for Dimensionality Reduction

The Gaussian process latent variable model (GP-LVM) has been identified to be an effective probabilistic approach for dimensionality reduction because it can obtain a low-dimensional manifold of a data set in an unsupervised fashion. Consequently, the GP-LVM is insufficient for supervised learning tasks (e. g., classification and regression) because it ignores the class label information for dimensionality reduction. In this paper, a supervised GP-LVM is developed for supervised learning tasks, and the maximum a posteriori algorithm is introduced to estimate positions of all samples in the latent variable space. We present experimental evidences suggesting that the supervised GP-LVM is able to use the class label information effectively, and thus, it outperforms the GP-LVM and the discriminative extension of the GP-LVM consistently. The comparison with some supervised classification methods, such as Gaussian process classification and support vector machines, is also given to illustrate the advantage of the proposed method.

滑坡泥石流信息遥感获取与空间预测

On the issue of geological hazard evaluation（GHE）, taking remote sensing and GIS systems as experimental environment, assisting with some programming development, this thesis combines multi-knowledges of geo-hazard mechanism, statistic learning, remote sensing (RS), high-spectral recognition, spatial analysis, digital photogrammetry as well as mineralogy, and selects geo-hazard samples from Hong Kong and Three Parallel River region as experimental data, to study two kinds of core questions of GHE, geo-hazard information acquiring and evaluation model. In the aspect of landslide information acquiring by RS, three detailed topics are presented, image enhance for visual interpretation, automatic recognition of landslide as well as quantitative mineral mapping. As to the evaluation model, the latest and powerful data mining method, support vector machine (SVM), is introduced to GHE field, and a serious of comparing experiments are carried out to verify its feasibility and efficiency. Furthermore, this paper proposes a method to forecast the distribution of landslides if rainfall in future is known baseing on historical rainfall and corresponding landslide susceptibility map. The details are as following: (a) Remote sensing image enhancing methods for geo-hazard visual interpretation. The effect of visual interpretation is determined by RS data and image enhancing method, for which the most effective and regular technique is image merge between high-spatial image and multi-spectral image, but there are few researches concerning the merging methods of geo-hazard recognition. By the comparing experimental of six mainstream merging methods and combination of different remote sensing data source, this thesis presents merits of each method ,and qualitatively analyzes the effect of spatial resolution, spectral resolution and time phase on merging image. (b) Automatic recognition of shallow landslide by RS image. The inventory of landslide is the base of landslide forecast and landslide study. If persistent collecting of landslide events, updating the geo-hazard inventory in time, and promoting prediction model incessantly, the accuracy of forecast would be boosted step by step. RS technique is a feasible method to obtain landslide information, which is determined by the feature of geo-hazard distribution. An automatic hierarchical approach is proposed to identify shallow landslides in vegetable region by the combination of multi-spectral RS imagery and DEM derivatives, and the experiment is also drilled to inspect its efficiency. (c) Hazard-causing factors obtaining. Accurate environmental factors are the key to analyze and predict the risk of regional geological hazard. As to predict huge debris flow, the main challenge is still to determine the startup material and its volume in debris flow source region. Exerting the merits of various RS technique, this thesis presents the methods to obtain two important hazard-causing factors, DEM and alteration mineral, and through spatial analysis, finds the relationship between hydrothermal clay alteration minerals and geo-hazards in the arid-hot valleys of Three Parallel Rivers region. (d) Applying support vector machine (SVM) to landslide susceptibility mapping. Introduce the latest and powerful statistical learning theory, SVM, to RGHE. SVM that proved an efficient statistic learning method can deal with two-class and one-class samples, with feature avoiding produce ‘pseudo’ samples. 55 years historical samples in a natural terrain of Hong Kong are used to assess this method, whose susceptibility maps obtained by one-class SVM and two-class SVM are compared to that obtained by logistic regression method. It can conclude that two-class SVM possesses better prediction efficiency than logistic regression and one-class SVM. However, one-class SVM, only requires failed cases, has an advantage over the other two methods as only "failed" case information is usually available in landslide susceptibility mapping. (e) Predicting the distribution of rainfall-induced landslides by time-series analysis. Rainfall is the most dominating factor to bring in landslides. More than 90% losing and casualty by landslides is introduced by rainfall, so predicting landslide sites under certain rainfall is an important geological evaluating issue. With full considering the contribution of stable factors (landslide susceptibility map) and dynamic factors (rainfall), the time-series linear regression analysis between rainfall and landslide risk mapis presented, and experiments based on true samples prove that this method is perfect in natural region of Hong Kong. The following 4 practicable or original findings are obtained: 1) The RS ways to enhance geo-hazards image, automatic recognize shallow landslides, obtain DEM and mineral are studied, and the detailed operating steps are given through examples. The conclusion is practical strongly. 2) The explorative researching about relationship between geo-hazards and alteration mineral in arid-hot valley of Jinshajiang river is presented. Based on standard USGS mineral spectrum, the distribution of hydrothermal alteration mineral is mapped by SAM method. Through statistic analysis between debris flows and hazard-causing factors, the strong correlation between debris flows and clay minerals is found and validated. 3) Applying SVM theory (especially one-class SVM theory) to the landslide susceptibility mapping and system evaluation for its performance is also carried out, which proves that advantages of SVM in this field. 4) Establishing time-serial prediction method for rainfall induced landslide distribution. In a natural study area, the distribution of landslides induced by a storm is predicted successfully under a real maximum 24h rainfall based on the regression between 4 historical storms and corresponding landslides.

社会形势预警的几种建模方法比较

Based on social survey data conducted by local research group in some counties executed in the nearly past five years in China, the author proposed and solved two kernel problems in the field of social situation forecasting: i) How can the attitudes’ data on individual level be integrated with social situation data on macrolevel; ii) How can the powers of forecasting models’ constructed by different statistic methods be compared? Five integrative statistics were applied to the research: 1) algorithm average (MEAN); 2) standard deviation (SD); 3) coefficient variability (CV); 4) mixed secondary moment (M2); 5) Tendency (TD). To solve the former problem, the five statistics were taken to synthesize the individual and mocrolevel data of social situations on the levels of counties’ regions, and form novel integrative datasets, from the basis of which, the latter problem was accomplished by the author: modeling methods such as Multiple Regression Analysis (MRA), Discriminant Analysis (DA) and Support Vector Machine (SVM) were used to construct several forecasting models. Meanwhile, on the dimensions of stepwise vs. enter, short-term vs. long-term forecasting and different integrative (statistic) models, meta-analysis and power analysis were taken to compare the predicting power of each model within and among modeling methods. Finally, it can be concluded from the research of the dissertation: 1) Exactly significant difference exists among different integrative (statistic) models, in which, tendency (TD) integrative models have the highest power, but coefficient variability (CV) ones have the lowest; 2) There is no significant difference of the power between stepwise and enter models as well as short-term and long-term forecasting models; 3) There is significant difference among models constructed by different methods, of which, support vector machine (SVM) has the highest statistic power. This research founded basis in all facets for exploring the optimal forecasting models of social situation’s more deeply, further more, it is the first time methods of meta-analysis and power analysis were immersed into the assessments of such forecasting models.

应用SVM预测杜鹃属（Rhododendron L.）在中国的潜在分布中心

杜鹃属(Rhododendron L.)是中国种子植物中最大的属，其现代分布和分化中心是我国西南部的横断山区和东喜马拉雅地区。我国西部、西南部的云南、四川、西藏等地共有杜鹃达450种，仅特有种就有约300种。对杜鹃属分布的深入研究是横断山区生物多样性保护不可缺少的重要部分。 由于物种分布与环境因子之间存在着紧密的联系，利用环境因子作为预测物种分布模型的变量是当前最普遍的建模思路。但是绝大多数物种分布预测模型都遇到了难以解决的“高维小样本”问题――模型在标本数据不足时无法给出合理的预测，或者模型无法处理大量的环境变量。机器学习领域的理论和实践已经证明，基于结构风险最小化原理的支持向量机(Support Vector Machine, SVM)算法非常适合“高维小样本”的分类问题。为了探索其应用在物种分布预测问题上的可能性，本文创新性的实现了基于SVM算法的物种分布预测系统。然后，本文以30个杜鹃属(Rhododendron L.)物种为检验对象，利用其标本数据和11个1km的栅格环境变量图层作为模型变量，预测其在中国的潜在分布区。本文通过全面的模型评估——专家评估，ROC (Receiver Operator Characteristic)曲线和曲线下方面积AUC (Area Under the Curve)——来比较模型的性能。试验结果表明，我们所实现的以SVM为核心的物种分布预测系统无论在计算速度还是预测效果上都远远优于当前广泛使用的GARP (Genetic Algorithm for Rule-Set Prediction)预测系统。 之后，本文进一步探讨了SVM预测系统预测效果与环境变量维数和标本点个数的关系。试验结果表明，对于只有少量标本点的物种SVM的预测结果仍然具有相当的合理性。由此可见， SVM预测系统很好的解决了以前众多模型无法克服的稀有种和标本点稀少的物种的潜在分布区模拟问题。同时本文发现大的环境维数（高维）对于物种潜在分布区的预测有着决定性的作用，因此模型处理高维问题的能力显得至关重要。 最后，我们使用中国所有可获取的杜鹃属标本数据，以及83个1km的栅格环境变量图层，对400种杜鹃属物种的潜在分布区进行预测。根据预测出来的物种潜在分布区，我们得到了中国杜鹃属物种潜在多样性分布格局，特有物种潜在多样性分布格局，濒危杜物种潜在的分布格局，各亚属物种潜在分布格局，以及不同生活型物种潜在多样性分布格局。这些分布区图不仅可以对杜鹃属起源研究提供分析验证的条件，还能为其引种、保护和新种的搜寻提供有利的空间依据。

Biomimetic pattern recognition theory and its applications

Biomimetic pattern recogntion (BPR), which is based on "cognition" instead of "classification", is much closer to the function of human being. The basis of BPR is the Principle of homology-continuity (PHC), which means the difference between two samples of the same class must be gradually changed. The aim of BPR is to find an optimal covering in the feature space, which emphasizes the "similarity" among homologous group members, rather than "division" in traditional pattern recognition. Some applications of BPR are surveyed, in which the results of BPR are much better than the results of Support Vector Machine. A novel neuron model, Hyper sausage neuron (HSN), is shown as a kind of covering units in BPR. The mathematical description of HSN is given and the 2-dimensional discriminant boundary of HSN is shown. In two special cases, in which samples are distributed in a line segment and a circle, both the HSN networks and RBF networks are used for covering. The results show that HSN networks act better than RBF networks in generalization, especially for small sample set, which are consonant with the results of the applications of BPR. And a brief explanation of the HSN networks' advantages in covering general distributed samples is also given.

Geometrical learning, descriptive geometry, and biomimetic pattern recognition

Studies on learning problems from geometry perspective have attracted an ever increasing attention in machine learning, leaded by achievements on information geometry. This paper proposes a different geometrical learning from the perspective of high-dimensional descriptive geometry. Geometrical properties of high-dimensional structures underlying a set of samples are learned via successive projections from the higher dimension to the lower dimension until two-dimensional Euclidean plane, under guidance of the established properties and theorems in high-dimensional descriptive geometry. Specifically, we introduce a hyper sausage like geometry shape for learning samples and provides a geometrical learning algorithm for specifying the hyper sausage shapes, which is then applied to biomimetic pattern recognition. Experimental results are presented to show that the proposed approach outperforms three types of support vector machines with either a three degree polynomial kernel or a radial basis function kernel, especially in the cases of high-dimensional samples of a finite size. (c) 2005 Elsevier B.V. All rights reserved.

Mandarin speech emotion recognition based on high dimensional geometry theory

In this paper, a novel approach for mandarin speech emotion recognition, that is mandarin speech emotion recognition based on high dimensional geometry theory, is proposed. The human emotions are classified into 6 archetypal classes: fear, anger, happiness, sadness, surprise and disgust. According to the characteristics of these emotional speech signals, the amplitude, pitch frequency and formant are used as the feature parameters for speech emotion recognition. The new method called high dimensional geometry theory is applied for recognition. Compared with traditional GSVM model, the new method has some advantages. It is noted that this method has significant values for researches and applications henceforth.

支持向量机中引入后验概率的理论和方法研究

目前支持向量机解决模式识别问题是广大学者研究的热点，样本的后验概率在模式识别中至关重要，但是传统的支持向量机技术不提供后验概率．针对这一问题进行了３个方面的研究：①在给出样本点后验概率的基础上，将大规模优化问题分解成最大似然函数和最大分类边界两个小规模优化问题；②给出了一种新的用后验概率修正最优分离超平面的方法，并且分析了该新方法的合理性；③用图像分类的３组实例说明本方法的有效性．

基于支持向量机的Internet流量分类研究

准确的网络流量分类是众多网络研究工作的基础,也一直是网络测量领域的研究热点.近年来,利用机器学习方法处理流量分类问题成为了该领域一个新兴的研究方向.在目前研究中应用较多的是朴素贝叶斯(nave Bayes,NB)及其改进算法.这些方法具有实现简单、分类高效的特点.但该方法过分依赖于样本空间的分布,具有内在的不稳定性.因此,提出一种基于支持向量机(support vector machine,SVM)的流量分类方法.该方法利用非线性变换和结构风险最小化(structural risk minimization,SRM)原则将流量分类问题转化为二次寻优问题,具有良好的分类准确率和稳定性.在理论分析的基础上,通过在实际网络流集合上与朴素贝叶斯算法的对比实验,可以看出使用支持向量机方法处理流量分类问题,具有以下3个优势:1)网络流属性不必满足条件独立假设,无须进行属性过滤;2)能够在先验知识相对不足的情况下,仍保持较高的分类准确率;3)不依赖于样本空间的分布,具有较好的分类稳定性.

an empirical study on bug assignment automation using chinese bug data

Univ SE Calif, Ctr Syst & Software Engn, ABB, Microsoft Res, IEEE, ACMSIGSOFT, N Carolina State Univ Comp Sci

Efficient HOG human detection

While Histograms of Oriented Gradients (HOG) plus Support Vector Machine (SVM) (HOG+SVM) is the most successful human detection algorithm, it is time-consuming. This paper proposes two ways to deal with this problem. One way is to reuse the features in blocks to construct the HOG features for intersecting detection windows. Another way is to utilize sub-cell based interpolation to efficiently compute the HOG features for each block. The combination of the two ways results in significant increase in detecting humans-more than five times better. To evaluate the proposed method, we have established a top-view human database. Experimental results on the top-view database and the well-known INRIA data set have demonstrated the effectiveness and efficiency of the proposed method. (C) 2010 Elsevier B.V. All rights reserved.

基于支持向量多分类机的多类复杂手操作EEG信号模式识别

针对用于服务机器人的脑机接口系统中脑电信号模式识别精度不高,不能满足机器人多任务要求的问题,提出一种基于C-支持向量多分类机的多类复杂手操作EEG信号模式识别方法,并将其应用到复杂手操作的EEG信号模式识别试验中,实现一个4类复杂手操作的模式识别,实验结果表明,与之前用BP神经网络进行识别相比,识别率由85%提高到了90%。