Cumulative restricted Boltzmann machines for ordinal matrix data analysis

Ordinal data is omnipresent in almost all multiuser-generated feedback - questionnaires, preferences etc. This paper investigates modelling of ordinal data with Gaussian restricted Boltzmann machines (RBMs). In particular, we present the model architecture, learning and inference procedures for both vector-variate and matrix-variate ordinal data. We show that our model is able to capture latent opinion profile of citizens around the world, and is competitive against state-of-art collaborative filtering techniques on large-scale public datasets. The model thus has the potential to extend application of RBMs to diverse domains such as recommendation systems, product reviews and expert assessments.

Path planning for CNC machines considering centripetal acceleration and jerk

In planning an s-curve speed profile for a computer numerical control (CNC) machine, centripetal acceleration and its derivative have to be considered. In a CNC machine, these quantities dictate how much voltage and current should be applied to servo motor windings. In this paper, the necessity of considering centripetal jerk in speed profile generation especially in the look-ahead mode is explained. It is demonstrated that the magnitude of centripetal jerk is proportional to the curvature derivative of the path known as "sharpness". It is also explained that a proper limited jerk motion is only possible when a G2-continuous machining path is planned. Then using a simplified mathematical representation of clothoids, a novel method for approximating a given path with a sequence of clothoid segments is proposed. Using this method, a semi-parallel G2-continuous path with adjustable deviation from the original shape for a sample machining contour is generated. Maximum permissible feed rate for the generated path is also calculated.

Prediction interval estimation for electricity price and demand using support vector machines

Uncertainty is known to be a concomitant factor of almost all the real world commodities such as oil prices, stock prices, sales and demand of products. As a consequence, forecasting problems are becoming more and more challenging and ridden with uncertainty. Such uncertainties are generally quantified by statistical tools such as prediction intervals (Pis). Pis quantify the uncertainty related to forecasts by estimating the ranges of the targeted quantities. Pis generated by traditional neural network based approaches are limited by high computational burden and impractical assumptions about the distribution of the data. A novel technique for constructing high quality Pis using support vector machines (SVMs) is being proposed in this paper. The proposed technique directly estimates the upper and lower bounds of the PI in a short time and without any assumptions about the data distribution. The SVM parameters are tuned using particle swarm optimization technique by minimization of a modified Pi-based objective function. Electricity price and demand data of the Ontario electricity market is used to validate the performance of the proposed technique. Several case studies for different months indicate the superior performance of the proposed method in terms of high quality PI generation and shorter computational times.

A comprehensive analytical subdomain model and its field solutions for surface-mounted permanent magnet machines

This paper presents a comprehensive analytical subdomain model together with its field solutions for predicting the magnetic field distributions in surface-mounted permanent magnet (PM) machines. The tooth tips and slotting effects during open-circuit, armature reaction, and on-load conditions are considered when deriving the model and developing its solutions. The model derivations and field solutions are extended from a previous model, and can be applied to PM machines with any combinations of slot and pole numbers and any magnetization patterns in the magnets. This model is initially formulated according to Laplace's and Poisson's equations in 2-D polar coordinates by the separation of variables technique in four subdomains, such as magnet, airgap, winding slots, and slot-openings. The field solution of each subdomain is obtained applying the appropriate boundary conditions and interface conditions between every two subdomains, respectively, which can precisely account for the mutual influence between slots. Finite element analysis (FEA) is later deployed to validate the analytical results in a surface-mounted PM machine that has nonoverlapping winding arrangement. For validation purposes, PM machines having 3-slot/2-pole with parallel magnetization and 12-slot/10-pole with either parallel or radial magnetizations are used for comparisons. Computation of global quantities for the motor which include the phase back-EMF and cogging torque is also included. The results indicate that the proposed analytical model can accurately predict the magnetic field distributions in each subdomain and the motor's global quantities, which are in good agreement with those obtained from the FEA.

Prediction interval estimation of electricity prices using PSO-tuned support vector machines

Uncertainty of the electricity prices makes the task of accurate forecasting quite difficult for the electricity market participants. Prediction intervals (PIs) are statistical tools which quantify the uncertainty related to forecasts by estimating the ranges of the future electricity prices. Traditional approaches based on neural networks (NNs) generate PIs at the cost of high computational burden and doubtful assumptions about data distributions. In this work, we propose a novel technique that is not plagued with the above limitations and it generates high-quality PIs in a short time. The proposed method directly generates the lower and upper bounds of the future electricity prices using support vector machines (SVM). Optimal model parameters are obtained by the minimization of a modified PI-based objective function using a particle swarm optimization (PSO) technique. The efficiency of the proposed method is illustrated using data from Ontario, Pennsylvania-New Jersey-Maryland (PJM) interconnection day-ahead and real-time markets.

Stable feature selection with support vector machines

The support vector machine (SVM) is a popular method for classification, well known for finding the maximum-margin hyperplane. Combining SVM with l1-norm penalty further enables it to simultaneously perform feature selection and margin maximization within a single framework. However, l1-norm SVM shows instability in selecting features in presence of correlated features. We propose a new method to increase the stability of l1-norm SVM by encouraging similarities between feature weights based on feature correlations, which is captured via a feature covariance matrix. Our proposed method can capture both positive and negative correlations between features. We formulate the model as a convex optimization problem and propose a solution based on alternating minimization. Using both synthetic and real-world datasets, we show that our model achieves better stability and classification accuracy compared to several state-of-the-art regularized classification methods.

Graph-induced restricted Boltzmann machines for document modeling

Discovering knowledge from unstructured texts is a central theme in data mining and machine learning. We focus on fast discovery of thematic structures from a corpus. Our approach is based on a versatile probabilistic formulation – the restricted Boltzmann machine (RBM) –where the underlying graphical model is an undirected bipartite graph. Inference is efficient document representation can be computed with a single matrix projection, making RBMs suitable for massive text corpora available today. Standard RBMs, however, operate on bag-of-words assumption, ignoring the inherent underlying relational structures among words. This results in less coherent word thematic grouping. We introduce graph-based regularization schemes that exploit the linguistic structures, which in turn can be constructed from either corpus statistics or domain knowledge. We demonstrate that the proposed technique improves the group coherence, facilitates visualization, provides means for estimation of intrinsic dimensionality, reduces overfitting, and possibly leads to better classification accuracy.

Tensor-variate restricted Boltzmann machines

Restricted Boltzmann Machines (RBMs) are an important class of latent variable models for representing vector data. An under-explored area is multimode data, where each data point is a matrix or a tensor. Standard RBMs applying to such data would require vectorizing matrices and tensors, thus resulting in unnecessarily high dimensionality and at the same time, destroying the inherent higher-order interaction structures. This paper introduces Tensor-variate Restricted Boltzmann Machines (TvRBMs) which generalize RBMs to capture the multiplicative interaction between data modes and the latent variables. TvRBMs are highly compact in that the number of free parameters grows only linear with the number of modes. We demonstrate the capacity of TvRBMs on three real-world applications: handwritten digit classification, face recognition and EEG-based alcoholic diagnosis. The learnt features of the model are more discriminative than the rivals, resulting in better classification performance.

Learning vector representation of medical objects via EMR-driven nonnegative restricted Boltzmann machines (eNRBM)

Electronic medical record (EMR) offers promises for novel analytics. However, manual feature engineering from EMR is labor intensive because EMR is complex - it contains temporal, mixed-type and multimodal data packed in irregular episodes. We present a computational framework to harness EMR with minimal human supervision via restricted Boltzmann machine (RBM). The framework derives a new representation of medical objects by embedding them in a low-dimensional vector space. This new representation facilitates algebraic and statistical manipulations such as projection onto 2D plane (thereby offering intuitive visualization), object grouping (hence enabling automated phenotyping), and risk stratification. To enhance model interpretability, we introduced two constraints into model parameters: (a) nonnegative coefficients, and (b) structural smoothness. These result in a novel model called eNRBM (EMR-driven nonnegative RBM). We demonstrate the capability of the eNRBM on a cohort of 7578 mental health patients under suicide risk assessment. The derived representation not only shows clinically meaningful feature grouping but also facilitates short-term risk stratification. The F-scores, 0.21 for moderate-risk and 0.36 for high-risk, are significantly higher than those obtained by clinicians and competitive with the results obtained by support vector machines.

Learning entry profiles of children with autism from multivariate treatment information using restricted boltzmann machines

Entry profiles can be generated before children with Autism Spectrum Disorders (ASD) begin to traverse an intervention program. They can help evaluate the progress of each child on the dedicated syllabus in addition to enabling narrowing down the best intervention course over time. However, the traits of ASD are expressed in different ways in every individual affected. The resulting spectrum nature of the disorder makes it challenging to discover profiles of children with ASD. Using data from 491 children, traversing the syllabus of a comprehensive intervention program on iPad called TOBY Playpad, we learn the entry profiles of the children based on their age, sex and performance on their first skills of the syllabus. Mixed-variate restricted Boltzmann machines allow us to integrate the heterogeneous data into one model making it a suitable technique. The data based discovery of entry profiles may assist in developing systems that can automatically suggest best suitable paths through the syllabus by clustering the children based on the characteristics they present at the beginning of the program. This may open the pathway for personalised intervention.

Thurstonian Boltzmann machines: learning from multiple inequalities

We introduce Thurstonian Boltzmann Machines (TBM), a unified architecture that can naturally incorporate a wide range of data inputs at the same time. Our motivation rests in the Thurstonian view that many discrete data types can be considered as being generated from a subset of underlying latent continuous variables, and in the observation that each realisation of a discrete type imposes certain inequalities on those variables. Thus learning and inference in TBM reduce to making sense of a set of inequalities. Our proposed TBM naturally supports the following types: Gaussian, intervals, censored, binary, categorical, muticategorical, ordinal, (in)-complete rank with and without ties. We demonstrate the versatility and capacity of the proposed model on three applications of very different natures; namely handwritten digit recognition, collaborative filtering and complex social survey analysis. Copyright 2013 by the author(s).

Vibration Attenuation in Rotating Machines Using Smart Spring Mechanism

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Automatic Classification of Three-Phase Flow Patterns of Heavy Oil in a Horizontal Pipe Using Support Vector Machines

The pipe flow of a viscous-oil-gas-water mixture such as that involved in heavy oil production is a rather complex thereto-fluid dynamical problem. Considering the complexity of three-phase flow, it is of fundamental importance the introduction of a flow pattern classification tool to obtain useful information about the flow structure. Flow patterns are important because they indicate the degree of mixing during flow and the spatial distribution of phases. In particular, the pressure drop and temperature evolution along the pipe is highly dependent on the spatial configuration of the phases. In this work we investigate the three-phase water-assisted flow patterns, i.e. those configurations where water is injected in order to reduce friction caused by the viscous oil. Phase flow rates and pressure drop data from previous laboratory experiments in a horizontal pipe are used for flow pattern identification by means of the 'support vector machine' technique (SVM).