Statistical mechanics of mutual information maximization

An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.

Learning curves for Gaussian processes models: fluctuations and universality

Based on a statistical mechanics approach, we develop a method for approximately computing average case learning curves and their sample fluctuations for Gaussian process regression models. We give examples for the Wiener process and show that universal relations (that are independent of the input distribution) between error measures can be derived.

A variational approach to learning curves

We combine the replica approach from statistical physics with a variational approach to analyze learning curves analytically. We apply the method to Gaussian process regression. As a main result we derive approximative relations between empirical error measures, the generalization error and the posterior variance.

CDMA multiuser detection, neural networks, and statistical mechanics

A novel approach, based on statistical mechanics, to analyze typical performance of optimum code-division multiple-access (CDMA) multiuser detectors is reviewed. A `black-box' view ot the basic CDMA channel is introduced, based on which the CDMA multiuser detection problem is regarded as a `learning-from-examples' problem of the `binary linear perceptron' in the neural network literature. Adopting Bayes framework, analysis of the performance of the optimum CDMA multiuser detectors is reduced to evaluation of the average of the cumulant generating function of a relevant posterior distribution. The evaluation of the average cumulant generating function is done, based on formal analogy with a similar calculation appearing in the spin glass theory in statistical mechanics, by making use of the replica method, a method developed in the spin glass theory.

A Bayesian approach to on-line learning

Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.

Does reviewing lead to better learning and decision making? Answers from a randomized stock market experiment

Background - The literature is not univocal about the effects of Peer Review (PR) within the context of constructivist learning. Due to the predominant focus on using PR as an assessment tool, rather than a constructivist learning activity, and because most studies implicitly assume that the benefits of PR are limited to the reviewee, little is known about the effects upon students who are required to review their peers. Much of the theoretical debate in the literature is focused on explaining how and why constructivist learning is beneficial. At the same time these discussions are marked by an underlying presupposition of a causal relationship between reviewing and deep learning. Objectives - The purpose of the study is to investigate whether the writing of PR feedback causes students to benefit in terms of: perceived utility about statistics, actual use of statistics, better understanding of statistical concepts and associated methods, changed attitudes towards market risks, and outcomes of decisions that were made. Methods - We conducted a randomized experiment, assigning students randomly to receive PR or non–PR treatments and used two cohorts with a different time span. The paper discusses the experimental design and all the software components that we used to support the learning process: Reproducible Computing technology which allows students to reproduce or re–use statistical results from peers, Collaborative PR, and an AI–enhanced Stock Market Engine. Results - The results establish that the writing of PR feedback messages causes students to experience benefits in terms of Behavior, Non–Rote Learning, and Attitudes, provided the sequence of PR activities are maintained for a period that is sufficiently long.

Learning in ultrametric committee machines

The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.

A Statistical Convergence Aplication for the Hopfield Networks

When Recurrent Neural Networks (RNN) are going to be used as Pattern Recognition systems, the problem to be considered is how to impose prescribed prototype vectors ξ^1,ξ^2,...,ξ^p as fixed points. The synaptic matrix W should be interpreted as a sort of sign correlation matrix of the prototypes, In the classical approach. The weak point in this approach, comes from the fact that it does not have the appropriate tools to deal efficiently with the correlation between the state vectors and the prototype vectors The capacity of the net is very poor because one can only know if one given vector is adequately correlated with the prototypes or not and we are not able to know what its exact correlation degree. The interest of our approach lies precisely in the fact that it provides these tools. In this paper, a geometrical vision of the dynamic of states is explained. A fixed point is viewed as a point in the Euclidean plane R2. The retrieving procedure is analyzed trough statistical frequency distribution of the prototypes. The capacity of the net is improved and the spurious states are reduced. In order to clarify and corroborate the theoretical results, together with the formal theory, an application is presented

Using the e-Class Open Platform for e-Learning Support at the TEI of Larissa, Greece

In the current paper we firstly give a short introduction on e-learning platforms and review the case of the e-class open e-learning platform being used by the Greek tertiary education sector. Our analysis includes strategic selection issues and outcomes in general and operational and adoption issues in the case of the Technological Educational Institute (TEI) of Larissa, Greece. The methodology is being based on qualitative analysis of interviews with key actors using the platform, and statistical analysis of quantitative data related to adoption and usage in the relevant populations. The author has been a key actor in all stages and describes his insights as an early adopter, diffuser and innovative user. We try to explain the issues under consideration using existing past research outcomes and we also arrive to some conclusions and points for further research.

A Taxonomy of Big Data for Optimal Predictive Machine Learning and Data Mining

Big data comes in various ways, types, shapes, forms and sizes. Indeed, almost all areas of science, technology, medicine, public health, economics, business, linguistics and social science are bombarded by ever increasing flows of data begging to be analyzed efficiently and effectively. In this paper, we propose a rough idea of a possible taxonomy of big data, along with some of the most commonly used tools for handling each particular category of bigness. The dimensionality p of the input space and the sample size n are usually the main ingredients in the characterization of data bigness. The specific statistical machine learning technique used to handle a particular big data set will depend on which category it falls in within the bigness taxonomy. Large p small n data sets for instance require a different set of tools from the large n small p variety. Among other tools, we discuss Preprocessing, Standardization, Imputation, Projection, Regularization, Penalization, Compression, Reduction, Selection, Kernelization, Hybridization, Parallelization, Aggregation, Randomization, Replication, Sequentialization. Indeed, it is important to emphasize right away that the so-called no free lunch theorem applies here, in the sense that there is no universally superior method that outperforms all other methods on all categories of bigness. It is also important to stress the fact that simplicity in the sense of Ockham’s razor non-plurality principle of parsimony tends to reign supreme when it comes to massive data. We conclude with a comparison of the predictive performance of some of the most commonly used methods on a few data sets.

A Comparative Analysis of Predictive Learning Algorithms on High-Dimensional Microarray Cancer Data

This research evaluates pattern recognition techniques on a subclass of big data where the dimensionality of the input space (p) is much larger than the number of observations (n). Specifically, we evaluate massive gene expression microarray cancer data where the ratio κ is less than one. We explore the statistical and computational challenges inherent in these high dimensional low sample size (HDLSS) problems and present statistical machine learning methods used to tackle and circumvent these difficulties. Regularization and kernel algorithms were explored in this research using seven datasets where κ < 1. These techniques require special attention to tuning necessitating several extensions of cross-validation to be investigated to support better predictive performance. While no single algorithm was universally the best predictor, the regularization technique produced lower test errors in five of the seven datasets studied.

Teaching Statistics to Engineers: Learning from Experiential Data

The purpose of the work is to claim that engineers can be motivated to study statistical concepts by using the applications in their experience connected with Statistical ideas. The main idea is to choose a data from the manufacturing factility (for example, output from CMM machine) and explain that even if the parts used do not meet exact specifications they are used in production. By graphing the data one can show that the error is random but follows a distribution, that is, there is regularily in the data in statistical sense. As the error distribution is continuous, we advocate that the concept of randomness be introducted starting with continuous random variables with probabilities connected with areas under the density. The discrete random variables are then introduced in terms of decision connected with size of the errors before generalizing to abstract concept of probability. Using software, they can then be motivated to study statistical analysis of the data they encounter and the use of this analysis to make engineering and management decisions.

How Students Study? An Analysis of Learning Strategies in Engineering and Technology Courses

Report published in the Proceedings of the National Conference on "Education and Research in the Information Society", Plovdiv, May, 2014

The effect of cooperative learning on the attitudes toward science and the achievement of students in a non-science majors' general biology laboratory course at an urban community college

The purpose of this study was to evaluate the effect of cooperative learning strategies on students' attitudes toward science and achievement in BSC 1005L, a non-science majors' general biology laboratory course at an urban community college. Data were gathered on the participants' attitudes toward science and cognitive biology level pre and post treatment in BSC 1005L. Elements of the Learning Together model developed by Johnson and Johnson and the Student Team-Achievement Divisions model created by Slavin were incorporated into the experimental sections of BSC 1005L.^ Four sections of BSC 1005L participated in this study. Participants were enrolled in the 1998 spring (January) term. Students met weekly in a two hour laboratory session. The treatment was administered to the experimental group over a ten week period. A quasi-experimental pretest-posttest control group design was used. Students in the cooperative learning group (n$\sb1$ = 27) were administered the Test of Science-Related Attitudes (TOSRA) and the cognitive biology test at the same time as the control group (n$\sb2$ = 19) (at the beginning and end of the term).^ Statistical analyses confirmed that both groups were equivalent regarding ethnicity, gender, college grade point average and number of absences. Independent sample t-tests performed on pretest mean scores indicated no significant differences in the TOSRA scale two or biology knowledge between the cooperative learning group and the control group. The scores of TOSRA scales: one, three, four, five, six, and seven were significantly lower in the cooperative learning group. Independent sample t-tests of the mean score differences did not show any significant differences in posttest attitudes toward science or biology knowledge between the two groups. Paired t-tests did not indicate any significant differences on the TOSRA or biology knowledge within the cooperative learning group. Paired t-tests did show significant differences within the control group on TOSRA scale two and biology knowledge. ANCOVAs did not indicate any significant differences on the post mean scores of the TOSRA or biology knowledge adjusted by differences in the pretest mean scores. Analysis of the research data did not show any significant correlation between attitudes toward science and biology knowledge. ^

Group presence in virtual worlds: Supporting collaborative e-learning

Even though e-learning endeavors have significantly proliferated in recent years, current e-learning technologies provide poor support for group-oriented learning. The now popular virtual world's technologies offer a possible solution. Virtual worlds provide the users with a 3D - computer generated shared space in which they can meet and interact through their virtual representations. Virtual worlds are very successful in developing high levels of engagement, presence and group presence in the users. These elements are also desired in educational settings since they are expected to enhance performance. The goal of this research is to test the hypothesis that a virtual world learning environment provides better support for group-oriented collaborative e-learning than other learning environments, because it facilitates the emergence of group presence. To achieve this, a quasi-experimental study was conducted and data was gathered through the use of various survey instruments and a set of collaborative tasks assigned to the participants. Data was gathered on the dependent variables: Engagement, Group Presence, Individual Presence, Perceived Individual Presence, Perceived Group Presence and Performance. The data was analyzed using the statistical procedures of Factor Analysis, Path Analysis, Analysis of Variance (ANOVA) and Multivariate Analysis of Variance (MANOVA). The study provides support for the hypothesis. The results also show that virtual world learning environments are better than other learning environments in supporting the development of all the dependent variables. It also shows that while only Individual Presence has a significant direct effect on Performance; it is highly correlated with both Engagement and Group Presence. This suggests that these are also important in regards to performance. Developers of e-learning endeavors and educators should incorporate virtual world technologies in their efforts in order to take advantage of the benefit they provide for e-learning group collaboration.