Crowd Counting Using Multiple Local Features

In public venues, crowd size is a key indicator of crowd safety and stability. Crowding levels can be detected using holistic image features, however this requires a large amount of training data to capture the wide variations in crowd distribution. If a crowd counting algorithm is to be deployed across a large number of cameras, such a large and burdensome training requirement is far from ideal. In this paper we propose an approach that uses local features to count the number of people in each foreground blob segment, so that the total crowd estimate is the sum of the group sizes. This results in an approach that is scalable to crowd volumes not seen in the training data, and can be trained on a very small data set. As a local approach is used, the proposed algorithm can easily be used to estimate crowd density throughout different regions of the scene and be used in a multi-camera environment. A unique localised approach to ground truth annotation reduces the required training data is also presented, as a localised approach to crowd counting has different training requirements to a holistic one. Testing on a large pedestrian database compares the proposed technique to existing holistic techniques and demonstrates improved accuracy, and superior performance when test conditions are unseen in the training set, or a minimal training set is used.

Crowd counting using group tracking and local features

In public venues, crowd size is a key indicator of crowd safety and stability. In this paper we propose a crowd counting algorithm that uses tracking and local features to count the number of people in each group as represented by a foreground blob segment, so that the total crowd estimate is the sum of the group sizes. Tracking is employed to improve the robustness of the estimate, by analysing the history of each group, including splitting and merging events. A simplified ground truth annotation strategy results in an approach with minimal setup requirements that is highly accurate.

Wide baseline correspondence extraction beyond local features

Robust, affine covariant, feature extractors provide a means to extract correspondences between images captured by widely separated cameras. Advances in wide baseline correspondence extraction require looking beyond the robust feature extraction and matching approach. This study examines new techniques of extracting correspondences that take advantage of information contained in affine feature matches. Methods of improving the accuracy of a set of putative matches, eliminating incorrect matches and extracting large numbers of additional correspondences are explored. It is assumed that knowledge of the camera geometry is not available and not immediately recoverable. The new techniques are evaluated by means of an epipolar geometry estimation task. It is shown that these methods enable the computation of camera geometry in many cases where existing feature extractors cannot produce sufficient numbers of accurate correspondences.

ReadingAct RGB-D action dataset and human action recognition from local features

For general home monitoring, a system should automatically interpret people’s actions. The system should be non-intrusive, and able to deal with a cluttered background, and loose clothes. An approach based on spatio-temporal local features and a Bag-of-Words (BoW) model is proposed for single-person action recognition from combined intensity and depth images. To restore the temporal structure lost in the traditional BoW method, a dynamic time alignment technique with temporal binning is applied in this work, which has not been previously implemented in the literature for human action recognition on depth imagery. A novel human action dataset with depth data has been created using two Microsoft Kinect sensors. The ReadingAct dataset contains 20 subjects and 19 actions for a total of 2340 videos. To investigate the effect of using depth images and the proposed method, testing was conducted on three depth datasets, and the proposed method was compared to traditional Bag-of-Words methods. Results showed that the proposed method improves recognition accuracy when adding depth to the conventional intensity data, and has advantages when dealing with long actions.

Beyond the average: Detecting global singular nodes from local features in complex networks

Deviations from the average can provide valuable insights about the organization of natural systems. The present article extends this important principle to the systematic identification and analysis of singular motifs in complex networks. Six measurements quantifying different and complementary features of the connectivity around each node of a network were calculated, and multivariate statistical methods applied to identify singular nodes. The potential of the presented concepts and methodology was illustrated with respect to different types of complex real-world networks, namely the US air transportation network, the protein-protein interactions of the yeast Saccharomyces cerevisiae and the Roget thesaurus networks. The obtained singular motifs possessed unique functional roles in the networks. Three classic theoretical network models were also investigated, with the Barabasi-Albert model resulting in singular motifs corresponding to hubs, confirming the potential of the approach. Interestingly, the number of different types of singular node motifs as well as the number of their instances were found to be considerably higher in the real-world networks than in any of the benchmark networks. Copyright (C) EPLA, 2009

Object categorization via sparse representation of local features

Sparse representation has been introduced to address many recognition problems in computer vision. In this paper, we propose a new framework for object categorization based on sparse representation of local features. Unlike most of previous sparse coding based methods in object classification that only use sparse coding to extract high-level features, the proposed method incorporates sparse representation and classification into a unified framework. Therefore, it does not need a further classifier. Experimental results show that the proposed method achieved better or comparable accuracy than the well known bag-of-features representation with various classifiers.

Mach edges: local features predicted by 3rd derivative spatial filtering

Edges are key points of information in visual scenes. One important class of models supposes that edges correspond to the steepest parts of the luminance profile, implying that they can be found as peaks and troughs in the response of a gradient (1st derivative) filter, or as zero-crossings in the 2nd derivative (ZCs). We tested those ideas using a stimulus that has no local peaks of gradient and no ZCs, at any scale. The stimulus profile is analogous to the Mach ramp, but it is the luminance gradient (not the absolute luminance) that increases as a linear ramp between two plateaux; the luminance profile is a blurred triangle-wave. For all image-blurs tested, observers marked edges at or close to the corner points in the gradient profile, even though these were not gradient maxima. These Mach edges correspond to peaks and troughs in the 3rd derivative. Thus Mach edges are inconsistent with many standard edge-detection schemes, but are nicely predicted by a recent model that finds edge points with a 2-stage sequence of 1st then 2nd derivative operators, each followed by a half-wave rectifier.

Hierarchically grouped 2D local features applied to edge contour localisation

One of the most significant research topics in computer vision is object detection. Most of the reported object detection results localise the detected object within a bounding box, but do not explicitly label the edge contours of the object. Since object contours provide a fundamental diagnostic of object shape, some researchers have initiated work on linear contour feature representations for object detection and localisation. However, linear contour feature-based localisation is highly dependent on the performance of linear contour detection within natural images, and this can be perturbed significantly by a cluttered background. In addition, the conventional approach to achieving rotation-invariant features is to rotate the feature receptive field to align with the local dominant orientation before computing the feature representation. Grid resampling after rotation adds extra computational cost and increases the total time consumption for computing the feature descriptor. Though it is not an expensive process if using current computers, it is appreciated that if each step of the implementation is faster to compute especially when the number of local features is increasing and the application is implemented on resource limited ”smart devices”, such as mobile phones, in real-time. Motivated by the above issues, a 2D object localisation system is proposed in this thesis that matches features of edge contour points, which is an alternative method that takes advantage of the shape information for object localisation. This is inspired by edge contour points comprising the basic components of shape contours. In addition, edge point detection is usually simpler to achieve than linear edge contour detection. Therefore, the proposed localization system could avoid the need for linear contour detection and reduce the pathological disruption from the image background. Moreover, since natural images usually comprise many more edge contour points than interest points (i.e. corner points), we also propose new methods to generate rotation-invariant local feature descriptors without pre-rotating the feature receptive field to improve the computational efficiency of the whole system. In detail, the 2D object localisation system is achieved by matching edge contour points features in a constrained search area based on the initial pose-estimate produced by a prior object detection process. The local feature descriptor obtains rotation invariance by making use of rotational symmetry of the hexagonal structure. Therefore, a set of local feature descriptors is proposed based on the hierarchically hexagonal grouping structure. Ultimately, the 2D object localisation system achieves a very promising performance based on matching the proposed features of edge contour points with the mean correct labelling rate of the edge contour points 0.8654 and the mean false labelling rate 0.0314 applied on the data from Amsterdam Library of Object Images (ALOI). Furthermore, the proposed descriptors are evaluated by comparing to the state-of-the-art descriptors and achieve competitive performances in terms of pose estimate with around half-pixel pose error.

The whole song is greater than the sum of its parts: Local and structural features in music listening

Four experiments reported here demonstrate the importance of structural as well as local features in listening to contemporary popular music. Experiment 1 established that listeners without formal musical training regard as salient the formal structure that links individual sections of songs. When asked to listen to and assemble the individual sections of unfamiliar contemporary songs to form new compositions, participants positioned the sections in ways consistent with the true structure of the music. In Experiment 2, participants were provided with only the song lyrics with which to arrange the individual sections of contemporary songs. It was found that in addition to musical features
studied in Experiment 1, lyrical content of contemporary music also acts as a strong cue to a song’s formal structure. Experiments 3 and 4 revealed that listeners’ enjoyment of music is influenced both by structural features and local features of music, which were carried by the individual song sections.
The influence of structural features on music listening was most apparent over repeated hearings. In Experiment 4, listeners’ liking for contemporary music followed an inverted U-shape trend with repeated exposure, in which liking for music took a downward turn after just four repeated hearings. In contrast, liking for restructured music increased with repeated hearings and almost eliminated an initial negative effect of restructuring by the sixth hearing. In sum, our findings demonstrate that structural features as well as local features of contemporary music are salient and important to
listeners.

Semi-binary based video features for activity representation

Efficient and effective feature detection and representation is an important consideration when processing videos, and a large number of applications such as motion analysis, 3D scene understanding, tracking etc. depend on this. Amongst several feature description methods, local features are becoming increasingly popular for representing videos because of their simplicity and efficiency. While they achieve state-of-the-art performance with low computational complexity, their performance is still too limited for real world applications. Furthermore, rapid increases in the uptake of mobile devices has increased the demand for algorithms that can run with reduced memory and computational requirements. In this paper we propose a semi binary based feature detectordescriptor based on the BRISK detector, which can detect and represent videos with significantly reduced computational requirements, while achieving comparable performance to the state of the art spatio-temporal feature descriptors. First, the BRISK feature detector is applied on a frame by frame basis to detect interest points, then the detected key points are compared against consecutive frames for significant motion. Key points with significant motion are encoded with the BRISK descriptor in the spatial domain and Motion Boundary Histogram in the temporal domain. This descriptor is not only lightweight but also has lower memory requirements because of the binary nature of the BRISK descriptor, allowing the possibility of applications using hand held devices.We evaluate the combination of detectordescriptor performance in the context of action classification with a standard, popular bag-of-features with SVM framework. Experiments are carried out on two popular datasets with varying complexity and we demonstrate comparable performance with other descriptors with reduced computational complexity.

An evaluation of crowd counting methods, features and regression models

Existing crowd counting algorithms rely on holistic, local or histogram based features to capture crowd properties. Regression is then employed to estimate the crowd size. Insufficient testing across multiple datasets has made it difficult to compare and contrast different methodologies. This paper presents an evaluation across multiple datasets to compare holistic, local and histogram based methods, and to compare various image features and regression models. A K-fold cross validation protocol is followed to evaluate the performance across five public datasets: UCSD, PETS 2009, Fudan, Mall and Grand Central datasets. Image features are categorised into five types: size, shape, edges, keypoints and textures. The regression models evaluated are: Gaussian process regression (GPR), linear regression, K nearest neighbours (KNN) and neural networks (NN). The results demonstrate that local features outperform equivalent holistic and histogram based features; optimal performance is observed using all image features except for textures; and that GPR outperforms linear, KNN and NN regression

Spatial variation of climate and the impact of disturbances on local climate and forest recovery in northern Finland

The structure and function of northern ecosystems are strongly influenced by climate change and variability and by human-induced disturbances. The projected global change is likely to have a pronounced effect on the distribution and productivity of different species, generating large changes in the equilibrium at the tree-line. In turn, movement of the tree-line and the redistribution of species produce feedback to both the local and the regional climate. This research was initiated with the objective of examining the influence of natural conditions on the small-scale spatial variation of climate in Finnish Lapland, and to study the interaction and feedback mechanisms in the climate-disturbances-vegetation system near the climatological border of boreal forest. The high (1 km) resolution spatial variation of climate parameters over northern Finland was determined by applying the Kriging interpolation method that takes into account the effect of external forcing variables, i.e., geographical coordinates, elevation, sea and lake coverage. Of all the natural factors shaping the climate, the geographical position, local topography and altitude proved to be the determining ones. Spatial analyses of temperature- and precipitation-derived parameters based on a 30-year dataset (1971-2000) provide a detailed description of the local climate. Maps of the mean, maximum and minimum temperatures, the frost-free period and the growing season indicate that the most favourable thermal conditions exist in the south-western part of Lapland, around large water bodies and in the Kemijoki basin, while the coldest regions are in highland and fell Lapland. The distribution of precipitation is predominantly longitudinally dependent but with the definite influence of local features. The impact of human-induced disturbances, i.e., forest fires, on local climate and its implication for forest recovery near the northern timberline was evaluated in the Tuntsa area of eastern Lapland, damaged by a widespread forest fire in 1960 and suffering repeatedly-failed vegetation recovery since that. Direct measurements of the local climate and simulated heat and water fluxes indicated the development of a more severe climate and physical conditions on the fire-disturbed site. Removal of the original, predominantly Norway spruce and downy birch vegetation and its substitution by tundra vegetation has generated increased wind velocity and reduced snow accumulation, associated with a large variation in soil temperature and moisture and deep soil frost. The changed structural parameters of the canopy have determined changes in energy fluxes by reducing the latter over the tundra vegetation. The altered surface and soil conditions, as well as the evolved severe local climate, have negatively affected seedling growth and survival, leading to more unfavourable conditions for the reproduction of boreal vegetation and thereby causing deviations in the regional position of the timberline. However it should be noted that other factors, such as an inadequate seed source or seedbed, the poor quality of the soil and the intensive logging of damaged trees could also exacerbate the poor tree regeneration. In spite of the failed forest recovery at Tunsta, the position and composition of the timberline and tree-line in Finnish Lapland may also benefit from present and future changes in climate. The already-observed and the projected increase in temperature, the prolonged growing season, as well as changes in the precipitation regime foster tree growth and new regeneration, resulting in an advance of the timberline and tree-line northward and upward. This shift in the distribution of vegetation might be decelerated or even halted by local topoclimatic conditions and by the expected increase in the frequency of disturbances.

Um esquema de Fourier local para análise tempo-frequência de sinais não-estacionários aplicado a ruído eletroquímico

Sinais diversos estão presentes em nosso cotidiano, assim como nas medidas realizadas nas atividades de ciência e tecnologia. Dentre estes sinais, tem grande importância tecnológica aqueles associados à corrosão de estruturas metálicas. Assim, esta tese propõe o estudo de um esquema local de transformada de Fourier janelada, com a janela variando em função da curtose, aplicada a sinais de ruído eletroquímico. A curtose foi avaliada nos domínios do tempo e da frequência e processada pelo programa desenvolvido para esse fim. O esquema foi aplicado a sinais de ruído eletroquímico dos aços UNS S31600, UNS G10200 e UNS S32750 imersos em três soluções: FeCl3 0,1 mol=L (cloreto férrico), H2SO4 5%(ácido sulfúrico) e NaOH 0,1%(hidróxido de sódio). Para os aços inoxidáveis, estas soluções promovem corrosão localizada, uniforme e passivação, respectivamente. Visando testar o desempenho do esquema de Fourier desenvolvido, testes foram realizados utilizando-se inicialmente sinais sintéticos e em seguida sinais de ruído eletroquímico. Notou-se que os sinais têm características de não-estacionaridade e a maior parte da energia está presente em baixa frequência. Os intervalos de tempo e de frequência onde se concentra a maior parte da energia do sinal foram correlacionados. Para os picos máximos dos sinais de potencial e corrente obtidos de amperimetria de resistência nula, a correlação entre eles foi baixa, independente da forma de corrosão presente. Conclui-se que o método se adaptou bastante bem às características locais do sinal eletroquímico permitindo o monitoramento dos espectros tempo-frequência. O fato de ser sensível às características locais do sinal permite analisar aspectos dos sinais que do modo clássico não podem ser diretamente processados. O método da transformada de Fourier janelada variável (Variable Short-Time Fourier Transform - VSTFT) adaptou-se muito bem no monitoramento dos sinais originados de potencial de circuito aberto e amperimetria de resistência nula.