Automatic tracking, super-resolution and recognition of human faces from surveillance video

Identifying an individual from surveillance video is a difficult, time consuming and labour intensive process. The proposed system aims to streamline this process by filtering out unwanted scenes and enhancing an individual's face through super-resolution. An automatic face recognition system is then used to identify the subject or present the human operator with likely matches from a database. A person tracker is used to speed up the subject detection and super-resolution process by tracking moving subjects and cropping a region of interest around the subject's face to reduce the number and size of the image frames to be super-resolved respectively. In this paper, experiments have been conducted to demonstrate how the optical flow super-resolution method used improves surveillance imagery for visual inspection as well as automatic face recognition on an Eigenface and Elastic Bunch Graph Matching system. The optical flow based method has also been benchmarked against the ``hallucination'' algorithm, interpolation methods and the original low-resolution images. Results show that both super-resolution algorithms improved recognition rates significantly. Although the hallucination method resulted in slightly higher recognition rates, the optical flow method produced less artifacts and more visually correct images suitable for human consumption.

The impact of presbyopic spectacles and contact lenses on driving performance

Presbyopia affects individuals from the age of 45 years onwards, resulting in difficulty in accurately focusing on near objects. There are many optical corrections available including spectacles or contact lenses that are designed to enable presbyopes to see clearly at both far and near distances. However, presbyopic vision corrections also disturb aspects of visual function under certain circumstances. The impact of these changes on activities of daily living such as driving are, however, poorly understood. Therefore, the aim of this study was to determine which aspects of driving performance might be affected by wearing different types of presbyopic vision corrections. In order to achieve this aim, three experiments were undertaken. The first experiment involved administration of a questionnaire to compare the subjective driving difficulties experienced when wearing a range of common presbyopic contact lens and spectacle corrections. The questionnaire was developed and piloted, and included a series of items regarding difficulties experienced while driving under day and night-time conditions. Two hundred and fifty five presbyopic patients responded to the questionnaire and were categorised into five groups, including those wearing no vision correction for driving (n = 50), bifocal spectacles (BIF, n = 54), progressive addition lenses spectacles (PAL, n = 50), monovision (MV, n = 53) and multifocal contact lenses (MTF CL, n = 48). Overall, ratings of satisfaction during daytime driving were relatively high for all correction types. However, MV and MTF CL wearers were significantly less satisfied with aspects of their vision during night-time than daytime driving, particularly with regard to disturbances from glare and haloes. Progressive addition lens wearers noticed more distortion of peripheral vision, while BIF wearers reported more difficulties with tasks requiring changes in focus and those who wore no vision correction for driving reported problems with intermediate and near tasks. Overall, the mean level of satisfaction for daytime driving was quite high for all of the groups (over 80%), with the BIF wearers being the least satisfied with their vision for driving. Conversely, at night, MTF CL wearers expressed the least satisfaction. Research into eye and head movements has become increasingly of interest in driving research as it provides a means of understanding how the driver responds to visual stimuli in traffic. Previous studies have found that wearing PAL can affect eye and head movement performance resulting in slower eye movement velocities and longer times to stabilize the gaze for fixation. These changes in eye and head movement patterns may have implications for driving safety, given that the visual tasks for driving include a range of dynamic search tasks. Therefore, the second study was designed to investigate the influence of different presbyopic corrections on driving-related eye and head movements under standardized laboratory-based conditions. Twenty presbyopes (mean age: 56.1 ± 5.7 years) who had no experience of wearing presbyopic vision corrections, apart from single vision reading spectacles, were recruited. Each participant wore five different types of vision correction: single vision distance lenses (SV), PAL, BIF, MV and MTF CL. For each visual condition, participants were required to view videotape recordings of traffic scenes, track a reference vehicle and identify a series of peripherally presented targets while their eye and head movements were recorded using the faceLAB® eye and head tracking system. Digital numerical display panels were also included as near visual stimuli (simulating the visual displays of a vehicle speedometer and radio). The results demonstrated that the path length of eye movements while viewing and responding to driving-related traffic scenes was significantly longer when wearing BIF and PAL than MV and MTF CL. The path length of head movements was greater with SV, BIF and PAL than MV and MTF CL. Target recognition was less accurate when the near stimulus was located at eccentricities inferiorly and to the left, rather than directly below the primary position of gaze, regardless of vision correction type. The third experiment aimed to investigate the real world driving performance of presbyopes while wearing different vision corrections measured on a closed-road circuit at night-time. Eye movements were recorded using the ASL Mobile Eye, eye tracking system (as the faceLAB® system proved to be impractical for use outside of the laboratory). Eleven participants (mean age: 57.25 ± 5.78 years) were fitted with four types of prescribed vision corrections (SV, PAL, MV and MTF CL). The measures of driving performance on the closed-road circuit included distance to sign recognition, near target recognition, peripheral light-emitting-diode (LED) recognition, low contrast road hazards recognition and avoidance, recognition of all the road signs, time to complete the course, and driving behaviours such as braking, accelerating, and cornering. The results demonstrated that driving performance at night was most affected by MTF CL compared to PAL, resulting in shorter distances to read signs, slower driving speeds, and longer times spent fixating road signs. Monovision resulted in worse performance in the task of distance to read a signs compared to SV and PAL. The SV condition resulted in significantly more errors made in interpreting information from in-vehicle devices, despite spending longer time fixating on these devices. Progressive addition lenses were ranked as the most preferred vision correction, while MTF CL were the least preferred vision correction for night-time driving. This thesis addressed the research question of how presbyopic vision corrections affect driving performance and the results of the three experiments demonstrated that the different types of presbyopic vision corrections (e.g. BIF, PAL, MV and MTF CL) can affect driving performance in different ways. Distance-related driving tasks showed reduced performance with MV and MTF CL, while tasks which involved viewing in-vehicle devices were significantly hampered by wearing SV corrections. Wearing spectacles such as SV, BIF and PAL induced greater eye and head movements in the simulated driving condition, however this did not directly translate to impaired performance on the closed- road circuit tasks. These findings are important for understanding the influence of presbyopic vision corrections on vision under real world driving conditions. They will also assist the eye care practitioner to understand and convey to patients the potential driving difficulties associated with wearing certain types of presbyopic vision corrections and accordingly to support them in the process of matching patients to optical corrections which meet their visual needs.

Fast and robust stereo object recognition for spheres

The paper presents a fast and robust stereo object recognition method. The method is currently unable to identify the rotation of objects. This makes it very good at locating spheres which are rotationally independent. Approximate methods for located non-spherical objects have been developed. Fundamental to the method is that the correspondence problem is solved using information about the dimensions of the object being located. This is in contrast to previous stereo object recognition systems where the scene is first reconstructed by point matching techniques. The method is suitable for real-time application on low-power devices.

Speech recognition using ad-hoc microphone arrays

While close talking microphones give the best signal quality and produce the highest accuracy from current Automatic Speech Recognition (ASR) systems, the speech signal enhanced by microphone array has been shown to be an effective alternative in a noisy environment. The use of microphone arrays in contrast to close talking microphones alleviates the feeling of discomfort and distraction to the user. For this reason, microphone arrays are popular and have been used in a wide range of applications such as teleconferencing, hearing aids, speaker tracking, and as the front-end to speech recognition systems. With advances in sensor and sensor network technology, there is considerable potential for applications that employ ad-hoc networks of microphone-equipped devices collaboratively as a virtual microphone array. By allowing such devices to be distributed throughout the users’ environment, the microphone positions are no longer constrained to traditional fixed geometrical arrangements. This flexibility in the means of data acquisition allows different audio scenes to be captured to give a complete picture of the working environment. In such ad-hoc deployment of microphone sensors, however, the lack of information about the location of devices and active speakers poses technical challenges for array signal processing algorithms which must be addressed to allow deployment in real-world applications. While not an ad-hoc sensor network, conditions approaching this have in effect been imposed in recent National Institute of Standards and Technology (NIST) ASR evaluations on distant microphone recordings of meetings. The NIST evaluation data comes from multiple sites, each with different and often loosely specified distant microphone configurations. This research investigates how microphone array methods can be applied for ad-hoc microphone arrays. A particular focus is on devising methods that are robust to unknown microphone placements in order to improve the overall speech quality and recognition performance provided by the beamforming algorithms. In ad-hoc situations, microphone positions and likely source locations are not known and beamforming must be achieved blindly. There are two general approaches that can be employed to blindly estimate the steering vector for beamforming. The first is direct estimation without regard to the microphone and source locations. An alternative approach is instead to first determine the unknown microphone positions through array calibration methods and then to use the traditional geometrical formulation for the steering vector. Following these two major approaches investigated in this thesis, a novel clustered approach which includes clustering the microphones and selecting the clusters based on their proximity to the speaker is proposed. Novel experiments are conducted to demonstrate that the proposed method to automatically select clusters of microphones (ie, a subarray), closely located both to each other and to the desired speech source, may in fact provide a more robust speech enhancement and recognition than the full array could.

Automated object identification using optical video cameras on construction sites

Visual recording devices such as video cameras, CCTVs, or webcams have been broadly used to facilitate work progress or safety monitoring on construction sites. Without human intervention, however, both real-time reasoning about captured scenes and interpretation of recorded images are challenging tasks. This article presents an exploratory method for automated object identification using standard video cameras on construction sites. The proposed method supports real-time detection and classification of mobile heavy equipment and workers. The background subtraction algorithm extracts motion pixels from an image sequence, the pixels are then grouped into regions to represent moving objects, and finally the regions are identified as a certain object using classifiers. For evaluating the method, the formulated computer-aided process was implemented on actual construction sites, and promising results were obtained. This article is expected to contribute to future applications of automated monitoring systems of work zone safety or productivity.

Optical non-destructive evaluation of articular cartilage integrity : a review

This paper reviews the current status of the application of optical non-destructive methods, particularly infrared (IR) and near infrared (NIR), in the evaluation of the physiological integrity of articular cartilage. It is concluded that a significant amount of work is still required in order to achieve specificity and clinical applicability of these methods in the assessment and treatment of dysfunctional articular joints.

Super-resolved faces for improved face recognition from surveillance video advances in biometrics

Characteristics of surveillance video generally include low resolution and poor quality due to environmental, storage and processing limitations. It is extremely difficult for computers and human operators to identify individuals from these videos. To overcome this problem, super-resolution can be used in conjunction with an automated face recognition system to enhance the spatial resolution of video frames containing the subject and narrow down the number of manual verifications performed by the human operator by presenting a list of most likely candidates from the database. As the super-resolution reconstruction process is ill-posed, visual artifacts are often generated as a result. These artifacts can be visually distracting to humans and/or affect machine recognition algorithms. While it is intuitive that higher resolution should lead to improved recognition accuracy, the effects of super-resolution and such artifacts on face recognition performance have not been systematically studied. This paper aims to address this gap while illustrating that super-resolution allows more accurate identification of individuals from low-resolution surveillance footage. The proposed optical flow-based super-resolution method is benchmarked against Baker et al.’s hallucination and Schultz et al.’s super-resolution techniques on images from the Terrascope and XM2VTS databases. Ground truth and interpolated images were also tested to provide a baseline for comparison. Results show that a suitable super-resolution system can improve the discriminability of surveillance video and enhance face recognition accuracy. The experiments also show that Schultz et al.’s method fails when dealing surveillance footage due to its assumption of rigid objects in the scene. The hallucination and optical flow-based methods performed comparably, with the optical flow-based method producing less visually distracting artifacts that interfered with human recognition.

Automated recognition of drunk driving on highways from video sequences

A new method for the detection of abnormal vehicle trajectories is proposed. It couples optical flow extraction of vehicle velocities with a neural network classifier. Abnormal trajectories are indicative of drunk or sleepy drivers. A single feature of the vehicle, eg., a tail light, is isolated and the optical flow computed only around this feature rather than at each pixel in the image.

Optical and microscopic study of a polymer-nanotube mixture for organic photovoltaic applications

Organic solar cells based on bulk heterojunction between a conductive polymer and a carbon nanostructure offer potential advantages compared to conventional inorganic cells. Low cost, light weight, flexibility and high peak power per unit weight are all features that can be considered a reality for organic photovoltaics. Although polymer/carbon nanotubes solar cells have been proposed, only low power conversion efficiencies have been reached without addressing the mechanisms responsible for this poor performance. The purpose of this work is therefore to investigate the basic interaction between carbon nanotubes and poly(3-hexylthiophene) in order to demonstrate how this interaction affects the performance of photovoltaic devices. The outcomes of this study are the contributions made to the knowledge of the phenomena explaining the behaviour of electronic devices based on carbon nanotubes and poly(3-hexylthiophene). In this PhD, polymer thin films with the inclusion of uniformly distributed carbon nanotubes were deposited from solution and characterised. The bulk properties of the composites were studied with microscopy and spectroscopy techniques to provide evidence of higher degrees of polymer order when interacting with carbon nanotubes. Although bulk investigation techniques provided useful information about the interaction between the polymer and the nanotubes, clear evidence of the phenomena affecting the heterojunction formed between the two species was investigated at nanoscale. Identifying chirality-driven polymer assisted assembly on the carbon nanotube surface was one of the major achievements of this study. Moreover, the analysis of the electrical behaviour of the heterojunction between the polymer and the nanotube highlighted the charge transfer responsible for the low performance of photovoltaic devices. Polymer and carbon nanotube composite-based devices were fabricated and characterised in order to study their electronic properties. The carbon nanotube introduction in the polymer matrix evidenced a strong electrical conductivity enhancement but also a lower photoconductivity response. Moreover, the extension of pristine polymer device characterisation models to composites based devices evidenced the conduction mechanisms related to nanotubes. Finally, the introduction of carbon nanotubes in the polymer matrix was demonstrated to improve the pristine polymer solar cell performance and the spectral response even though the power conversion efficiency is still too low.

The backfilled GEI : a cross-capture modality gait feature for frontal and side-view gait recognition

In this paper, we propose a novel direction for gait recognition research by proposing a new capture-modality independent, appearance-based feature which we call the Back-filled Gait Energy Image (BGEI). It can can be constructed from both frontal depth images, as well as the more commonly used side-view silhouettes, allowing the feature to be applied across these two differing capturing systems using the same enrolled database. To evaluate this new feature, a frontally captured depth-based gait dataset was created containing 37 unique subjects, a subset of which also contained sequences captured from the side. The results demonstrate that the BGEI can effectively be used to identify subjects through their gait across these two differing input devices, achieving rank-1 match rate of 100%, in our experiments. We also compare the BGEI against the GEI and GEV in their respective domains, using the CASIA dataset and our depth dataset, showing that it compares favourably against them. The experiments conducted were performed using a sparse representation based classifier with a locally discriminating input feature space, which show significant improvement in performance over other classifiers used in gait recognition literature, achieving state of the art results with the GEI on the CASIA dataset.

Structure character in small-carbon-cluster deposition on diamond surface

Experimentally, hydrogen-free diamond-like carbon (DLC) films were assembled by means of pulsed laser deposition (PLD), where energetic small-carbon-clusters were deposited on the substrate. In this paper, the chemisorption of energetic C2 and C10 clusters on diamond (001)-( 2×1) surface was investigated by molecular dynamics simulation. The influence of cluster size and the impact energy on the structure character of the deposited clusters is mainly addressed. The impact energy was varied from a few tens eV to 100 eV. The chemisorption of C10 was found to occur only when its incident energy is above a threshold value ( E th). While, the C2 cluster was easily to adsorb on the surface even at much lower incident energy. With increasing the impact energy, the structures of the deposited C2 and C10 are different from the free clusters. Finally, the growth of films synthesized by energetic C2 and C10 clusters were simulated. The statistics indicate the C2 cluster has high probability of adsorption and films assembled of C2 present slightly higher SP3 fraction than that of C10-films, especially at higher impact energy and lower substrate temperature. Our result supports the experimental findings. Moreover, the simulation underlines the deposition mechanism at atomic scale.

High performance electrochromic devices based on anodized nanoporous Nb2O5

Despite the predictions, the true potential of Nb2O5 for electrochromic applications has yet to be fully realized. In this work, three-dimensional (3D) compact and well-ordered nanoporous Nb2O5 films are synthesized by the electrochemical anodization of niobium thin films. These films are formed using RF sputtering and then anodized in an electrolyte containing ethylene glycol, ammonium fluoride, and small water content (4%) at 50 °C which resulted in low embedded impurities within the structure. Characterization of the anodized films shows that a highly crystalline orthorhombic phase of Nb2O5 is obtained after annealing at 450 °C. The 3D structure provides a template consisting of a large concentration of active sites for ion intercalation, while also ensuring low scattering directional paths for electrons. These features enhance the coloration efficiency to 47.0 cm2 C?1 (at 550 nm) for a 500 nm thick film upon Li+ ion intercalation. Additionally, the Nb2O5 electrochromic device shows a high bleached state transparency and large optical modulation.

Optical coupling of Au nanoparticles on vertical graphenes to maximize SERS response

Gold particle interaction with few-layer graphenes is of interest for the development of numerous optical nanodevices. The results of numerical studies of the coupling of gold nanoparticles with few-layer vertical graphene sheets are presented. The field strengths are computed and the optimum nanoparticle configurations for the formation of SERS hotpots are obtained. The nanoparticles are modeled as 8 nm diameter spheres atop 1.5 nm (5 layers) graphene sheet. The vertical orientation is of particular interest as it is possible to use both sides of the graphene structure and potentially double the number of particles in the system. Our results show that with the addition of an opposing particle a much stronger signal can be obtained as well as the particle separation can be controlled by the number of atomic carbon layers. These results provide further insights and contribute to the development of next-generation plasmonic devices based on nanostructures with hybrid dimensionality.

Inpatient facial expression recognition with SMS alert

Facial expression recognition (FER) has been dramatically developed in recent years, thanks to the advancements in related fields, especially machine learning, image processing and human recognition. Accordingly, the impact and potential usage of automatic FER have been growing in a wide range of applications, including human-computer interaction, robot control and driver state surveillance. However, to date, robust recognition of facial expressions from images and videos is still a challenging task due to the difficulty in accurately extracting the useful emotional features. These features are often represented in different forms, such as static, dynamic, point-based geometric or region-based appearance. Facial movement features, which include feature position and shape changes, are generally caused by the movements of facial elements and muscles during the course of emotional expression. The facial elements, especially key elements, will constantly change their positions when subjects are expressing emotions. As a consequence, the same feature in different images usually has different positions. In some cases, the shape of the feature may also be distorted due to the subtle facial muscle movements. Therefore, for any feature representing a certain emotion, the geometric-based position and appearance-based shape normally changes from one image to another image in image databases, as well as in videos. This kind of movement features represents a rich pool of both static and dynamic characteristics of expressions, which playa critical role for FER. The vast majority of the past work on FER does not take the dynamics of facial expressions into account. Some efforts have been made on capturing and utilizing facial movement features, and almost all of them are static based. These efforts try to adopt either geometric features of the tracked facial points, or appearance difference between holistic facial regions in consequent frames or texture and motion changes in loca- facial regions. Although achieved promising results, these approaches often require accurate location and tracking of facial points, which remains problematic.

Plasma-aided hydrogenation and Al-doping : increasing the conductivity and optical transparency of ZnO transparent conducting oxide

Plasma-assisted magnetron sputtering with varying ambient conditions has been utilised to deposit Al-doped ZnO (AZO) transparent conductive thin films directly onto a glass substrate at a low substrate temperature of 400 °C. The effects of hydrogen addition on electrical, optical and structural properties of the deposited AZO films have been investigated using X-ray diffractometry (XRD), scanning electron microscopy (SEM), Hall effect measurements and UV–vis optical transmission spectroscopy. The results indicate that hydrogen addition has a remarkable effect on the film transparency and conductivity with the greatest effects observed with a hydrogen flux of approximately 3 sccm. It has been demonstrated that the conductivity and the average transmittance in the visible range can increase simultaneously contrary to the effects observed by other authors. In addition, hydrogen incorporation further leads to the absorption edge shifting to a shorter wavelength due to the Burstein–Moss effect. These results are of particular relevance to the development of the next generation of optoelectronic and photovoltaic devices based on highly transparent conducting oxides with controllable electronic and optical properties.