Optimising container processes at multimodal seaport terminals : an integrated approach and application

A Multimodal Seaport Container Terminal (MSCT) is a complex system which requires careful planning and control in order to operate efficiently. It consists of a number of subsystems that require optimisation of the operations within them, as well as synchronisation of machines and containers between the various subsystems. Inefficiency in the terminal can delay ships from their scheduled timetables, as well as cause delays in delivering containers to their inland destinations, both of which can be very costly to their operators. The purpose of this PhD thesis is to use Operations Research methodologies to optimise and synchronise these subsystems as an integrated application. An initial model is developed for the overall MSCT; however, due to a large number of assumptions that had to be made, as well as other issues, it is found to be too inaccurate and infeasible for practical use. Instead, a method of developing models for each subsystem is proposed that then be integrated with each other. Mathematical models are developed for the Storage Area System (SAS) and Intra-terminal Transportation System (ITTS). The SAS deals with the movement and assignment of containers to stacks within the storage area, both when they arrive and when they are rehandled to retrieve containers below them. The ITTS deals with scheduling the movement of containers and machines between the storage areas and other sections of the terminal, such as the berth and road/rail terminals. Various constructive heuristics are explored and compared for these models to produce good initial solutions for large-sized problems, which are otherwise impractical to compute by exact methods. These initial solutions are further improved through the use of an innovative hyper-heuristic algorithm that integrates the SAS and ITTS solutions together and optimises them through meta-heuristic techniques. The method by which the two models can interact with each other as an integrated system will be discussed, as well as how this method can be extended to the other subsystems of the MSCT.

Optimising container transfers at multimodal terminals

The use of containers have greatly reduced handling operations at ports and at all other transfer points, thus increasing the efficiency and speed of transportation. This was done in an attempt to cut down the cost of maritime transport, mainly by reducing cargo handling and costs, and ships' time in port by speeding up handling operations. This paper discusses the major factors influencing the transfer efficiency of seaport container terminals. A network model is designed to analyse container progress in the system and applied to a seaport container terminal. The model presented here can be seen as a decision support system in the context of investment appraisal of multimodal container terminals. (C) 2000 Elsevier Science Ltd.

Some Reflections on the Emergence of Multimodal Assemblages of Meaning

In this commentary I reflect upon the conceptualisation of human meaning-making, utilised in the two target articles, that relies heavily on speech as the main mode of semiosis and considers time only in its chronological form. Instead I argue that human existence is embodied and lived through multiple modalities, and involves not only sequential experience of time, but also experience of emergence. In order to move towards a conception of meaning-making that takes this into account, I introduce the social-semiotic theory of multimodality (Kress 2010) and discuss notions of ‘real duration’ (Bergson 1907/1998) and ‘lived time’ (Martin-Vallas 2009). I argue that dialogical (idiographic) researchers need to develop analytic and methodological tools that allow exploring the emergence of multimodal assemblages of meaning in addition to trying to avoid the monologisation of complex dynamic dialogical phenomena.

Study protocol of the YOU CALL - WE CALL TRIAL: Impact of a multimodal support intervention after a "mild" stroke

Background More than 60% of new strokes each year are "mild" in severity and this proportion is expected to rise in the years to come. Within our current health care system those with "mild" stroke are typically discharged home within days, without further referral to health or rehabilitation services other than advice to see their family physician. Those with mild stroke often have limited access to support from health professionals with stroke-specific knowledge who would typically provide critical information on topics such as secondary stroke prevention, community reintegration, medication counselling and problem solving with regard to specific concerns that arise. Isolation and lack of knowledge may lead to a worsening of health problems including stroke recurrence and unnecessary and costly health care utilization. The purpose of this study is to assess the effectiveness, for individuals who experience a first "mild" stroke, of a sustainable, low cost, multimodal support intervention (comprising information, education and telephone support) - "WE CALL" compared to a passive intervention (providing the name and phone number of a resource person available if they feel the need to) - "YOU CALL", on two primary outcomes: unplanned-use of health services for negative events and quality of life. Method/Design We will recruit 384 adults who meet inclusion criteria for a first mild stroke across six Canadian sites. Baseline measures will be taken within the first month after stroke onset. Participants will be stratified according to comorbidity level and randomised to one of two groups: YOU CALL or WE CALL. Both interventions will be offered over a six months period. Primary outcomes include unplanned use of heath services for negative event (frequency calendar) and quality of life (EQ-5D and Quality of Life Index). Secondary outcomes include participation level (LIFE-H), depression (Beck Depression Inventory II) and use of health services for health promotion or prevention (frequency calendar). Blind assessors will gather data at mid-intervention, end of intervention and one year follow up. Discussion If effective, this multimodal intervention could be delivered in both urban and rural environments. For example, existing infrastructure such as regional stroke centers and existing secondary stroke prevention clinics, make this intervention, if effective, deliverable and sustainable.

Multimodal therapy for complete regression of malignant melanoma using constrained nonlinear optimal dynamic inversion

Using a realistic nonlinear mathematical model for melanoma dynamics and the technique of optimal dynamic inversion (exact feedback linearization with static optimization), a multimodal automatic drug dosage strategy is proposed in this paper for complete regression of melanoma cancer in humans. The proposed strategy computes different drug dosages and gives a nonlinear state feedback solution for driving the number of cancer cells to zero. However, it is observed that when tumor is regressed to certain value, then there is no need of external drug dosages as immune system and other therapeutic states are able to regress tumor at a sufficiently fast rate which is more than exponential rate. As model has three different drug dosages, after applying dynamic inversion philosophy, drug dosages can be selected in optimized manner without crossing their toxicity limits. The combination of drug dosages is decided by appropriately selecting the control design parameter values based on physical constraints. The process is automated for all possible combinations of the chemotherapy and immunotherapy drug dosages with preferential emphasis of having maximum possible variety of drug inputs at any given point of time. Simulation study with a standard patient model shows that tumor cells are regressed from 2 x 107 to order of 105 cells because of external drug dosages in 36.93 days. After this no external drug dosages are required as immune system and other therapeutic states are able to regress tumor at greater than exponential rate and hence, tumor goes to zero (less than 0.01) in 48.77 days and healthy immune system of the patient is restored. Study with different chemotherapy drug resistance value is also carried out. (C) 2014 Elsevier Ltd. All rights reserved.

A novel acoustic-vibratory multimodal duet

The communication strategy of most crickets and bushcrickets typically consists of males broadcasting loud acoustic calling songs, while females perform phonotaxis, moving towards the source of the call. Males of the pseudophylline bushcricket species Onomarchus uninotatus produce an unusually low-pitched call, and we found that the immediate and most robust response of females to the male acoustic call was a bodily vibration, or tremulation, following each syllable of the call. We hypothesized that these bodily oscillations might send out a vibrational signal along the substrate on which the female stands, which males could use to localize her position. We quantified these vibrational signals using a laser vibrometer and found a clear phase relationship of alternation between the chirps of the male acoustic call and the female vibrational response. This system therefore constitutes a novel multimodal duet with a reliable temporal structure. We also found that males could localize the source of vibration but only if both the acoustic and vibratory components of the duet were played back. This unique multimodal duetting system may have evolved in response to higher levels of bat predation on searching bushcricket females than calling males, shifting part of the risk associated with partner localization onto the male. This is the first known example of bushcricket female tremulation in response to a long-range male acoustic signal and the first known example of a multimodal duet among animals.

Microscopia multimodal prática: registro automático de imagens de microscopia ótica e de microscopia eletrônica de varredura

A discriminação de fases que são praticamente indistinguíveis ao microscópio ótico de luz refletida ou ao microscópio eletrônico de varredura (MEV) é um dos problemas clássicos da microscopia de minérios. Com o objetivo de resolver este problema vem sendo recentemente empregada a técnica de microscopia colocalizada, que consiste na junção de duas modalidades de microscopia, microscopia ótica e microscopia eletrônica de varredura. O objetivo da técnica é fornecer uma imagem de microscopia multimodal, tornando possível a identificação, em amostras de minerais, de fases que não seriam distinguíveis com o uso de uma única modalidade, superando assim as limitações individuais dos dois sistemas. O método de registro até então disponível na literatura para a fusão das imagens de microscopia ótica e de microscopia eletrônica de varredura é um procedimento trabalhoso e extremamente dependente da interação do operador, uma vez que envolve a calibração do sistema com uma malha padrão a cada rotina de aquisição de imagens. Por esse motivo a técnica existente não é prática. Este trabalho propõe uma metodologia para automatizar o processo de registro de imagens de microscopia ótica e de microscopia eletrônica de varredura de maneira a aperfeiçoar e simplificar o uso da técnica de microscopia colocalizada. O método proposto pode ser subdividido em dois procedimentos: obtenção da transformação e registro das imagens com uso desta transformação. A obtenção da transformação envolve, primeiramente, o pré-processamento dos pares de forma a executar um registro grosseiro entre as imagens de cada par. Em seguida, são obtidos pontos homólogos, nas imagens óticas e de MEV. Para tal, foram utilizados dois métodos, o primeiro desenvolvido com base no algoritmo SIFT e o segundo definido a partir da varredura pelo máximo valor do coeficiente de correlação. Na etapa seguinte é calculada a transformação. Foram empregadas duas abordagens distintas: a média ponderada local (LWM) e os mínimos quadrados ponderados com polinômios ortogonais (MQPPO). O LWM recebe como entradas os chamados pseudo-homólogos, pontos que são forçadamente distribuídos de forma regular na imagem de referência, e que revelam, na imagem a ser registrada, os deslocamentos locais relativos entre as imagens. Tais pseudo-homólogos podem ser obtidos tanto pelo SIFT como pelo método do coeficiente de correlação. Por outro lado, o MQPPO recebe um conjunto de pontos com a distribuição natural. A análise dos registro de imagens obtidos empregou como métrica o valor da correlação entre as imagens obtidas. Observou-se que com o uso das variantes propostas SIFT-LWM e SIFT-Correlação foram obtidos resultados ligeiramente superiores aos do método com a malha padrão e LWM. Assim, a proposta, além de reduzir drasticamente a intervenção do operador, ainda possibilitou resultados mais precisos. Por outro lado, o método baseado na transformação fornecida pelos mínimos quadrados ponderados com polinômios ortogonais mostrou resultados inferiores aos produzidos pelo método que faz uso da malha padrão.

Navite: A Neural Network System For Sensory-Based Robot Navigation

A neural network system, NAVITE, for incremental trajectory generation and obstacle avoidance is presented. Unlike other approaches, the system is effective in unstructured environments. Multimodal inforrnation from visual and range data is used for obstacle detection and to eliminate uncertainty in the measurements. Optimal paths are computed without explicitly optimizing cost functions, therefore reducing computational expenses. Simulations of a planar mobile robot (including the dynamic characteristics of the plant) in obstacle-free and object avoidance trajectories are presented. The system can be extended to incorporate global map information into the local decision-making process.

Developing Single-Laser Sources for Multimodal Coherent Anti-Stokes Raman Scattering Microscopy

Coherent anti-Stokes Raman scattering (CARS) microscopy has developed rapidly and is opening the door to new types of experiments. This work describes the development of new laser sources for CARS microscopy and their use for different applications. It is specifically focused on multimodal nonlinear optical microscopy—the simultaneous combination of different imaging techniques. This allows us to address a diverse range of applications, such as the study of biomaterials, fluid inclusions, atherosclerosis, hepatitis C infection in cells, and ice formation in cells. For these applications new laser sources are developed that allow for practical multimodal imaging. For example, it is shown that using a single Ti:sapphire oscillator with a photonic crystal fiber, it is possible to develop a versatile multimodal imaging system using optimally chirped laser pulses. This system can perform simultaneous two photon excited fluorescence, second harmonic generation, and CARS microscopy. The versatility of the system is further demonstrated by showing that it is possible to probe different Raman modes using CARS microscopy simply by changing a time delay between the excitation beams. Using optimally chirped pulses also enables further simplification of the laser system required by using a single fiber laser combined with nonlinear optical fibers to perform effective multimodal imaging. While these sources are useful for practical multimodal imaging, it is believed that for further improvements in CARS microscopy sensitivity, new excitation schemes are necessary. This has led to the design of a new, high power, extended cavity oscillator that should be capable of implementing new excitation schemes for CARS microscopy as well as other techniques. Our interest in multimodal imaging has led us to other areas of research as well. For example, a fiber-coupling scheme for signal collection in the forward direction is demonstrated that allows for fluorescence lifetime imaging without significant temporal distortion. Also highlighted is an imaging artifact that is unique to CARS microscopy that can alter image interpretation, especially when using multimodal imaging. By combining expertise in nonlinear optics, laser development, fiber optics, and microscopy, we have developed systems and techniques that will be of benefit for multimodal CARS microscopy.

Multimodal Biometric Human Recognition for Perceptual Human-Computer Interaction

In this paper, a novel video-based multimodal biometric verification scheme using the subspace-based low-level feature fusion of face and speech is developed for specific speaker recognition for perceptual human--computer interaction (HCI). In the proposed scheme, human face is tracked and face pose is estimated to weight the detected facelike regions in successive frames, where ill-posed faces and false-positive detections are assigned with lower credit to enhance the accuracy. In the audio modality, mel-frequency cepstral coefficients are extracted for voice-based biometric verification. In the fusion step, features from both modalities are projected into nonlinear Laplacian Eigenmap subspace for multimodal speaker recognition and combined at low level. The proposed approach is tested on the video database of ten human subjects, and the results show that the proposed scheme can attain better accuracy in comparison with the conventional multimodal fusion using latent semantic analysis as well as the single-modality verifications. The experiment on MATLAB shows the potential of the proposed scheme to attain the real-time performance for perceptual HCI applications.

The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent

SEMAINE has created a large audiovisual database as a part of an iterative approach to building Sensitive Artificial Listener (SAL) agents that can engage a person in a sustained, emotionally colored conversation. Data used to build the agents came from interactions between users and an operator simulating a SAL agent, in different configurations: Solid SAL (designed so that operators displayed an appropriate nonverbal behavior) and Semi-automatic SAL (designed so that users' experience approximated interacting with a machine). We then recorded user interactions with the developed system, Automatic SAL, comparing the most communicatively competent version to versions with reduced nonverbal skills. High quality recording was provided by five high-resolution, high-framerate cameras, and four microphones, recorded synchronously. Recordings total 150 participants, for a total of 959 conversations with individual SAL characters, lasting approximately 5 minutes each. Solid SAL recordings are transcribed and extensively annotated: 6-8 raters per clip traced five affective dimensions and 27 associated categories. Other scenarios are labeled on the same pattern, but less fully. Additional information includes FACS annotation on selected extracts, identification of laughs, nods, and shakes, and measures of user engagement with the automatic system. The material is available through a web-accessible database. © 2010-2012 IEEE.

Robust Multimodal Person Identification With Limited Training Data

This paper presents a novel method of audio-visual feature-level fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there are limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature representation and a modified cosine similarity are introduced to combine and compare bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal dataset created from the SPIDRE speaker recognition database and AR face recognition database with variable noise corruption of speech and occlusion in the face images. The system's speaker identification performance on the SPIDRE database, and facial identification performance on the AR database, is comparable with the literature. Combining both modalities using the new method of multimodal fusion leads to significantly improved accuracy over the unimodal systems, even when both modalities have been corrupted. The new method also shows improved identification accuracy compared with the bimodal systems based on multicondition model training or missing-feature decoding alone.

Cascaded multimodal biometric recognition framework

A practically viable multi-biometric recognition system should not only be stable, robust and accurate but should also adhere to real-time processing speed and memory constraints. This study proposes a cascaded classifier-based framework for use in biometric recognition systems. The proposed framework utilises a set of weak classifiers to reduce the enrolled users' dataset to a small list of candidate users. This list is then used by a strong classifier set as the final stage of the cascade to formulate the decision. At each stage, the candidate list is generated by a Mahalanobis distance-based match score quality measure. One of the key features of the authors framework is that each classifier in the ensemble can be designed to use a different modality thus providing the advantages of a truly multimodal biometric recognition system. In addition, it is one of the first truly multimodal cascaded classifier-based approaches for biometric recognition. The performance of the proposed system is evaluated both for single and multimodalities to demonstrate the effectiveness of the approach.

A biosignal embedded system for physiological computing

Thesis submitted in the fulfilment of the requirements for the Degree of Master in Electronic and Telecomunications Engineering

Multimodal image fusion of anatomical structures for diagnosis, therapy planning and assistance

This paper provides an overview of work done in recent years by our research group to fuse multimodal images of the trunk of patients with Adolescent Idiopathic Scoliosis (AIS) treated at Sainte-Justine University Hospital Center (CHU). We first describe our surface acquisition system and introduce a set of clinical measurements (indices) based on the trunk's external shape, to quantify its degree of asymmetry. We then describe our 3D reconstruction system of the spine and rib cage from biplanar radiographs and present our methodology for multimodal fusion of MRI, X-ray and external surface images of the trunk We finally present a physical model of the human trunk including bone and soft tissue for the simulation of the surgical outcome on the external trunk shape in AIS.