Automating condition monitoring of vegetation on railway trackbeds and embankments

Vegetation growing on railway trackbeds and embankments present potential problems. The presence of vegetation threatens the safety of personnel inspecting the railway infrastructure. In addition vegetation growth clogs the ballast and results in inadequate track drainage which in turn could lead to the collapse of the railway embankment. Assessing vegetation within the realm of railway maintenance is mainly carried out manually by making visual inspections along the track. This is done either on-site or by watching videos recorded by maintenance vehicles mainly operated by the national railway administrative body. A need for the automated detection and characterisation of vegetation on railways (a subset of vegetation control/management) has been identified in collaboration with local railway maintenance subcontractors and Trafikverket, the Swedish Transport Administration (STA). The latter is responsible for long-term planning of the transport system for all types of traffic, as well as for the building, operation and maintenance of public roads and railways. The purpose of this research project was to investigate how vegetation can be measured and quantified by human raters and how machine vision can automate the same process. Data were acquired at railway trackbeds and embankments during field measurement experiments. All field data (such as images) in this thesis work was acquired on operational, lightly trafficked railway tracks, mostly trafficked by goods trains. Data were also generated by letting (human) raters conduct visual estimates of plant cover and/or count the number of plants, either on-site or in-house by making visual estimates of the images acquired from the field experiments. Later, the degree of reliability of(human) raters’ visual estimates were investigated and compared against machine vision algorithms. The overall results of the investigations involving human raters showed inconsistency in their estimates, and are therefore unreliable. As a result of the exploration of machine vision, computational methods and algorithms enabling automatic detection and characterisation of vegetation along railways were developed. The results achieved in the current work have shown that the use of image data for detecting vegetation is indeed possible and that such results could form the base for decisions regarding vegetation control. The performance of the machine vision algorithm which quantifies the vegetation cover was able to process 98% of the im-age data. Investigations of classifying plants from images were conducted in in order to recognise the specie. The classification rate accuracy was 95%.Objective measurements such as the ones proposed in thesis offers easy access to the measurements to all the involved parties and makes the subcontracting process easier i.e., both the subcontractors and the national railway administration are given the same reference framework concerning vegetation before signing a contract, which can then be crosschecked post maintenance.A very important issue which comes with an increasing ability to recognise species is the maintenance of biological diversity. Biological diversity along the trackbeds and embankments can be mapped, and maintained, through better and robust monitoring procedures. Continuously monitoring the state of vegetation along railways is highly recommended in order to identify a need for maintenance actions, and in addition to keep track of biodiversity. The computational methods or algorithms developed form the foundation of an automatic inspection system capable of objectively supporting manual inspections, or replacing manual inspections.

Monitoring vegetation on railway embankments : supporting maintenance decisions

The national railway administrations in Scandinavia, Germany, and Austria mainly resort to manual inspections to control vegetation growth along railway embankments. Manually inspecting railways is slow and time consuming. A more worrying aspect concerns the fact that human observers are often unable to estimate the true cover of vegetation on railway embankments. Further human observers often tend to disagree with each other when more than one observer is engaged for inspection. Lack of proper techniques to identify the true cover of vegetation even result in the excess usage of herbicides; seriously harming the environment and threating the ecology. Hence work in this study has investigated aspects relevant to human variationand agreement to be able to report better inspection routines. This was studied by mainly carrying out two separate yet relevant investigations.First, thirteen observers were separately asked to estimate the vegetation cover in nine imagesacquired (in nadir view) over the railway tracks. All such estimates were compared relatively and an analysis of variance resulted in a significant difference on the observers’ cover estimates (p<0.05). Bearing in difference between the observers, a second follow-up field-study on the railway tracks was initiated and properly investigated. Two railway segments (strata) representingdifferent levels of vegetationwere carefully selected. Five sample plots (each covering an area of one-by-one meter) were randomizedfrom each stratumalong the rails from the aforementioned segments and ten images were acquired in nadir view. Further three observers (with knowledge in the railway maintenance domain) were separately asked to estimate the plant cover by visually examining theplots. Again an analysis of variance resulted in a significant difference on the observers’ cover estimates (p<0.05) confirming the result from the first investigation.The differences in observations are compared against a computer vision algorithm which detects the "true" cover of vegetation in a given image. The true cover is defined as the amount of greenish pixels in each image as detected by the computer vision algorithm. Results achieved through comparison strongly indicate that inconsistency is prevalent among the estimates reported by the observers. Hence, an automated approach reporting the use of computer vision is suggested, thus transferring the manual inspections into objective monitored inspections

Detecting Plants on Railway Embankments

This paper investigates problems concerning vegetation along railways and proposes automatic means of detecting ground vegetation. Digital images of railway embankments have been acquired and used for the purpose. The current work mainly proposes two algorithms to be able to achieve automation. Initially a vegetation detection algorithm has been investigated for the purpose of detecting vegetation. Further a rail detection algorithm that is capable of identifying the rails and eventually the valid sampling area has been investigated. Results achieved in the current work report satisfactory (qualitative) detection rates.

Utveckling av metod för att bedöma behovet av ogräsbekämpning i spår

A system for weed management on railway embankments that is both adapted to the environment and efficient in terms of resources requires knowledge and understanding about the growing conditions of vegetation so that methods to control its growth can be adapted accordingly. Automated records could complement present-day manual inspections and over time come to replace these. One challenge is to devise a method that will result in a reasonable breakdown of gathered information that can be managed rationally by affected parties and, at the same time, serve as a basis for decisions with sufficient precision. The project examined two automated methods that may be useful for the Swedish Transport Administration in the future: 1) A machine vision method, which makes use of camera sensors as a way of sensing the environment in the visible and near infrared spectrum; and 2) An N-Sensor method, which transmits light within an area that is reflected by the chlorophyll in the plants. The amount of chlorophyll provides a value that can be correlated with the biomass. The choice of technique depends on how the information is to be used. If the purpose is to form a general picture of the growth of vegetation on railway embankments as a way to plan for maintenance measures, then the N-Sensor technique may be the right choice. If the plan is to form a general picture as well as monitor and survey current and exact vegetation status on the surface over time as a way to fight specific vegetation with the correct means, then the machine vision method is the better of the two. Both techniques involve registering data using GPS positioning. In the future, it will be possible to store this information in databases that are directly accessible to stakeholders online during or in conjunction with measures to deal with the vegetation. The two techniques were compared with manual (visual) estimations as to the levels of vegetation growth. The observers (raters) visual estimation of weed coverage (%) differed statistically from person to person. In terms of estimating the frequency (number) of woody plants (trees and bushes) in the test areas, the observers were generally in agreement. The same person is often consistent in his or her estimation: it is the comparison with the estimations of others that can lead to misleading results. The system for using the information about vegetation growth requires development. The threshold for the amount of weeds that can be tolerated in different track types is an important component in such a system. The classification system must be capable of dealing with the demands placed on it so as to ensure the quality of the track and other pre-conditions such as traffic levels, conditions pertaining to track location, and the characteristics of the vegetation. The project recommends that the Swedish Transport Administration: Discusses how threshold values for the growth of vegetation on railway embankments can be determined Carries out registration of the growth of vegetation over longer and a larger number of railway sections using one or more of the methods studied in the project Introduces a system that effectively matches the information about vegetation to its position Includes information about the growth of vegetation in the records that are currently maintained of the track’s technical quality, and link the data material to other maintenance-related databases Establishes a number of representative surfaces in which weed inventories (by measuring) are regularly conducted, as a means of developing an overview of the long-term development that can serve as a basis for more precise prognoses in terms of vegetation growth Ensures that necessary opportunities for education are put in place

A machine learning approach for recognising woody plants on railway trackbeds

The purpose of this work in progress study was to test the concept of recognising plants using images acquired by image sensors in a controlled noise-free environment. The presence of vegetation on railway trackbeds and embankments presents potential problems. Woody plants (e.g. Scots pine, Norway spruce and birch) often establish themselves on railway trackbeds. This may cause problems because legal herbicides are not effective in controlling them; this is particularly the case for conifers. Thus, if maintenance administrators knew the spatial position of plants along the railway system, it may be feasible to mechanically harvest them. Primary data were collected outdoors comprising around 700 leaves and conifer seedlings from 11 species. These were then photographed in a laboratory environment. In order to classify the species in the acquired image set, a machine learning approach known as Bag-of-Features (BoF) was chosen. Irrespective of the chosen type of feature extraction and classifier, the ability to classify a previously unseen plant correctly was greater than 85%. The maintenance planning of vegetation control could be improved if plants were recognised and localised. It may be feasible to mechanically harvest them (in particular, woody plants). In addition, listed endangered species growing on the trackbeds can be avoided. Both cases are likely to reduce the amount of herbicides, which often is in the interest of public opinion. Bearing in mind that natural objects like plants are often more heterogeneous within their own class rather than outside it, the results do indeed present a stable classification performance, which is a sound prerequisite in order to later take the next step to include a natural background. Where relevant, species can also be listed under the Endangered Species Act.