Automating condition monitoring of vegetation on railway trackbeds and embankments

Vegetation growing on railway trackbeds and embankments present potential problems. The presence of vegetation threatens the safety of personnel inspecting the railway infrastructure. In addition vegetation growth clogs the ballast and results in inadequate track drainage which in turn could lead to the collapse of the railway embankment. Assessing vegetation within the realm of railway maintenance is mainly carried out manually by making visual inspections along the track. This is done either on-site or by watching videos recorded by maintenance vehicles mainly operated by the national railway administrative body. A need for the automated detection and characterisation of vegetation on railways (a subset of vegetation control/management) has been identified in collaboration with local railway maintenance subcontractors and Trafikverket, the Swedish Transport Administration (STA). The latter is responsible for long-term planning of the transport system for all types of traffic, as well as for the building, operation and maintenance of public roads and railways. The purpose of this research project was to investigate how vegetation can be measured and quantified by human raters and how machine vision can automate the same process. Data were acquired at railway trackbeds and embankments during field measurement experiments. All field data (such as images) in this thesis work was acquired on operational, lightly trafficked railway tracks, mostly trafficked by goods trains. Data were also generated by letting (human) raters conduct visual estimates of plant cover and/or count the number of plants, either on-site or in-house by making visual estimates of the images acquired from the field experiments. Later, the degree of reliability of(human) raters’ visual estimates were investigated and compared against machine vision algorithms. The overall results of the investigations involving human raters showed inconsistency in their estimates, and are therefore unreliable. As a result of the exploration of machine vision, computational methods and algorithms enabling automatic detection and characterisation of vegetation along railways were developed. The results achieved in the current work have shown that the use of image data for detecting vegetation is indeed possible and that such results could form the base for decisions regarding vegetation control. The performance of the machine vision algorithm which quantifies the vegetation cover was able to process 98% of the im-age data. Investigations of classifying plants from images were conducted in in order to recognise the specie. The classification rate accuracy was 95%.Objective measurements such as the ones proposed in thesis offers easy access to the measurements to all the involved parties and makes the subcontracting process easier i.e., both the subcontractors and the national railway administration are given the same reference framework concerning vegetation before signing a contract, which can then be crosschecked post maintenance.A very important issue which comes with an increasing ability to recognise species is the maintenance of biological diversity. Biological diversity along the trackbeds and embankments can be mapped, and maintained, through better and robust monitoring procedures. Continuously monitoring the state of vegetation along railways is highly recommended in order to identify a need for maintenance actions, and in addition to keep track of biodiversity. The computational methods or algorithms developed form the foundation of an automatic inspection system capable of objectively supporting manual inspections, or replacing manual inspections.

Utveckling av metod för att bedöma behovet av ogräsbekämpning i spår

A system for weed management on railway embankments that is both adapted to the environment and efficient in terms of resources requires knowledge and understanding about the growing conditions of vegetation so that methods to control its growth can be adapted accordingly. Automated records could complement present-day manual inspections and over time come to replace these. One challenge is to devise a method that will result in a reasonable breakdown of gathered information that can be managed rationally by affected parties and, at the same time, serve as a basis for decisions with sufficient precision. The project examined two automated methods that may be useful for the Swedish Transport Administration in the future: 1) A machine vision method, which makes use of camera sensors as a way of sensing the environment in the visible and near infrared spectrum; and 2) An N-Sensor method, which transmits light within an area that is reflected by the chlorophyll in the plants. The amount of chlorophyll provides a value that can be correlated with the biomass. The choice of technique depends on how the information is to be used. If the purpose is to form a general picture of the growth of vegetation on railway embankments as a way to plan for maintenance measures, then the N-Sensor technique may be the right choice. If the plan is to form a general picture as well as monitor and survey current and exact vegetation status on the surface over time as a way to fight specific vegetation with the correct means, then the machine vision method is the better of the two. Both techniques involve registering data using GPS positioning. In the future, it will be possible to store this information in databases that are directly accessible to stakeholders online during or in conjunction with measures to deal with the vegetation. The two techniques were compared with manual (visual) estimations as to the levels of vegetation growth. The observers (raters) visual estimation of weed coverage (%) differed statistically from person to person. In terms of estimating the frequency (number) of woody plants (trees and bushes) in the test areas, the observers were generally in agreement. The same person is often consistent in his or her estimation: it is the comparison with the estimations of others that can lead to misleading results. The system for using the information about vegetation growth requires development. The threshold for the amount of weeds that can be tolerated in different track types is an important component in such a system. The classification system must be capable of dealing with the demands placed on it so as to ensure the quality of the track and other pre-conditions such as traffic levels, conditions pertaining to track location, and the characteristics of the vegetation. The project recommends that the Swedish Transport Administration: Discusses how threshold values for the growth of vegetation on railway embankments can be determined Carries out registration of the growth of vegetation over longer and a larger number of railway sections using one or more of the methods studied in the project Introduces a system that effectively matches the information about vegetation to its position Includes information about the growth of vegetation in the records that are currently maintained of the track’s technical quality, and link the data material to other maintenance-related databases Establishes a number of representative surfaces in which weed inventories (by measuring) are regularly conducted, as a means of developing an overview of the long-term development that can serve as a basis for more precise prognoses in terms of vegetation growth Ensures that necessary opportunities for education are put in place

Monitoring vegetation on railway embankments : supporting maintenance decisions

The national railway administrations in Scandinavia, Germany, and Austria mainly resort to manual inspections to control vegetation growth along railway embankments. Manually inspecting railways is slow and time consuming. A more worrying aspect concerns the fact that human observers are often unable to estimate the true cover of vegetation on railway embankments. Further human observers often tend to disagree with each other when more than one observer is engaged for inspection. Lack of proper techniques to identify the true cover of vegetation even result in the excess usage of herbicides; seriously harming the environment and threating the ecology. Hence work in this study has investigated aspects relevant to human variationand agreement to be able to report better inspection routines. This was studied by mainly carrying out two separate yet relevant investigations.First, thirteen observers were separately asked to estimate the vegetation cover in nine imagesacquired (in nadir view) over the railway tracks. All such estimates were compared relatively and an analysis of variance resulted in a significant difference on the observers’ cover estimates (p<0.05). Bearing in difference between the observers, a second follow-up field-study on the railway tracks was initiated and properly investigated. Two railway segments (strata) representingdifferent levels of vegetationwere carefully selected. Five sample plots (each covering an area of one-by-one meter) were randomizedfrom each stratumalong the rails from the aforementioned segments and ten images were acquired in nadir view. Further three observers (with knowledge in the railway maintenance domain) were separately asked to estimate the plant cover by visually examining theplots. Again an analysis of variance resulted in a significant difference on the observers’ cover estimates (p<0.05) confirming the result from the first investigation.The differences in observations are compared against a computer vision algorithm which detects the "true" cover of vegetation in a given image. The true cover is defined as the amount of greenish pixels in each image as detected by the computer vision algorithm. Results achieved through comparison strongly indicate that inconsistency is prevalent among the estimates reported by the observers. Hence, an automated approach reporting the use of computer vision is suggested, thus transferring the manual inspections into objective monitored inspections

Detecting Plants on Railway Embankments

This paper investigates problems concerning vegetation along railways and proposes automatic means of detecting ground vegetation. Digital images of railway embankments have been acquired and used for the purpose. The current work mainly proposes two algorithms to be able to achieve automation. Initially a vegetation detection algorithm has been investigated for the purpose of detecting vegetation. Further a rail detection algorithm that is capable of identifying the rails and eventually the valid sampling area has been investigated. Results achieved in the current work report satisfactory (qualitative) detection rates.

Renar och vindkraft II : Vindkraft i drift och effekter på renar och renskötsel

Utbyggnaden av vindkraft inom renskötselområdet har ökat markant det senaste decenniet, trots att kunskapen om påverkan av vindkraftsetableringar ännu inte är fullt utredd och dokumenterad. I den här rapporten beskriver vi framförallt hur vindkraftparker i driftsfas påverkar renarna och renskötseln i tre olika områden. I Malå sameby har vi studerat kalvningsområdet kring Storliden och Jokkmokkslidens vindkraftparker. I Vilhelmina Norra sameby har vi studerat vinterbetesområdet kring Stor-Rotlidens vindkraftpark, samt Lögdeålandets betesområde med Gabrielsbergets vindkraftpark som används av Byrkije reinbetesdistrikt från Norge. För att få en helhetsbild av hur renarna använder sitt betesområde är det viktigt att studera renarnas betes- och förflyttningsmönster långsiktigt och över hela deras betesområde och inte bara inom det lokala området nära parken. Det är också viktigt att ta hänsyn till att renarnas betesutnyttjande skiftar från år till år och mellan olika årstider beroende på väderlek och andra yttre förutsättningar. Vi vill också understryka vikten av att kombinera den traditionella kunskapen från renskötarna med vedertagna vetenskapliga analysmetoder för att besvara de frågor som är viktiga för att kunna bedriva en hållbar renskötsel. Vi har undersökt renarnas användning av områdena genom att utföra spillningsinventeringar under åren 2009-2015 (endast i Malå sameby), och genom att följa renar utrustade med GPS-halsband under åren 2005-2015. Datat är insamlat före och under byggfas och under driftsfas (för Gabrielsberget finns GPS-data endast för driftsfasen). Vi har analyserat data genom att utveckla statistiska modeller för val av betesområde för varje område där vi har beräknat hur renarna förhåller sig till vindkraftparksområdet före, under och efter byggnation, och på Gabrielsberget när parken varit avstängd under 40 dagar och under drift vid olika renskötselsituationer. Genom intervjuer, möten och samtal, samt information från Gabrielsbergets vindkraftparks kontrollprogram, har vi tagit del av renskötarnas erfarenheter av hur renarnas beteende, och därmed även renskötseln, påverkats av vindkraftsutbyggnaden i respektive område. Våra resultat visar att renarna både på kalvnings- och på vinterbetesområden påverkas negativt av vindkraftsetableringarna (Tabell a). Renarna undviker att beta i områden där de kan se och/eller höra vindkraftsverken och föredrar att vistas i områden där vindkraftverken är skymda. I kalvningsområdet i Malå ökade användningen av skymda områden med 60 % under driftsfas. I vinterbetesområdet på Gabrielsberget, när renarna utfodrades i parken och kantbevakades intensivt för att stanna i parkområdet under driftsfas, ökade användningen av skymda områden med 13 % jämfört med när de inte var utfodrade och fick ströva mer fritt. Resultaten visar också att renarna minskar sin användning av området nära vindkraftparkerna. I kalvningslandet i Malå minskar renarna sin användning av områden inom 5 km från parkerna med 16-20 %. Vintertid vid Gabrielsbergets vindkraftpark undvek renarna parken med 3 km. Våra resultat visar även att renarnas betesro minskar inom en radie på 4 km från vindkraftparkerna under kalvningsperioden och tiden därefter i jämförelse med perioden före byggfas. Exakta avstånd som renarna påverkas beror på förutsättningarna i respektive område, exempelvis hur topografin ser ut eller om området är begränsat av stängsel eller annan infrastruktur. Förändringarna i habitatutnyttjande i våra studieområden blev tydligare när parkerna var centralt belägna i renarnas betesområde, som i kalvningsområdet i Malå eller i vinterbeteslandet på Gabrielsberget, medan det inte var lika tydliga effekter kring Stor-Rotlidens park, som ligger i utkanten av ett huvudbetesområde. Oftast är snöförhållandena bättre ur betessynpunkt högre upp i terrängen än nere i dalgångarna, på grund av stabilare temperatur, vind som blåser bort snötäcket och mer variation i topografin. Därför kan etablering av vindkraftparker i höglänta områden försämra möjligheten att använda sådana viktiga reservbetesområden under vintrar med i övrigt dåliga snöförhållanden, vilka blir allt vanligare i och med klimatförändringarna. Våra resultat tyder inte direkt på att renarna påverkats negativt under dåliga betesvintrar men fler år av studier behövs för att ytterligare klargöra hur vindkraft påverkar renarna under dessa vintrar. Våra studier har visat att etablering av vindkraft har konsekvenser för renskötseln under både barmarkssäsongen och under vintern, men effekterna förmodas få störst inverkan inom vinterbetesområdet där det är svårt att hitta alternativa betesområden för renarna. Under sommaren är betestillgången oftast mindre begränsad och renarna kan lättare hitta alternativa områden. En direkt konsekvens av Gabrielsbergets vindkraftpark som är placerad mitt i ett vinterbetesområde har blivit att renarna behöver tillskottsutfodras och bevakas intensivare för att de inte ska gå ut ur området. När den naturliga vandringen mellan olika betesområden störs för att renarna undviker att vistas i ett område kan det leda till att den totala tillgången till naturligt bete minskar och att man permanent måste tillskottsutfodra, alternativt minska antalet renar. Annan infrastruktur som vägar och kraftledningar påverkar också renarna. Vid Storliden och Jokkmokksliden och vid Stor-Rotliden där data samlats in innan vindkraftparken uppfördes visar våra resultat att renarna undviker de omkringliggande landsvägarna redan innan parkerna etablerades. Vid Stor-Rotliden ökar dock renarna användningen av områden nära vägarna efter att parken är byggd. På Gabrielsberget, där vi endast har data under drifttiden, är renarna närmare vägarna (även stora vägar som E4) när renskötarna minskar på kantbevakningen för att inte hålla renarna nära parken. Detta ökar naturligtvis risken för trafikolyckor och innebär att renskötarna måste bevaka dessa områden intensivare. Sist i rapporten presenterar vi förslag till åtgärder som kan användas för att underlätta arbetet för renskötseln om det är så att en vindkraftpark redan är byggd. Några exempel på åtgärder som är direkt kopplat till parken är att stänga av vägarna in i vindkraftparken för att förhindra nöjeskörning med skoter och bil under den tiden renarna vistas i området samt tät dialog mellan vindkraftsbolag och sameby angående vinterväghållningen av vägarna till och inom vindkraftparken. Andra mer regionala åtgärder för att förbättra förutsättningarna för renskötselarbetet på andra platser för samebyn, kan vara att sätta stängsel längst större vägar och järnvägar (t.ex. E4:an eller stambanan) i kombination med strategiskt utplacerade ekodukter. Detta för att underlätta och återställa möjligheterna till renarnas fria strövning och renskötarnas flytt av renar mellan olika betesområden.   

A machine learning approach for recognising woody plants on railway trackbeds

The purpose of this work in progress study was to test the concept of recognising plants using images acquired by image sensors in a controlled noise-free environment. The presence of vegetation on railway trackbeds and embankments presents potential problems. Woody plants (e.g. Scots pine, Norway spruce and birch) often establish themselves on railway trackbeds. This may cause problems because legal herbicides are not effective in controlling them; this is particularly the case for conifers. Thus, if maintenance administrators knew the spatial position of plants along the railway system, it may be feasible to mechanically harvest them. Primary data were collected outdoors comprising around 700 leaves and conifer seedlings from 11 species. These were then photographed in a laboratory environment. In order to classify the species in the acquired image set, a machine learning approach known as Bag-of-Features (BoF) was chosen. Irrespective of the chosen type of feature extraction and classifier, the ability to classify a previously unseen plant correctly was greater than 85%. The maintenance planning of vegetation control could be improved if plants were recognised and localised. It may be feasible to mechanically harvest them (in particular, woody plants). In addition, listed endangered species growing on the trackbeds can be avoided. Both cases are likely to reduce the amount of herbicides, which often is in the interest of public opinion. Bearing in mind that natural objects like plants are often more heterogeneous within their own class rather than outside it, the results do indeed present a stable classification performance, which is a sound prerequisite in order to later take the next step to include a natural background. Where relevant, species can also be listed under the Endangered Species Act.