Web Data Extraction from Query Result Pages Based on Visual and Content Features

A rapidly increasing number of Web databases are now become accessible via
their HTML form-based query interfaces. Query result pages are dynamically generated
in response to user queries, which encode structured data and are displayed for human
use. Query result pages usually contain other types of information in addition to query
results, e.g., advertisements, navigation bar etc. The problem of extracting structured data
from query result pages is critical for web data integration applications, such as comparison
shopping, meta-search engines etc, and has been intensively studied. A number of approaches
have been proposed. As the structures of Web pages become more and more complex, the
existing approaches start to fail, and most of them do not remove irrelevant contents which
may a®ect the accuracy of data record extraction. We propose an automated approach for
Web data extraction. First, it makes use of visual features and query terms to identify data
sections and extracts data records in these sections. We also represent several content and
visual features of visual blocks in a data section, and use them to ¯lter out noisy blocks.
Second, it measures similarity between data items in di®erent data records based on their
visual and content features, and aligns them into di®erent groups so that the data in the
same group have the same semantics. The results of our experiments with a large set of
Web query result pages in di®erent domains show that our proposed approaches are highly
e®ective.

Search filters: mapping the literature using a novel form of content analysis.

Background: Search filters are combinations of words and phrases designed to retrieve an optimal set of records on a particular topic (subject filters) or study design (methodological filters). Information specialists are increasingly turning to reusable filters to focus their searches. However, the extent of the academic literature on search filters is unknown. We provide a broad overview to the academic literature on search filters.
Objectives: To map the academic literature on search filters from 2004 to 2015 using a novel form of content analysis.
Methods: We conducted a comprehensive search for literature between 2004 and 2015 across eight databases using a subjectively derived search strategy. We identified key words from titles, grouped them into categories, and examined their frequency and co-occurrences.
Results: The majority of records were housed in Embase (n = 178) and MEDLINE (n = 154). Over the last decade, both databases appeared to exhibit a bimodal distribution with the number of publications on search filters rising until 2006, before dipping in 2007, and steadily increasing until 2012. Few articles appeared in social science databases over the same time frame (e.g. Social Services Abstracts, n = 3).
Unsurprisingly, the term ‘search’ appeared in most titles, and quite often, was used as a noun adjunct for the word 'filter' and ‘strategy’. Across the papers, the purpose of searches as a means of 'identifying' information and gathering ‘evidence’ from 'databases' emerged quite strongly. Other terms relating to the methodological assessment of search filters, such as precision and validation, also appeared albeit less frequently.
Conclusions: Our findings show surprising commonality across the papers with regard to the literature on search filters. Much of the literature seems to be focused on developing search filters to identify and retrieve information, as opposed to testing or validating such filters. Furthermore, the literature is mostly housed in health-related databases, namely MEDLINE, CINAHL, and Embase, implying that it is medically driven. Relatively few papers focus on the use of search filters in the social sciences.

Magnitude, form and bioavailability of fluvial carbon exports from Irish organic soils under pasture

Organic soils are widespread in Ireland and vulnerable to degradation via drainage for agriculture. The soil-landuse combination of pasture on organic soils may play a disproportionate role in regional C dynamics but is yet to receive study. Fluvial C fluxes and labile organic fractions were determined for two such sites at nested field (c.4 ha) and subcatchment scales (>40 ha); one relatively dry and nutrient rich, the other wetter and nutrient poor. Field scale flux from the nutrient poor site over 2 years was 38.9 ± 6.6 g C m⁻² yr⁻¹ with DIC > DOC > POC at 57, 32 and 11 % respectively, and 72 % DIC was comprised of above equilibrium CO₂. At the nutrient rich site, which overlies limestone geology, field scale export over an individual year was 90.4 g C m⁻² with DIC > DOC > POC at 49, 42 and 9 %, but with 90 % DIC as bicarbonate. By comparison with the nutrient poor site, the magnitude and composition of inorganic C exports from the nutrient rich site implied considerable export of soil-respiratory C as bicarbonate, and lower evasion losses due to carbonate system buffering. Labile DOC determined using dark incubations indicated small fractions (5–10 %) available for remineralisation over typical downstream transit times of days to weeks. These fractions are probably conservative as photolysis in the environment can increase the proportion of labile compounds via photocleavage and directly remineralise organic matter. This study demonstrates that monitoring at soil–water interfaces can aid capture of total landscape fluvial fluxes by precluding the need to incorporate prior C evasion, although rapid runoff responses at field scales can necessitate high resolution flow proportional, and hydrograph sampling to constrain uncertainty of flux estimates.