Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

Symmetrical 1-pyrrolidineacetamide showing anti-HIV activity through a new binding site on HIV-1 integrase

Aim: To characterize the functional and pharmacological features of a symmetrical 1-pyrrolidineacetamide, N,N'-(methylene-di-4,1-phenylene) bis-1-pyrrolidineacetamide, as a new anti-HIV compound which could competitively inhibit HIV-1 integrase (IN) bindi

Threatened fishes of the world: Plesiomyzon baotingensis (Zheng & Chen, 1980) (Cypriniformes: Balitoridae)

We are grateful to all the voluntary donors of DNA samples in this study. We thank Hui Zhang and Yan-jiao Li for their technical help. We also thank Dr. Darren Curnoe for his critical reading of the manuscript. This study was supported by grants from the National 973 project of China (2007CB947701,2007CB815705), the Chinese Academy of Sciences (KSCX1YW-R-34, Westlight Doctoral Program), the National Natural Science Foundation of China (30525028, 30700445, 30630013 and 30771181), and the Natural Science Foundation of Yunnan Province of China (2007C100M, 2009CD107).

Population status, threats and conservation of the Yangtze finless porpoise

The Yangtze finless porpoise (Neophocaena phocaenoides asiaeorientalis) is currently limited to the middle and lower reaches of the Yangtze River from Yichang to Shanghai, China, and the adjoining Poyang and Dongting Lakes. Its population size has decreased remarkably during the last several decades due to the heavy impact of human activities, including overfishing of prey species, water development projects that cause attendant habitat loss and degradation, water pollution, and accidental deaths caused by harmful fishing gear and collisions with motorized vessels. It was estimated that the number of remaining individuals was down to approximately 1800 in 2006, a number that is decreasing at a rate as high as 5% per year. Three conservation measures - in situ and ex situ conservation and captive breeding have been applied to the protection of this unique porpoise since the early 1990s. Seven natural and two "semi-natural" reserves have so far been established. Since 1996, a small group of finless porpoises has been successfully reared in a facility at the Institute of Hydrobiology of the Chinese Academy of Sciences; three babies were born in captivity on July 5, 2005, June 2, 2007 and July 5, 2008. These are the first freshwater cetaceans ever born in captivity in the world. Several groups of these porpoises caught in the main stream of the Yangtze River, or rescued, have been introduced into the Tian'e-Zhou Semi-natural Reserve since 1990. These efforts have proven that, not only can these animals survive in the area, they are also to reproduce naturally and successfully. More than 30 calves had been born in the reserve since then, with one to three born each year. Taking deaths and transfers into account, there were approximately 30 individuals living in the reserve as of the end of 2007. Among eight mature females captured in April 2008, five were confirmed pregnant. This effort represents the first successful attempt at off-site protection of a cetacean species in the world, and establishes a solid base for conservation of the Yangtze finless porpoise. A lesson must be drawn from the tragedy of Chinese River Dolphin (Lipotes vexillifer), which has already been declared likely extinct. Strong, effective and appropriate protective measures must be carried out quickly to prevent the Yangtze finless porpoise from becoming a second Chinese River Dolphin, and save the biodiversity of the Yangtze River as a whole.

Threatened fishes of the world: Tanichthys albonubes Lin 1932 (Cyprinidae)

The white cloud mountain minnow Tanichthys albonubes Lin is an endemic species to southern China and the genus has two species, Tanichthys albonubes Lin and Tanichthys micagemmae Freyhof et Herder. The distribution range of T. albonubes Lin and T. micagemmae is very narrow and only found in the mountain brooks of Baiyunshan Mountain (White Cloud Mountain), Huaxian Country and the vicinity of Guangzhou in Guangdong Province and Halong, Quang Ninh Province, Vietnam respectively. The wild populations of this fish had already been on the verge of extinction when Shu-Yan Lin first discovered it in 1932 at the Baiyunshan Mountain. It was believed to be extinct in the wild because there were no reports of this fish in the wild since 1980. In September 2003, a small and isolated population of the fish was discovered in a mountain puddle in the north vicinity of Guangzhou. Additional studies are needed to determine the survival and propagation of the released fish. The protection of their natural habitat should be implemented.

Body loadings and health risk assessment of polychlorinated dibenzo-p-dioxins and dibenzofurans at an intensive electronic waste recycling site in China

This study is one of the very few investigating the dioxin body burden of a group of child-bearing-aged women at an electronic waste (e-waste) recycling site (Taizhou, Zhejiang Province) (24 +/- 2.83 years of age, 40% were primiparae) and a reference site (Lin'an city, Zhejiang Province, about 245 km away from Taizhou) (24 +/- 2.35 years of age, 100% were primiparae) in China. Five sets of samples (each set consisted of human milk, placenta, and hair) were collected from each site. Body burdens of people from the e-waste processing site (human milk, 21.02 +/- 13.81 pg WHO-TEQ(1998/g) fat (World Health Organization toxic equivalency 1998); placenta, 31.15 +/- 15.67 pg WHO-TEQ(1998/g) fat; hair, 33.82 +/- 17.74 pg WHO-TEQ(1998/g) dry wt) showed significantly higher levels of polychlorinated dibenzo-p-dioxins and polychlorinated dibenzofurnas (PCDD/Fs) than those from the reference site (human milk, 9.35 +/- 7.39 pg WHO-TEQ(1998/g) fat, placenta, 11.91 +/- 7.05 pg WHO-TEQ(1998/g) fat; hair, 5.59 +/- 4.36 pg WHO-TEQ(1998/g) dry wt) and were comparatively higher than other studies. The difference between the two sites was due to e-waste recycling operations, for example, open burning, which led to high background levels. Moreover, mothers from the e-waste recycling site consumed more foods of animal origin. The estimated daily intake of PCDD/Fs within 6 months by breast-fed infants from the e-waste processing site was 2 times higher than that from the reference site. Both values exceeded the WHO tolerable daily intake for adults by at least 25 and 11 times, respectively. Our results implicated that e-waste recycling operations cause prominent PCDD/F levels in the environment and in humans. The elevated body burden may have health implications for the next generation.

Biotransfer of persistent organic pollutants from a large site in China used for the disassembly of electronic and electrical waste

Samples of groundwater, river water, river sediment, paddy soil, rice seeds, hen eggs, fish, umbilical cord blood, and newborn meconium were collected from October 2002 to October 2003 near a large site in China used for the disassembly of obsolete transformers and other electronic or electrical waste. Six indicator PCB congeners, three non-ortho dioxin-like PCB congeners, and six organochlorine pesticides were determined in the samples by GC with electron capture detector. The results demonstrated that the local environment and edible foods had been seriously polluted by toxic PCBs and organochlorine pesticides. The actual daily intakes (ADIs) of these pollutants were estimated for local residents living in the area. The intake data showed that the contents of PCBs in these local residents were substantial, as the ADI estimates greatly exceed the reference doses set by the World Health Organization and the United States Agency for Toxic Substances and Disease Registry. The presence of the indicator PCB congeners in the cord blood and the meconium samples, as well as significant correlations (r(2) > 0.80, p < 0.05) between these levels, suggests a potential biotransfer of these indicators from mothers to their newborns. This preliminary study showed that obsolete transformers and other electronic or electrical waste can be an important source for the emission of persistent organic pollutants into the local environment, such as through leakage, evaporation, runoff, and leaching. Contamination from this source appears to have reached the level considered to be a serious threat to environmental and human health around the disassembly site.

Detection of extracellular phosphatases in natural spring phytoplankton of a shallow eutrophic lake (Donghu, China)

The species-specific production of extracellular phosphatases in phytoplankton of a subtropical polymictic take was investigated from March to May 2004. Phosphatase activity was detected directly at the site of enzyme action using the enzyme-labelled fluorescence (ELF) technique. Size fractionation of bulk phosphatase activity (PA), concentrations of soluble reactive phosphorus (SRP), chlorophyll a, and phytoplankton composition were determined in parallel. Phosphatase-positive cells were present in every phytoplankton sample; labelled cells were detected in 33 algal taxa, including many chlorophytes, dinoflagellates and some diatoms, but never among cyanobacteria. We recorded an unusual dinoflagellate bloom (Peridiniopsis sp.), of which similar to 25% of the cells were phosphatase-positive. Several populations were partly phosphatase-positive whenever present, while some other species never showed any activity. The production of extracellular phosphatases was not primarily regulated by ambient P concentrations; algae produced these enzymes even if SRP concentrations were high. Moreover, heterotrophic nanoflagellates most probably contributed to the pool of particle-bound PA in some samples.

Classical two-site fidelity and quantum phase transitions in the Ising model and the extended Hubbard model

In this Letter, the classical two-site-ground-state fidelity (CTGF) is exploited to identify quantum phase transitions (QPTs) for the transverse field Ising model (TFIM) and the one-dimensional extended Hubbard model (EHM). Our results show that the CTGF exhibits an abrupt change around the regions of criticality and can be used to identify QPTs in spin and fermionic systems. The method is especially convenient when it is connected with the density-matrix renormalization group (DMRG) algorithm. (C) 2008 Elsevier B.V. All rights reserved.

Angular momentum of a brane-world model

In this paper we discuss the properties of the general covariant angular momentum of a five-dimensional brane-world model. Through calculating the total angular momentum of this model, we are able to analyze the properties of the total angular momentum in the inflationary RS model. We show that the space-like components of the total angular momentum of the inflationary RS model are all zero while the others are non-zero, which agrees with the results from ordinary RS model.

Starch grains from dental calculus reveal ancient plant foodstuffs at Chenqimogou site, Gansu Province

National Natural Science Foundation of China [40771205]; National Science Fund for Distinguished Young Scholars [40625002]; Chinese Academy of Sciences [KZCX2-YW-315]

Tourism Potential of Agricultural Heritage Systems

Traditional agricultural systems are threatened world-wide mainly due to the introduction of modern agricultural techniques and the emigration of farm labourers from remote rural villages. The objective of the programme 'Globally Important Agricultural Heritage Systems' (GIAHS), initiated by the Food and Agriculture Organization (FAO) of the United Nations in 2002, is dynamic conservation of traditional agricultural systems. This article addresses the definition and content of agricultural heritage systems and discusses conservation options in the light of developing rural tourism. An explorative survey was conducted in Longxian village, situated in Zhejiang Province, southern China, focusing on the tourism potential of a typical Rice-Fish Agricultural System. The identification of heritage resources is a first step in the process of transforming an agricultural landscape into a cultural tourism landscape. However, the future of these landscapes is in the hands of a range of stakeholders and depends on their capacity to manage, in a sustainable way, tourism development strategies alongside conservation policies.

Surface Reflectance Estimation and Natural Illumination Statistics

Humans recognize optical reflectance properties of surfaces such as metal, plastic, or paper from a single image without knowledge of illumination. We develop a machine vision system to perform similar recognition tasks automatically. Reflectance estimation under unknown, arbitrary illumination proves highly underconstrained due to the variety of potential illumination distributions and surface reflectance properties. We have found that the spatial structure of real-world illumination possesses some of the statistical regularities observed in the natural image statistics literature. A human or computer vision system may be able to exploit this prior information to determine the most likely surface reflectance given an observed image. We develop an algorithm for reflectance classification under unknown real-world illumination, which learns relationships between surface reflectance and certain features (statistics) computed from a single observed image. We also develop an automatic feature selection method.

Recognition of Surface Reflectance Properties from a Single Image under Unknown Real-World Illumination

This paper describes a machine vision system that classifies reflectance properties of surfaces such as metal, plastic, or paper, under unknown real-world illumination. We demonstrate performance of our algorithm for surfaces of arbitrary geometry. Reflectance estimation under arbitrary omnidirectional illumination proves highly underconstrained. Our reflectance estimation algorithm succeeds by learning relationships between surface reflectance and certain statistics computed from an observed image, which depend on statistical regularities in the spatial structure of real-world illumination. Although the algorithm assumes known geometry, its statistical nature makes it robust to inaccurate geometry estimates.