Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

An infinite-horizon robust filter for uncertain hidden Markov Models with conditional relative entropy constraints

We consider a robust filtering problem for uncertain discrete-time, homogeneous, first-order, finite-state hidden Markov models (HMMs). The class of uncertain HMMs considered is described by a conditional relative entropy constraint on measures perturbed from a nominal regular conditional probability distribution given the previous posterior state distribution and the latest measurement. Under this class of perturbations, a robust infinite horizon filtering problem is first formulated as a constrained optimization problem before being transformed via variational results into an unconstrained optimization problem; the latter can be elegantly solved using a risk-sensitive information-state based filtering.

Visual predictive control of spiral motion

This paper deals with constrained image-based visual servoing of circular and conical spiral motion about an unknown object approximating a single image point feature. Effective visual control of such trajectories has many applications for small unmanned aerial vehicles, including surveillance and inspection, forced landing (homing), and collision avoidance. A spherical camera model is used to derive a novel visual-predictive controller (VPC) using stability-based design methods for general nonlinear model-predictive control. In particular, a quasi-infinite horizon visual-predictive control scheme is derived. A terminal region, which is used as a constraint in the controller structure, can be used to guide appropriate reference image features for spiral tracking with respect to nominal stability and feasibility. Robustness properties are also discussed with respect to parameter uncertainty and additive noise. A comparison with competing visual-predictive control schemes is made, and some experimental results using a small quad rotor platform are given.

The economies within an online social network market : A case study of Ravelry

The capacity of the internet to handle micro-transactions and to cater to niche markets is a boon for some areas of the creative industries, which have always been associated with smallscale micro business activities. This paper looks at the specific case of the specialist Social Networking Site Ravelry: a site for knitters, crocheters, spinners and dyers. It traces the interactions between amateurs and professionals through the emergence of social networking sites. An analytic framework of social network markets (see Potts, Cunningham, Hartley and Omerod, 2008) is employed to allow for the inclusion of amateur, social, semi-professional,professional and institutional actors within a networked sphere of activity, rather than excluding some of these actors as outside of recognised value-production. The reliance on social networks to determine the economic success of design, production and consumption is exemplified in this small scale example. This paper eschews the dichotomy of commercial and non-commercial by bringing to the fore the hybridity of this site where financial and social economies co-exist.

The arobase in the libr@ry : new political economies of children’s literatures and literacies

This essay--part of a special issue on the work of Gunther Kress--uses the idea of affordances and constraints to explore the (im)possibilities of new environments for engaging with literature written for children (see Kress, 2003). In particular, it examines a festival of children's literature from an Australian education context that occurs online. The festival is part of a technologically mediated library space designated by the term libr@ry (Kapitzke & Bruce, 2006). The @ symbol (French word "arobase") inserted into the word library indicates that technological mediation has a history, an established set of social practices, and a political economy, which even chatrooms with "real" authors may alter but not fully supplant.

Creative Economies, Creative Cities: Asian-European Perspectives

Thirteen papers examine Asian and European experiences with developing national and city policy agendas around cultural and creative industries. Papers discuss policy transfer and the field of the cultural and creative industries--what can be learned from Europe; creative industries across cultural borders--the case of video games in Asia; spaces of culture and economy--mapping the cultural-creative cluster landscape; beyond networks and relations--toward rethinking creative cluster theory; the capital complex--Beijing's new creative clusters; the European creative class and regional development--how relevant Richard Florida's theory is for Europe; getting out of place--the mobile creative class taking on the local--a U.K. perspective on the creative class; Asian cities and limits to creative capital theory; the creative industries, governance, and economic development--a U.K. perspective; Shanghai's emergence into the global creative economy; Shanghai moderne--creative economy in a creative city?; urbanity as a political project--toward post-national European cities; and alternative policies in urban innovation. Contributors include economists. Kong is with the Department of Geography at the National University of Singapore. O'Connor is at Queensland University of Technology. Index.

Do developing economies require creative industries? Some old theory about new China.

This paper argues that media and communications theory, as with cultural and creative industries analysis, can benefit from a deeper understanding of economic growth theory. Economic growth theory is elucidated in the context of both cultural and media studies and with respect to modern Chinese economic development. Economic growth is a complex evolutionary process that is tightly integrated with socio-cultural and political processes. This paper seeks to explore this mechanism and to advance cultural theory from an erstwhile political economy perspective to one centred about the co-evolutionary dynamics of economic and socio-political systems. A generic model is presented in which economic and social systems co-evolve through the origination, adoption and retention of new ideas, and in which the creative industries are a key part of this process. The paper concludes that digital media capabilities are a primary source of economic development.

Manufacturing management programs : are developing economies bridging the strategic gap?

Over the past decade there have been a large number of significant innovations in manufacturing which have resulted in more flexible and cost efficient methods and higher quality products, as manufacturers have set about upgrading their processes, systems and performance. This study compares the use of technologies and improvement programs between OECD and Non-OECD countries, and also between small and large firms, and examines differences in the use and outcomes across the economies and different sized firms. The empirical analysis provides an opportunity to test whether the ‘capabilities’ or strengths of a firm can be linked to these activities and examines if developing economies are using technology or management programs such as quality management and business process re-engineering to catch up to manufacturers in developed economies. The analysis utilises data from the second International Manufacturing Strategy Survey (IMSS), which encompasses 703 firms in 23 countries.

Can book-to-market and size be risk factors that predict economic growth in Asia's emerging economies?

We investigate whether the two 2 zero cost portfolios, SMB and HML, have the ability to predict economic growth for markets investigated in this paper. Our findings show that there are only a limited number of cases when the coefficients are positive and significance is achieved in an even more limited number of cases. Our results are in stark contrast to Liew and Vassalou (2000) who find coefficients to be generally positive and of a similar magnitude. We go a step further and also employ the methodology of Lakonishok, Shleifer and Vishny (1994) and once again fail to support the risk-based hypothesis of Liew and Vassalou (2000). In sum, we argue that search for a robust economic explanation for firm size and book-to-market equity effects needs sustained effort as these two zero cost portfolios do not represent economically relevant risk.

Economic growth, inequality and technology adoption in transitional economies

The stylized facts that motivate this thesis include the diversity in growth patterns that are observed across countries during the process of economic development, and the divergence over time in income distributions both within and across countries. This thesis constructs a dynamic general equilibrium model in which technology adoption is costly and agents are heterogeneous in their initial holdings of resources. Given the households‟ resource level, this study examines how adoption costs influence the evolution of household income over time and the timing of transition to more productive technologies. The analytical results of the model constructed here characterize three growth outcomes associated with the technology adoption process depending on productivity differences between the technologies. These are appropriately labeled as „poverty trap‟, „dual economy‟ and „balanced growth‟. The model is then capable of explaining the observed diversity in growth patterns across countries, as well as divergence of incomes over time. Numerical simulations of the model furthermore illustrate features of this transition. They suggest that that differences in adoption costs account for the timing of households‟ decision to switch technology which leads to a disparity in incomes across households in the technology adoption process. Since this determines the timing of complete adoption of the technology within a country, the implications for cross-country income differences are obvious. Moreover, the timing of technology adoption appears to be impacts on patterns of growth of households, which are different across various income groups. The findings also show that, in the presence of costs associated with the adoption of more productive technologies, inequalities of income and wealth may increase over time tending to delay the convergence in income levels. Initial levels of inequalities in the resources also have an impact on the date of complete adoption of more productive technologies. The issue of increasing income inequality in the process of technology adoption opens up another direction for research. Specifically increasing inequality implies that distributive conflicts may emerge during the transitional process with political- economy consequences. The model is therefore extended to include such issues. Without any political considerations, taxes would leads to a reduction in inequality and convergence of incomes across agents. However this process is delayed if politico-economic influences are taken into account. Moreover, the political outcome is sub optimal. This is essentially due to the fact that there is a resistance associated with the complete adoption of the advanced technology.

Architecture for a free subjectivity : Deleuze and Guattari at the Horizon of the Real

Architecture for a Free Subjectivity reformulates the French philosopher Gilles Deleuze's model of subjectivity for architecture, by surveying the prolific effects of architectural encounter, and the spaces that figure in them. For Deleuze and his Lacanian collaborator Félix Guattari, subjectivity does not refer to a person, but to the potential for and event of matter becoming subject, and the myriad ways for this to take place. By extension, this book theorizes architecture as a self-actuating or creative agency for the liberation of purely "impersonal effects." Imagine a chemical reaction, a riot in the banlieues, indeed a walk through a city. Simone Brott declares that the architectural object does not merely take part in the production of subjectivity, but that it constitutes its own.

The role of culture in knowledge management : the case of developing economies

This paper explores the role of culture in Knowledge Management (KM) through a spectrum of cultural and institutional perspectives. The case studies cover a wide range of countries in Africa, Asia, the Middle East, Latin America as well as transition economies of the former socialist countries in Eastern Europe. The paper demonstrates how knowledge management processes and practices are influenced by local culture and institutions as well as interaction with the broader international community.