Linear programming for large-scale Markov decision problems

We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest goal of competing with a low-dimensional family of policies. We use the dual linear programming formulation of the MDP average cost problem, in which the variable is a stationary distribution over state-action pairs, and we consider a neighborhood of a low-dimensional subset of the set of stationary distributions (defined in terms of state-action features) as the comparison class. We propose a technique based on stochastic convex optimization and give bounds that show that the performance of our algorithm approaches the best achievable by any policy in the comparison class. Most importantly, this result depends on the size of the comparison class, but not on the size of the state space. Preliminary experiments show the effectiveness of the proposed algorithm in a queuing application.

A framework for evaluating the impact of obesity prevention strategies on socioeconomic inequalities in weight

We developed a theoretical framework to organize obesity prevention interventions by their likely impact on the socioeconomic gradient of weight. The degree to which an intervention involves individual agency versus structural change influences socioeconomic inequalities in weight. Agentic interventions, such as standalone social marketing, increase socioeconomic inequalities. Structural interventions, such as food procurement policies and restrictions on unhealthy foods in schools, show equal or greater benefit for lower socioeconomic groups. Many obesity prevention interventions belong to the agento–structural types of interventions, and account for the environment in which health behaviors occur, but they require a level of individual agency for behavioral change, including workplace design to encourage exercise and fiscal regulation of unhealthy foods or beverages. Obesity prevention interventions differ in their effectiveness across socioeconomic groups. Limiting further increases in socioeconomic inequalities in obesity requires implementation of structural interventions. Further empirical evaluation, especially of agento–structural type interventions, remains crucial.

Linear and weakly nonlinear stability analyses of cooperative car-following models

Stability analyses have been widely used to better understand the mechanism of traffic jam formation. In this paper, we consider the impact of cooperative systems (a.k.a. connected vehicles) on traffic dynamics and, more precisely, on flow stability. Cooperative systems are emerging technologies enabling communication between vehicles and/or with the infrastructure. In a distributed communication framework, equipped vehicles are able to send and receive information to/from other equipped vehicles. Here, the effects of cooperative traffic are modeled through a general bilateral multianticipative car-following law that improves cooperative drivers' perception of their surrounding traffic conditions within a given communication range. Linear stability analyses are performed for a broad class of car-following models. They point out different stability conditions in both multianticipative and nonmultianticipative situations. To better understand what happens in unstable conditions, information on the shock wave structure is studied in the weakly nonlinear regime by the mean of the reductive perturbation method. The shock wave equation is obtained for generic car-following models by deriving the Korteweg de Vries equations. We then derive traffic-state-dependent conditions for the sign of the solitary wave (soliton) amplitude. This analytical result is verified through simulations. Simulation results confirm the validity of the speed estimate. The variation of the soliton amplitude as a function of the communication range is provided. The performed linear and weakly nonlinear analyses help justify the potential benefits of vehicle-integrated communication systems and provide new insights supporting the future implementation of cooperative systems.

Effect of decellularization on the load-bearing characteristics of articular cartilage matrix

The application of decellularized extracellular matrices to aid tissue regeneration in reconstructive surgery and regenerative medicine has been promising. Several decellularization protocols for removing cellular materials from natural tissues such as heart valves are currently in use. This paper evaluates the feasibility of potential extension of this methodology relative to the desirable properties of load bearing joint tissues such as stiffness, porosity and ability to recover adequately after deformation to facilitate physiological function. Two decellularization protocols, namely: Trypsin and Triton X-100 were evaluated against their effects on bovine articular cartilage, using biomechanical, biochemical and microstructural techniques. These analyses revealed that decellularization with trypsin resulted in severe loss of mechanical stiffness including deleterious collapse of the collagen architecture which in turn significantly compromised the porosity of the construct. In contrast, triton X-100 detergent treatment yielded samples that retain mechanical stiffness relative to that of the normal intact cartilage sample, but the resulting construct contained ruminant cellular constituents. We conclude that both of these common decellularization protocols are inadequate for producing constructs that can serve as effective replacement and scaffolds to regenerate articular joint tissue.

Two linear beetle-type scanning tunneling microscopes

Two beetle-type scanning tunneling microscopes are described. Both designs have the thermal stability of the Besocke beetle and the simplicity of the Wilms beetle. Moreover, sample holders were designed that also allow both semiconductor wafers and metal single crystals to be studied. The coarse approach is a linear motion of the beetle towards the sample using inertial slip–stick motion. Ten wires are required to control the position of the beetle and scanner and measure the tunneling current. The two beetles were built with different sized piezolegs, and the vibrational properties of both beetles were studied in detail. It was found, in agreement with previous work, that the beetle bending mode is the lowest principal eigenmode. However, in contrast to previous vibrational studies of beetle-type scanning tunneling microscopes, we found that the beetles did not have the “rattling” modes that are thought to arise from the beetle sliding or rocking between surface asperities on the raceway. The mass of our beetles is 3–4 times larger than the mass of beetles where rattling modes have been observed. We conjecture that the mass of our beetles is above a “critical beetle mass.” This is defined to be the beetle mass that attenuates the rattling modes by elastically deforming the contact region to the extent that the rattling modes cannot be identified as distinct modes in cross-coupling measurements.

Feasibility of autologous bone marrow mesenchymal stem cell-derived extracellular matrix scaffold for cartilage tissue engineering

Extracellular matrix (ECM) materials are widely used in cartilage tissue engineering. However, the current ECM materials are unsatisfactory for clinical practice as most of them are derived from allogenous or xenogenous tissue. This study was designed to develop a novel autologous ECM scaffold for cartilage tissue engineering. The autologous bone marrow mesenchymal stem cell-derived ECM (aBMSC-dECM) membrane was collected and fabricated into a three-dimensional porous scaffold via cross-linking and freeze-drying techniques. Articular chondrocytes were seeded into the aBMSC-dECM scaffold and atelocollagen scaffold, respectively. An in vitro culture and an in vivo implantation in nude mice model were performed to evaluate the influence on engineered cartilage. The current results showed that the aBMSC-dECM scaffold had a good microstructure and biocompatibility. After 4 weeks in vitro culture, the engineered cartilage in the aBMSC-dECM scaffold group formed thicker cartilage tissue with more homogeneous structure and higher expressions of cartilaginous gene and protein compared with the atelocollagen scaffold group. Furthermore, the engineered cartilage based on the aBMSC-dECM scaffold showed better cartilage formation in terms of volume and homogeneity, cartilage matrix content, and compressive modulus after 3 weeks in vivo implantation. These results indicated that the aBMSC-dECM scaffold could be a successful novel candidate scaffold for cartilage tissue engineering.

Smoothed rank-based procedure for censored data

A smoothed rank-based procedure is developed for the accelerated failure time model to overcome computational issues. The proposed estimator is based on an EM-type procedure coupled with the induced smoothing. "The proposed iterative approach converges provided the initial value is based on a consistent estimator, and the limiting covariance matrix can be obtained from a sandwich-type formula. The consistency and asymptotic normality of the proposed estimator are also established. Extensive simulations show that the new estimator is not only computationally less demanding but also more reliable than the other existing estimators.

Efficient estimation for rank-based regression with clustered data

Rank-based inference is widely used because of its robustness. This article provides optimal rank-based estimating functions in analysis of clustered data with random cluster effects. The extensive simulation studies carried out to evaluate the performance of the proposed method demonstrate that it is robust to outliers and is highly efficient given the existence of strong cluster correlations. The performance of the proposed method is satisfactory even when the correlation structure is misspecified, or when heteroscedasticity in variance is present. Finally, a real dataset is analyzed for illustration.

Rank regression for accelerated failure time model with clustered and censored data

For clustered survival data, the traditional Gehan-type estimator is asymptotically equivalent to using only the between-cluster ranks, and the within-cluster ranks are ignored. The contribution of this paper is two fold: - (i) incorporating within-cluster ranks in censored data analysis, and; - (ii) applying the induced smoothing of Brown and Wang (2005, Biometrika) for computational convenience. Asymptotic properties of the resulting estimating functions are given. We also carry out numerical studies to assess the performance of the proposed approach and conclude that the proposed approach can lead to much improved estimators when strong clustering effects exist. A dataset from a litter-matched tumorigenesis experiment is used for illustration.

Rank regression analysis of correlated water quality data from South East Queensland

With growing population and fast urbanization in Australia, it is a challenging task to maintain our water quality. It is essential to develop an appropriate statistical methodology in analyzing water quality data in order to draw valid conclusions and hence provide useful advices in water management. This paper is to develop robust rank-based procedures for analyzing nonnormally distributed data collected over time at different sites. To take account of temporal correlations of the observations within sites, we consider the optimally combined estimating functions proposed by Wang and Zhu (Biometrika, 93:459-464, 2006) which leads to more efficient parameter estimation. Furthermore, we apply the induced smoothing method to reduce the computational burden. Smoothing leads to easy calculation of the parameter estimates and their variance-covariance matrix. Analysis of water quality data from Total Iron and Total Cyanophytes shows the differences between the traditional generalized linear mixed models and rank regression models. Our analysis also demonstrates the advantages of the rank regression models for analyzing nonnormal data.

Smooth bootstrap methods for analysis of longitudinal data

In analysis of longitudinal data, the variance matrix of the parameter estimates is usually estimated by the 'sandwich' method, in which the variance for each subject is estimated by its residual products. We propose smooth bootstrap methods by perturbing the estimating functions to obtain 'bootstrapped' realizations of the parameter estimates for statistical inference. Our extensive simulation studies indicate that the variance estimators by our proposed methods can not only correct the bias of the sandwich estimator but also improve the confidence interval coverage. We applied the proposed method to a data set from a clinical trial of antibiotics for leprosy.

A modified pseudolikelihood approach for analysis of longitudinal data

We consider the analysis of longitudinal data when the covariance function is modeled by additional parameters to the mean parameters. In general, inconsistent estimators of the covariance (variance/correlation) parameters will be produced when the "working" correlation matrix is misspecified, which may result in great loss of efficiency of the mean parameter estimators (albeit the consistency is preserved). We consider using different "Working" correlation models for the variance and the mean parameters. In particular, we find that an independence working model should be used for estimating the variance parameters to ensure their consistency in case the correlation structure is misspecified. The designated "working" correlation matrices should be used for estimating the mean and the correlation parameters to attain high efficiency for estimating the mean parameters. Simulation studies indicate that the proposed algorithm performs very well. We also applied different estimation procedures to a data set from a clinical trial for illustration.

Effects of variance-function misspecification in analysis of longitudinal data

The approach of generalized estimating equations (GEE) is based on the framework of generalized linear models but allows for specification of a working matrix for modeling within-subject correlations. The variance is often assumed to be a known function of the mean. This article investigates the impacts of misspecifying the variance function on estimators of the mean parameters for quantitative responses. Our numerical studies indicate that (1) correct specification of the variance function can improve the estimation efficiency even if the correlation structure is misspecified; (2) misspecification of the variance function impacts much more on estimators for within-cluster covariates than for cluster-level covariates; and (3) if the variance function is misspecified, correct choice of the correlation structure may not necessarily improve estimation efficiency. We illustrate impacts of different variance functions using a real data set from cow growth.

Iterative estimating equations: Linear convergence and asymptotic properties

We propose an iterative estimating equations procedure for analysis of longitudinal data. We show that, under very mild conditions, the probability that the procedure converges at an exponential rate tends to one as the sample size increases to infinity. Furthermore, we show that the limiting estimator is consistent and asymptotically efficient, as expected. The method applies to semiparametric regression models with unspecified covariances among the observations. In the special case of linear models, the procedure reduces to iterative reweighted least squares. Finite sample performance of the procedure is studied by simulations, and compared with other methods. A numerical example from a medical study is considered to illustrate the application of the method.

Induced smoothing for rank regression with censored survival times

Adaptions of weighted rank regression to the accelerated failure time model for censored survival data have been successful in yielding asymptotically normal estimates and flexible weighting schemes to increase statistical efficiencies. However, for only one simple weighting scheme, Gehan or Wilcoxon weights, are estimating equations guaranteed to be monotone in parameter components, and even in this case are step functions, requiring the equivalent of linear programming for computation. The lack of smoothness makes standard error or covariance matrix estimation even more difficult. An induced smoothing technique overcame these difficulties in various problems involving monotone but pure jump estimating equations, including conventional rank regression. The present paper applies induced smoothing to the Gehan-Wilcoxon weighted rank regression for the accelerated failure time model, for the more difficult case of survival time data subject to censoring, where the inapplicability of permutation arguments necessitates a new method of estimating null variance of estimating functions. Smooth monotone parameter estimation and rapid, reliable standard error or covariance matrix estimation is obtained.