Policy Improvement for POMDPs Using Normalized Importance Sampling

We present a new method for estimating the expected return of a POMDP from experience. The estimator does not assume any knowle ge of the POMDP and allows the experience to be gathered with an arbitrary set of policies. The return is estimated for any new policy of the POMDP. We motivate the estimator from function-approximation and importance sampling points-of-view and derive its theoretical properties. Although the estimator is biased, it has low variance and the bias is often irrelevant when the estimator is used for pair-wise comparisons.We conclude by extending the estimator to policies with memory and compare its performance in a greedy search algorithm to the REINFORCE algorithm showing an order of magnitude reduction in the number of trials required.

Variability in the Stability and Productivity of Transfected Genes in Chinese Hamster Ovary (CHO) cells

In the field of biologics production, productivity and stability of the transfected gene of interest are two very important attributes that dictate if a production process is viable. To further understand and improve these two traits, we would need to further our understanding of the factors affecting them. These would include integration site of the gene, gene copy number, cell phenotypic variation and cell environment. As these factors play different parts in the development process, they lead to variable productivity and stability of the transfected gene between clones, the well-known phenomenon of “clonal variation”. A study of this phenomenon and how the various factors contribute to it will thus shed light on strategies to improve productivity and stability in the production cell line. Of the four factors, the site of gene integration appears to be one of the most important. Hence, it is proposed that work is done on studying how different integration sites affect the productivity and stability of transfected genes in the development process. For the study to be more industrially relevant, it is proposed that the Chinese Hamster Ovary dhfr-deficient cell line, CHO-DG44, is used as the model system.