Transactional failure recovery for a distributed key-value store

With the advent of cloud computing, many applications have embraced the ensuing paradigm shift towards modern distributed key-value data stores, like HBase, in order to benefit from the elastic scalability on offer. However, many applications still hesitate to make the leap from the traditional relational database model simply because they cannot compromise on the standard transactional guarantees of atomicity, isolation, and durability. To get the best of both worlds, one option is to integrate an independent transaction management component with a distributed key-value store. In this paper, we discuss the implications of this approach for durability. In particular, if the transaction manager provides durability (e.g., through logging), then we can relax durability constraints in the key-value store. However, if a component fails (e.g., a client or a key-value server), then we need a coordinated recovery procedure to ensure that commits are persisted correctly. In our research, we integrate an independent transaction manager with HBase. Our main contribution is a failure recovery middleware for the integrated system, which tracks the progress of each commit as it is flushed down by the client and persisted within HBase, so that we can recover reliably from failures. During recovery, commits that were interrupted by the failure are replayed from the transaction management log. Importantly, the recovery process does not interrupt transaction processing on the available servers. Using a benchmark, we evaluate the impact of component failure, and subsequent recovery, on application performance.

Desarrollo de una solución SaaS para una Enterprise Store

En el marco del proyecto europeo FI-WARE, en el CoNWet Lab (laboratorio de la ETSI Informáticos de la UPM) se ha implementado la plataforma Web Wstore que es una implementación de referencia del Store Generic Enabler perteneciente a dicho proyecto. El objetivo de FI-WARE es crear la plataforma núcleo del Internet del Futuro (IoF) con la intención de incrementar la competitividad global europea en el mundo de las TI. El proyecto introduce una infraestructura innovadora para la creación y distribución de servicios digitales en internet. WStore ofrece a los proveedores de servicios la plataforma donde publicar sus ofertas y desde la cual los clientes podrán acceder ellas. Estos proveedores ofrecen servicios Web, aplicaciones, widgets y data sets del mismo modo que Google ofrece aplicaciones en la tienda online Google Play o Apple en el App Store. WStore está implementada actualmente como una plataforma Web, por lo que una organización que desee ofrecer el servicio de la store necesita instalar el software en un servidor propio y disponer de un dominio para ofrecer sus productos. El objetivo de este trabajo es migrar WStore a un entorno de computación en la nube de manera que con una única instancia se ofrezca el servicio a las organizaciones que deseen disponer de su propia plataforma, de la cual tendrán total control como si se encontrase en su propia infraestructura. Para esto se implementa una versión de WStore que será desplegada en una infraestructura cloud y ofrecida como Software as a Service. La implementación incluye una serie de módulos de código que se podrán añadir opcionalmente en el proceso de instalación si se desea que la instancia instalada sea Multitenant. Además, en este trabajo se estudian y prueban las herramientas que ofrece MongoDB para desplegar la plataforma Wstore Multitenant en una infraestructura cloud. Estas herramientas son replica sets y sharding que permiten desplegar una base de datos escalable y de alta disponibilidad. ---ABSTRACT---In the context of the European project FI-WARE, the CoNWeT Lab (IT Lab from ETSIINF UPM university) has been implemented the web platform WStore. WStore is a reference implementation of the Generic Enabler Store from FI-WARE project. The FI-WARE goal is to create the core platform of the Future Internet (IoF) with the intention of enhancing Europe's global competitiveness in IT technologies. FI-WARE introduces an innovative infrastructure for the creation and distribution of digital services over the Internet. WStore offers to service providers a platform to publicate offerings and where customers can access them. The providers offer web services, applications, widgets and data sets in the same way that Google offers online applications on Google Play or Apple on App Store plataforms. WStore is currently implemented as a web platform, so if an organization wants to offer the store service, it need to install the software on it’s own serves and have a domain to offer their products. The objective of this paper is to migrate WStore to a cloud computing environment where a single instance of the WStore is offered as a web service to organizations who want their own store. Customers (tenants) of the WStore web service will have total control over the software and WStore administration. The implementation includes several code modules that can be optionally added in the installation process to build a Multitenant instance. In addition, this paper review the tools that MongoDB provide for scalability and high availability (replica sets and sharding) with the purpose of deploying multi-tenant WStore on a cloud infrastructure.

Prototipo de e-store para un ecosistema y un framework de provisión de servicios

El trabajo realizado se encuentra enmarcado dentro del proyecto de I+D+I del 7o programa marco de la Comisión Europea Fi-WARE: The future Internet core platform que forma parte de la iniciativa Future Internet PPP. En concreto, se ha desarrollado la especificación de un Generic Enabler con funcionalidad de tienda virtual que de soporte a la publicación y adquisición o subscripción de aplicaciones y servicios dentro del denominado Business Framework Ecosystem (BFE), además de una implementación de referencia de este Generic Enabler (GE) que ha sido utilizada para la realización de una prueba de concepto con el objetivo de comprobar la adecuación del comportamiento de la especificación dentro del BFE. La primera tarea realizada ha consistido en un estudio de otras stores (o tiendas digitales) existentes, mirando aspectos tales como la funcionalidad proporcionada, la información mostrada de los distintos productos ofrecidos o la organización de la interfaz de usuario y la metáfora visual. Este estudio ha tenido como objetivo establecer un punto de partida desde el que empezar a analizar las distintas funcionalidades que deberá proveer el sistema.Utilizando como base el estudio anterior y las necesidades concretas de la plataforma Fi-WARE se paso a la educación de los requisitos generales del sistema en los cuales se especifica a grandes rasgos la funcionalidad que debe proveer esta tienda digital así como algunos aspectos concretos de la experiencia de usuario. Una vez definida la funcionalidad de la store se ha abordado el diseño del sistema. Para realizar este diseño se ha trabajado en dos tareas principales: La primera de estas tareas ha consistido en realizar el diseño de la arquitectura del Store GE, en el que se especifican todos los módulos que debe contener el sistema para poder satisfacer los requisitos, así como las distintas conexiones del Store GE con otros componentes del proyecto Fi-Ware y de sus interrelaciones con el resto de componentes de dicho proyecto. Esto ofrece una visión global de la ubicación del Store GE dentro de la arquitectura general del proyecto Fi-Ware. La segunda tarea ha consistido en el desarrollo de la especicación abierta (Open specication) del Store GE. Esta tarea es probablemente la más relevante de cara a cumplir con los objetivos del proyecto Fi-Ware, ya que Fi-Ware se propone como objetivo principal proporcionar las especificaciones de una plataforma tecnológica abierta para la Internet del futuro, formada por un conjunto de componentes (denominados Generic Enablers), entre los que se encuentra el Store GE. En este documento ha quedado descrito con todo detalle en que consiste el Store GE y cuales son sus APIs, sobre las que se construirán las aplicaciones de la futura Internet basadas en Fi-Ware, de manera que sea posible que cualquier empresa pueda realizar una implementación diferente a la que se está desarrollando en este proyecto (si bien ésta será su implementación de referencia). Para esta Open specication se han desarrollado un modelo de gestión de usuarios y roles, un modelo de datos, diagramas de interacción que definen todas las posibles comunicaciones de la store con otros Generic Enablers del proyecto Fi-Ware, la definición del ciclo de vida de una oferta y las APIs REST del Store GE, incluyendo el contenido de las peticiones y los tipos MIME soportados. En este punto se pudo comenzar a trabajar en la implementación de referencia del Store GE. La primera tarea ha consistido en realizar la integración con el Marketplace GE, otro de los Generic Enablers del proyecto Fi-Ware, para ello se definieron unos requisitos específicos y se realizó un diseño de bajo nivel de este móodulo seguido de la propia implementación y un conjunto exhaustivo de pruebas unitarias para comprobar su correcto funcionamiento. A continuación se pasó a realizar la integracióon con el Repository GE siguiendo los mismos pasos que con la integración con el Marketplace GE. La siguiente tarea realizada ha consistido en la realización de los móodulos necesarios para permitir crear nuevas ofertas en la implementación de referencia de Store GE incluyendo nuevamente una fase de educación de requisitos específicos, un diseño de bajo nivel, la propia implementación y una serie de pruebas unitarias. Una vez implementada la creación de nuevas ofertas, se pasó a la realización de la funcionalidad necesaria para la recuperación y visualizacion de estas ofertas así como a la realización del soporte necesario para el registro de recursos y para la vinculación de estos a determinadas ofertas, siguiendo nuevamente la metodología antes mencionada. Finalmente se ha dado el soporte para la publicación y la adquisición de ofertas. En este caso la adquisición de ofertas se ha realizado tan solo en la parte servidora de la aplicación y no se ha llegado a dar soporte a esta funcionalidad en la interfaz Web al no ser necesaria para la realización de la prueba de concepto prevista. No obstante esta funcionalidad será implementada junto con otras funcionalidades como el soporte de características sociales, ya fuera del ámbito de este Trabajo de fin de grado. Como paso previo a la realización de la prueba de concepto se ha trabajado en la plataforma Wirecloud, que es una implementación de referencia del denominado Application Mashup GE, modicando su funcionalidad para integrarla con la API de compras realizada dentro de la implementación de referencia del Store GE. La úultima tarea realizada para este Trabajo de fin de grado ha consistido por fin en la realización de la prueba de concepto del Store GE integrando su implementación de referencia con las del resto de Generic Enablers, lo cual ha permitido comprobar así el fucionamiento de la arquitectura y modelo propuestos.

Coordinate regulation of gonadotropin-releasing hormone neuronal firing patterns by cytosolic calcium and store depletion

Elevation of cytosolic free Ca2+ concentration ([Ca2+]i) in excitable cells often acts as a negative feedback signal on firing of action potentials and the associated voltage-gated Ca2+ influx. Increased [Ca2+]i stimulates Ca2+-sensitive K+ channels (IK-Ca), and this, in turn, hyperpolarizes the cell and inhibits Ca2+ influx. However, in some cells expressing IK-Ca the elevation in [Ca2+]i by depletion of intracellular stores facilitates voltage-gated Ca2+ influx. This phenomenon was studied in hypothalamic GT1 neuronal cells during store depletion caused by activation of gonadotropin-releasing hormone (GnRH) receptors and inhibition of endoplasmic reticulum (Ca2+)ATPase with thapsigargin. GnRH induced a rapid spike increase in [Ca2+]i accompanied by transient hyperpolarization, followed by a sustained [Ca2+]i plateau during which the depolarized cells fired with higher frequency. The transient hyperpolarization was caused by the initial spike in [Ca2+]i and was mediated by apamin-sensitive IK-Ca channels, which also were operative during the subsequent depolarization phase. Agonist-induced depolarization and increased firing were independent of [Ca2+]i and were not mediated by inhibition of K+ current, but by facilitation of a voltage-insensitive, Ca2+-conducting inward current. Store depletion by thapsigargin also activated this inward depolarizing current and increased the firing frequency. Thus, the pattern of firing in GT1 neurons is regulated coordinately by apamin-sensitive SK current and store depletion-activated Ca2+ current. This dual control of pacemaker activity facilitates voltage-gated Ca2+ influx at elevated [Ca2+]i levels, but also protects cells from Ca2+ overload. This process may also provide a general mechanism for the integration of voltage-gated Ca2+ influx into receptor-controlled Ca2+ mobilization.

Modulation of Ca2+ entry by polypeptides of the inositol 1,4,5-trisphosphate receptor (IP3R) that bind transient receptor potential (TRP): Evidence for roles of TRP and IP3R in store depletion-activated Ca2+ entry

Homologues of Drosophilia transient receptor potential (TRP) have been proposed to be unitary subunits of plasma membrane ion channels that are activated as a consequence of active or passive depletion of Ca2+ stores. In agreement with this hypothesis, cells expressing TRPs display novel Ca2+-permeable cation channels that can be activated by the inositol 1,4,5-trisphosphate receptor (IP3R) protein. Expression of TRPs alters cells in many ways, including up-regulation of IP3Rs not coded for by TRP genes, and proof that TRP forms channels of these and other cells is still missing. Here, we document physical interaction of TRP and IP3R by coimmunoprecipitation and glutathione S-transferase-pulldown experiments and identify two regions of IP3R, F2q and F2g, that interact with one region of TRP, C7. These interacting regions were expressed in cells with an unmodified complement of TRPs and IP3Rs to study their effect on agonist- as well as store depletion-induced Ca2+ entry and to test for a role of their respective binding partners in Ca2+ entry. C7 and an F2q-containing fragment of IP3R decreased both forms of Ca2+ entry. In contrast, F2g enhanced the two forms of Ca2+ entry. We conclude that store depletion-activated Ca2+ entry occurs through channels that have TRPs as one of their normal structural components, and that these channels are directly activated by IP3Rs. IP3Rs, therefore, have the dual role of releasing Ca2+ from stores and activating Ca2+ influx in response to either increasing IP3 or decreasing luminal Ca2+.

Loperamide: A positive modulator for store-operated calcium channels?

The depletion of inositol trisphosphate-sensitive intracellular pools of calcium causes activation of store-operated calcium (SOC) channels. Loperamide at 10–30 μM has no effect on intracellular calcium levels alone, but augments calcium levels in cultured cells when SOC channels have been activated. In HL-60 leukemic cells, the apparent positive modulatory effect of loperamide on SOC channels occurs when these channels have been activated after ATP, thapsigargin, or ionomycin-elicited depletion of calcium from intracellular storage sites. Loperamide has no effect when levels of intracellular calcium are elevated through a mechanism not involving SOC channels by using sphingosine. Loperamide caused augmentation of intracellular calcium levels after activation of SOC channels in NIH 3T3 fibroblasts, astrocytoma 1321N cells, smooth muscle DDT-MF2 cells, RBL-2H3 mast cells, and pituitary GH4C1 cells. Only in astrocytoma cells did loperamide cause an elevation in intracellular calcium in the absence of activation of SOC channels. The augmentation of intracellular calcium elicited by loperamide in cultured cells was dependent on extracellular calcium and was somewhat resistant to agents (SKF 96365, miconazole, clotrimazole, nitrendipine, and trifluoperazine) that in the absence of loperamide effectively blocked SOC channels. It appears that loperamide augments influx of calcium through activated SOC channels.

Delayed Activation of the Store-operated Calcium Current Induced by Calreticulin Overexpression in RBL-1 Cells

Calreticulin (CRT) is a high-capacity, low-affinity Ca2+-binding protein located in the lumen of the endoplasmic reticulum (ER) of all eukaryotic cells investigated so far. Its high level of conservation among different species suggests that it serves functions fundamental to cell survival. The role originally proposed for CRT, i.e., the main Ca2+ buffer of the ER, has been obscured or even casted by its implication in processes as diverse as gene expression, protein folding, and cell adhesion. In this work we seek the role of CRT in Ca2+ storing and signaling by evaluating its effects on the kinetics and amplitude of the store-operated Ca2+ current (ICRAC). We show that, in the rat basophilic leukemia cell line RBL-1, overexpression of CRT, but not of its mutant lacking the high-capacity Ca2+-binding domain, markedly retards the ICRAC development, however, only when store depletion is slower than the rate of current activation. On the contrary, when store depletion is rapid and complete, overexpression of CRT has no effect. The present results are compatible with a major Ca2+-buffering role of CRT within the ER but exclude a direct, or indirect, role of this protein on the mechanism of ICRAC activation.

Mouse trp2, the homologue of the human trpc2 pseudogene, encodes mTrp2, a store depletion-activated capacitative Ca2+ entry channel

Capacitative Ca2+ entry (CCE) is Ca2+ entering after stimulation of inositol 1,4,5-trisphosphate (IP3) formation and initiation of Ca2+ store depletion. One hallmark of CCE is that it can also be triggered merely by store depletion, as occurs after inhibition of internal Ca2+ pumps with thapsigargin. Evidence has accumulated in support of a role of transient receptor potential (Trp) proteins as structural subunits of a class of Ca2+-permeable cation channels activated by agonists that stimulate IP3 formation—very likely through a direct interaction between the IP3 receptor and a Trp subunit of the Ca2+ entry channel. The role of Trp’s in Ca2+ entry triggered by store depletion alone is less clear. Only a few of the cloned Trp’s appear to enhance this type of Ca2+ entry, and when they do, the effect requires special conditions to be observed, which native CCE does not. Here we report the full-length cDNA of mouse trp2, the homologue of the human trp2 pseudogene. Mouse Trp2 is shown to be readily activated not only after stimulation with an agonist but also by store depletion in the absence of an agonist. In contrast to other Trp proteins, Trp2-mediated Ca2+ entry activated by store depletion is seen under the same conditions that reveal endogenous store depletion-activated Ca2+ entry, i.e., classical CCE. The findings support the general hypothesis that Trp proteins are subunits of store- and receptor-operated Ca2+ channels.

TRPC1, a human homolog of a Drosophila store-operated channel.

In many vertebrate and invertebrate cells, inositol 1,4,5-trisphospate production induces a biphasic Ca2+ signal. Mobilization of Ca2+ from internal stores drives the initial burst. The second phase, referred to as store-operated Ca2+ entry (formerly capacitative Ca2+ entry), occurs when depletion of intracellular Ca2+ pools activates a non-voltage-sensitive plasma membrane Ca2+ conductance. Despite the prevalence of store-operated Ca2+ entry, no vertebrate channel responsible for store-operated Ca2+ entry has been reported. trp (transient receptor potential), a Drosophila gene required in phototransduction, encodes the only known candidate for such a channel throughout phylogeny. In this report, we describe the molecular characterization of a human homolog of trp, TRPC1. TRPC1 (transient receptor potential channel-related protein 1) was 40% identical to Drosophila TRP over most of the protein and lacked the charged residues in the S4 transmembrane region proposed to be required for the voltage sensor in many voltage-gated ion channels. TRPC1 was expressed at the highest levels in the fetal brain and in the adult heart, brain, testis, and ovaries. Evidence is also presented that TRPC1 represents the archetype of a family of related human proteins.

Department-store education: an acount of the training methods developed at the Boston School of Salemanship under the direction of Lucinda Wyman Prince

by Helen Rich Norton.

Lake sediments from Store Koldewey, Northeast Greenland

A 2.9 m long sedimentary record was studied from a small lake, here referred to as Duck Lake, located at 76°25'N, 18°45'W on Store Koldewey, an elongated island off the coast of Northeast Greenland. The sediments were investigated for their geophysical and biogeochemical characteristics, and for their fossil chironomid assemblages. Organic matter began to accumulate in the lake at 9.1 cal. kyr BP, which provides a minimum age for the deglaciation of the basin. Although the early to mid-Holocene is known as a thermal maximum in East Greenland, organic matter accumulation in the lake remained low during the early Holocene, likely due to late plant immigration and lack of nutrient availability. Organic matter accumulation increased during the middle and late Holocene, when temperatures in East Greenland gradually decreased. Enhanced soil formation probably led to higher nutrient availability and increased production in the lake. Chironomids are abundant throughout the record after 9.1 cal. kyr BP and seem to react sensitively to changes in temperature and nutrient availability. It is concluded that relative temperature reconstructions based on biogeochemical data have to be regarded critically, particularly in the period shortly after deglaciation when nutrient availability was low. Chironomids may be a suitable tool for climatic reconstructions even in those high arctic environments. However, a better understanding of the ecology of chironomids under these extreme conditions is needed.