Sviluppo di un modello di costo per Spark Sql

La quantità di dati che vengono generati e immagazzinati sta aumentando sempre più grazie alle nuove tecnologie e al numero di utenti sempre maggiore. Questi dati, elaborati correttamente, permettono quindi di ottenere delle informazioni di valore strategico che aiutano nell’effettuare decisioni aziendali a qualsiasi livello, dalla produzione fino al marketing. Sono nati soprattutto negli ultimi anni numerosi framework proprietari e open source che permettono l'elaborazione di questi dati sfruttando un cluster. In particolare tra i più utilizzati e attivi in questo momento a livello open source troviamo Hadoop e Spark. Obiettivo di questa tesi è realizzare un modello di Spark per realizzare una funzione di costo che sia non solo implementabile all’interno dell’ottimizzatore di Spark SQL, ma anche per poter effettuare delle simulazioni di esecuzione di query su tale sistema. Si è quindi studiato nel dettaglio con ducumentazione e test il comportamento del sistema per realizzare un modello. I dati ottenuti sono infine stati confrontati con dati sperimentali ottenuti tramite l'utilizzo di un cluster. Con la presenza di tale modello non solo risulta possibile comprendere in maniera più approfondita il reale comportamento di Spark ma permette anche di programmare applicazioni più efficienti e progettare con maggiore precisione sistemi per la gestione dei dataset che sfruttino tali framework.

Stochastic knock detection model for spark ignited engines

This report presents the development of a Stochastic Knock Detection (SKD) method for combustion knock detection in a spark-ignition engine using a model based design approach. Knock Signal Simulator (KSS) was developed as the plant model for the engine. The KSS as the plant model for the engine generates cycle-to-cycle accelerometer knock intensities following a stochastic approach with intensities that are generated using a Monte Carlo method from a lognormal distribution whose parameters have been predetermined from engine tests and dependent upon spark-timing, engine speed and load. The lognormal distribution has been shown to be a good approximation to the distribution of measured knock intensities over a range of engine conditions and spark-timings for multiple engines in previous studies. The SKD method is implemented in Knock Detection Module (KDM) which processes the knock intensities generated by KSS with a stochastic distribution estimation algorithm and outputs estimates of high and low knock intensity levels which characterize knock and reference level respectively. These estimates are then used to determine a knock factor which provides quantitative measure of knock level and can be used as a feedback signal to control engine knock. The knock factor is analyzed and compared with a traditional knock detection method to detect engine knock under various engine operating conditions. To verify the effectiveness of the SKD method, a knock controller was also developed and tested in a model-in-loop (MIL) system. The objective of the knock controller is to allow the engine to operate as close as possible to its border-line spark-timing without significant engine knock. The controller parameters were tuned to minimize the cycle-to-cycle variation in spark timing and the settling time of the controller in responding to step increase in spark advance resulting in the onset of engine knock. The simulation results showed that the combined system can be used adequately to model engine knock and evaluated knock control strategies for a wide range of engine operating conditions.

Parametric combustion modeling for ethanol-gasoline fuelled spark ignition engines

Ethanol-gasoline fuel blends are increasingly being used in spark ignition (SI) engines due to continued growth in renewable fuels as part of a growing renewable portfolio standard (RPS). This leads to the need for a simple and accurate ethanol-gasoline blends combustion model that is applicable to one-dimensional engine simulation. A parametric combustion model has been developed, integrated into an engine simulation tool, and validated using SI engine experimental data. The parametric combustion model was built inside a user compound in GT-Power. In this model, selected burn durations were computed using correlations as functions of physically based non-dimensional groups that have been developed using the experimental engine database over a wide range of ethanol-gasoline blends, engine geometries, and operating conditions. A coefficient of variance (COV) of gross indicated mean effective pressure (IMEP) correlation was also added to the parametric combustion model. This correlation enables the cycle combustion variation modeling as a function of engine geometry and operating conditions. The computed burn durations were then used to fit single and double Wiebe functions. The single-Wiebe parametric combustion compound used the least squares method to compute the single-Wiebe parameters, while the double-Wiebe parametric combustion compound used an analytical solution to compute the double-Wiebe parameters. These compounds were then integrated into the engine model in GT-Power through the multi-Wiebe combustion template in which the values of Wiebe parameters (single-Wiebe or double-Wiebe) were sensed via RLT-dependence. The parametric combustion models were validated by overlaying the simulated pressure trace from GT-Power on to experimentally measured pressure traces. A thermodynamic engine model was also developed to study the effect of fuel blends, engine geometries and operating conditions on both the burn durations and COV of gross IMEP simulation results.

EFFECT OF ELECTRICAL DISCHARGE PATTERN ON SPARK IGNITED FLAME KERNEL DEVELOPMENT

Fuel-lean combustion and exhaust gas recirculation (EGR) in spark ignition engines improve engine efficiency and reduce emission. However, flame initiation becomes more difficult in lean and dilute fuel-air mixture with traditional spark discharge. This research proposal will first provide an intensive review on topics related to spark ignition including properties of electrical discharge, flame kernel behavior and spark ignition modeling and simulation. Focus will be laid on electrical discharge pattern effect as it is showing prospect in extending ignition limits in SI engines. An experimental setup has been built with an optically accessible constant volume combustion vessel. Multiple imaging techniques as well as spectroscopy will be applied. By varying spark discharge patterns, preliminary test results are available on consequent flame kernel development. In addition to experimental investigation of spark plasma and flame kernel development, spark ignition modeling with detailed description of plasma channel is also proposed for this study.

DEVELOPMENT OF A PREDICTIVE COMBUSTION MODEL OF A SPARK IGNITED ENGINE WITH GASOLINE DIRECT INJECTION, VARIABLE VALVE TIMING, DURATION AND LIFT TECHNOLOGIES

There is a need by engine manufactures for computationally efficient and accurate predictive combustion modeling tools for integration in engine simulation software for the assessment of combustion system hardware designs and early development of engine calibrations. This thesis discusses the process for the development and validation of a combustion modeling tool for Gasoline Direct Injected Spark Ignited Engine with variable valve timing, lift and duration valvetrain hardware from experimental data. Data was correlated and regressed from accepted methods for calculating the turbulent flow and flame propagation characteristics for an internal combustion engine. A non-linear regression modeling method was utilized to develop a combustion model to determine the fuel mass burn rate at multiple points during the combustion process. The computational fluid dynamic software Converge ©, was used to simulate and correlate the 3-D combustion system, port and piston geometry to the turbulent flow development within the cylinder to properly predict the experimental data turbulent flow parameters through the intake, compression and expansion processes. The engine simulation software GT-Power © is then used to determine the 1-D flow characteristics of the engine hardware being tested to correlate the regressed combustion modeling tool to experimental data to determine accuracy. The results of the combustion modeling tool show accurate trends capturing the combustion sensitivities to turbulent flow, thermodynamic and internal residual effects with changes in intake and exhaust valve timing, lift and duration.

STUDY OF SPARK IGNITION ENGINE COMBUSTION MODEL FOR THE ANALYSIS OF CYCLIC VARIATION AND COMBUSTION STABILITY AT LEAN OPERATING CONDITIONS

A fundamental combustion model for spark-ignition engine is studied in this report. The model is implemented in SIMULINK to simulate engine outputs (mass fraction burn and in-cylinder pressure) under various engine operation conditions. The combustion model includes a turbulent propagation and eddy burning processes based on literature [1]. The turbulence propagation and eddy burning processes are simulated by zero-dimensional method and the flame is assumed as sphere. To predict pressure, temperature and other in-cylinder variables, a two-zone thermodynamic model is used. The predicted results of this model match well with the engine test data under various engine speeds, loads, spark ignition timings and air fuel mass ratios. The developed model is used to study cyclic variation and combustion stability at lean (or diluted) combustion conditions. Several variation sources are introduced into the combustion model to simulate engine performance observed in experimental data. The relations between combustion stability and the introduced variation amount are analyzed at various lean combustion levels.

IMPROVING STARTABILITY AND REDUCING EMISSIONS IN FLEXFUEL SPARK IGNITION DIRECT INJECTION VARIABLE CAM TIMING ENGINE

Experimental work and analysis was done to investigate engine startup robustness and emissions of a flex-fuel spark ignition (SI) direct injection (DI) engine. The vaporization and other characteristics of ethanol fuel blends present a challenge at engine startup. Strategies to reduce the enrichment requirements for the first engine startup cycle and emissions for the second and third fired cycle at 25°C ± 1°C engine and intake air temperature were investigated. Research work was conducted on a single cylinder SIDI engine with gasoline and E85 fuels, to study the effect on first fired cycle of engine startup. Piston configurations that included a compression ratio change (11 vs 15.5) and piston geometry change (flattop vs bowl) were tested, along with changes in intake cam timing (95,110,125) and fuel pressure (0.4 MPa vs 3 MPa). The goal was to replicate the engine speed, manifold pressure, fuel pressure and testing temperature from an engine startup trace for investigating the first fired cycle for the engine. Results showed bowl piston was able to enable lower equivalence ratio engine starts with gasoline fuel, while also showing lower IMEP at the same equivalence ratio compared to flat top piston. With E85, bowl piston showed reduced IMEP as compression ratio increased at the same equivalence ratio. A preference for constant intake valve timing across fuels seemed to indicate that flattop piston might be a good flex-fuel piston. Significant improvements were seen with higher CR bowl piston with high fuel pressure starts, but showed no improvement with low fuel pressures. Simulation work was conducted to analyze initial three cycles of engine startup in GT-POWER for the same set of hardware used in the experimentations. A steady state validated model was modified for startup conditions. The results of which allowed an understanding of the relative residual levels and IMEP at the test points in the cam phasing space. This allowed selecting additional test points that enable use of higher residual levels, eliminating those with smaller trapped mass incapable of producing required IMEP for proper engine turnover. The second phase of experimental testing results for 2nd and 3rd startup cycle revealed both E10 and E85 prefer the same SOI of 240°bTDC at second and third startup cycle for the flat top piston and high injection pressures. E85 fuel optimal cam timing for startup showed that it tolerates more residuals compared to E10 fuel. Higher internal residuals drives down the Ø requirement for both fuels up to their combustion stability limit, this is thought to be direct benefit to vaporization due to increased cycle start temperature. Benefits are shown for an advance IMOP and retarded EMOP strategy at engine startup. Overall the amount of residuals preferred by an engine for E10 fuel at startup is thought to be constant across engine speed, thus could enable easier selection of optimized cam positions across the startup speeds.

Influence of the composition of natural gas-hydrogen blends (HCNG) on performance an CO2 emissions of a spark ignition engine

Addition of hydrogen to natural gas could be a short-term alternative to nowadays fossil fuels as the emissions of greenhouse gases may be reduced. The aim of this study is to evaluate the performance and emissions of a park ignition engine fuelled with pure natural gas, pure hydrogen and different blends of hydrogen and natural gas (HCNG). The increase of the hydrogen fraction leads to variations in the cylinder pressure and CO2 emissions. In this work, a combustion model based on thermodynamic equations is used considering separated zones for the burned and unburned gases. The results show that the maximum cylinder pressure gets higher as the fraction of hydrogen in the blend increases. The presence of hydrogen in the blend leads to a drecrease in the CO2 emissions. Due to hydrogen properties, leaner fuel-air mixtures can be used along with the appropiate spark timing, leading to an engine emissions improvement without a performance worsening.

CO2 emissions from a spark ignition engine operatin on natural-hydrogen blends (HCNG)

The addition of hydrogen to natural gas could be a short-term alternative to today’s fossil fuels, as greenhouse gas emissions may be reduced. The aim of this study is to evaluate the emissions and performance of a spark ignition engine fuelled by pure natural gas, pure hydrogen, and different blends of hydrogen and natural gas (HCNG). Increasing the hydrogen fraction leads to variations in cylinder pressure and CO2 emissions. In this study, a combustion model based on thermodynamic equations is used, considering separate zones for burned and unburned gases. The results show that the maximum cylinder pressure rises as the fraction of hydrogen in the blend increases. The presence of hydrogen in the blend leads to a decrease in CO2 emissions. Due to the properties of hydrogen, leaner fuel–air mixtures can be used along with the appropriate spark timing, leading to an improvement in engine emissions with no loss of performance.

The Caustic Pen Is Mightiest: A Tradition of Female Satire in the Novels of Jane Austen, Ivy Compton-Burnett, and Muriel Spark

Female satirists have long been treated by critics as anomalies within an androcentric genre because of the reticence to acknowledge women's right to express aggression through their writing. In Pride and Prejudice (1813), A House and Its Head (1935), and The Girls of Slender Means (1963), Jane Austen (1775-1817), Ivy Compton-Burnett (1884-1969), and Muriel Spark (1918-2006) all combine elements of realism and satire within the vehicle of the domestic novel to target institutions of their patriarchal societies, including marriage and family dynamics, as well as the evolving conceptions of domesticity and femininity, with a subtle feminism. These female satirists illuminate the problems they have with society more through presentation than judgment in their satire, which places them on the fringes of a society they wish to educate, distinguishing their satire from that written by male satirists who are judging from a privileged height above the society they are attempting to correct. All three women create heroines and secondary female characters who find ways to survive, and occasionally thrive, within the confines of a polite society that has a streak of savagery running just beneath its polished surface.

Generadores ScalaCheck para property-based testing de programas Spark y Spark Streaming

En los últimos años hemos sido testigos de la expansión del paradigma big data a una velocidad vertiginosa. Los cambios en este campo, nos permiten ampliar las áreas a tratar; lo que a su vez implica una mayor complejidad de los sistemas software asociados a estas tareas, como sucede en sistemas de monitorización o en el Internet de las Cosas (Internet of Things). Asimismo, la necesidad de implementar programas cada vez robustos y eficientes, es decir, que permitan el cómputo de datos a mayor velocidad y de los se obtengan información relevante, ahorrando costes y tiempo, ha propiciado la necesidad cada vez mayor de herramientas que permitan evaluar estos programas. En este contexto, el presente proyecto se centra en extender la herramienta sscheck. Sscheck permite la generación de casos de prueba basados en propiedades de programas escritos en Spark y Spark Streaming. Estos lenguajes forman parte de un mismo marco de código abierto para la computación distribuida en clúster. Dado que las pruebas basadas en propiedades generan datos aleatorios, es difícil reproducir los problemas encontrados en una cierta sesion; por ello, la extensión se centrará en cargar y guardar casos de test en disco mediante el muestreo de datos desde colecciones mayores.

A tiny spark /

Poems.