Generadores ScalaCheck para property-based testing de programas Spark y Spark Streaming

En los últimos años hemos sido testigos de la expansión del paradigma big data a una velocidad vertiginosa. Los cambios en este campo, nos permiten ampliar las áreas a tratar; lo que a su vez implica una mayor complejidad de los sistemas software asociados a estas tareas, como sucede en sistemas de monitorización o en el Internet de las Cosas (Internet of Things). Asimismo, la necesidad de implementar programas cada vez robustos y eficientes, es decir, que permitan el cómputo de datos a mayor velocidad y de los se obtengan información relevante, ahorrando costes y tiempo, ha propiciado la necesidad cada vez mayor de herramientas que permitan evaluar estos programas. En este contexto, el presente proyecto se centra en extender la herramienta sscheck. Sscheck permite la generación de casos de prueba basados en propiedades de programas escritos en Spark y Spark Streaming. Estos lenguajes forman parte de un mismo marco de código abierto para la computación distribuida en clúster. Dado que las pruebas basadas en propiedades generan datos aleatorios, es difícil reproducir los problemas encontrados en una cierta sesion; por ello, la extensión se centrará en cargar y guardar casos de test en disco mediante el muestreo de datos desde colecciones mayores.

Now Streaming: A Consortial PDA Video Pilot Project

In 2014, eight academic libraries in the state of New York collaborated on a group patron driven acquisition (PDA) pilot program with Kanopy, a video streaming service for libraries. The institutions, despite vast differences in size and profile, each launched Kanopy’s streaming solution on their campuses under a program where they would jointly contribute to and acquire films based on group usage. The pilot ran for seven months and led to some fascinating insights into the differences in demand for film across campuses, the possibility of PDA as a model for library acquisition, and the feasibility of a group approach to acquisition. This paper presents the background to, results of, and reflections on the pilot program from the three unique perspectives of the consortium, the vendor, and one of the libraries involved, providing a holistic view of the success of the pilot and the lessons learned.

Sviluppo di un'applicazione di sincronizzazione file per reti challenged

Questa tesi si pone l'obiettivo di implementare in ambiente Linux un'applicazione di sincronizzazione, chiamata DTNbox, che permetta lo scambio di file tra due nodi di una rete classificabile come Delay-/Disruption-Tolerant Network (DTN), ossia una rete in cui a causa di ritardi, interruzioni, partizionamento, non sia possibile utilizzare l'usuale architettura di rete TCP/IP. E' evidente che i problemi menzionati rendono estremamente più complessa la sincronizzazione fra cartelle rispetto ad Internet, da cui le peculiarità di DTNbox rispetto ad altre applicazioni in rete visto che, ad esempio, non è possibile la sincronizzazione tramite un nodo centrale, come in Dropbox e similari, ma occorre basarsi su comunicazioni peer-to-peer. L'oggetto della mia tesi si è quindi sviluppato principalmente su tre direzioni: • Implementare, utilizzando il linguaggio di programmazione C, le funzionalità previste dal nuovo progetto per Linux • Integrarne e modificarne le parti ritenute carenti, man mano che i test parziali ne hanno mostrato la necessità • Testarne il suo corretto funzionamento Si è deciso pertanto di dare precedenza alla scrittura delle parti fondamentali del programma quali i moduli di controllo, la struttura e gestione del database e lo scambio di messaggi tra due nodi appartenenti ad una rete DTN per poter arrivare ad una prima versione funzionante del programma stesso, in modo che eventuali future tesi possano concentrarsi sullo sviluppo di una interfaccia grafica e sull'aggiunta di nuovi comandi e funzionalità accessorie. Il programma realizzato è stato poi testato su macchine virtuali grazie all'uso dello strumento Virtualbricks.

High-speed UDP Data Transmission with Multithreading and Automatic Resource Allocation

In this paper a utilization of the high data-rates channels by threading of sending and receiving is studied. As a communication technology evolves the higher speeds are used more and more in various applications. But generating traffic with Gbps data-rates also brings some complications. Especially if UDP protocol is used and it is necessary to avoid packet fragmentation, for example for high-speed reliable transport protocols based on UDP. For such situation the Ethernet network packet size has to correspond to standard 1500 bytes MTU[1], which is widely used in the Internet. System may not has enough capacity to send messages with necessary rate in a single-threaded mode. A possible solution is to use more threads. It can be efficient on widespread multicore systems. Also the fact that in real network non-constant data flow can be expected brings another object of study –- an automatic adaptation to the traffic which is changing during runtime. Cases investigated in this paper include adjusting number of threads to a given speed and keeping speed on a given rate when CPU gets heavily loaded by other processes while sending data.

Development of a streaming-fiber oil spill control system, Stage II /

Final report, issued December 1976.

Development of a streaming-fiber oil spill control system : stage II : modifications to large-scale model /

Includes bibliographical references.

Think global, act global: Corporate content streaming and Australian media policy

Australia ’s media policy agenda has recently been dominated by debate over two key issues: media ownership reform, and the local content provisions of the Australia–United States Free Trade Agreement. Challenging the tendency to analyse these issues separately, the article considers them as interlinked indicators of fundamental shifts occurring in the digital media environment. Converged media corporations increasingly seek to achieve economies of scale through ‘content streaming’: multi-purposing proprietary content across numerous digitally enabled platforms. This has resulted in rivalries for control of delivery technologies (as witnessed in media ownership debates) as well as over market access for corporate content (in the case of local content debates). The article contextualises Australia’s contemporary media policy flashpoints within international developments and longer-term industry strategising. It further questions the power of media policy as it is currently conceived to deal adequately with the challenges raised by a converging digital media marketplace.

Sheet flow sediment transport under waves with acceleration skewness and boundary layer streaming

The effect of acceleration skewness on sheet flow sediment transport rates (q) over bar (s) is analysed using new data which have acceleration skewness and superimposed currents but no boundary layer streaming. Sediment mobilizing forces due to drag and to acceleration (similar to pressure gradients) are weighted by cosine and sine, respectively, of the angle phi(.)(tau)phi(tau) = 0 thus corresponds to drag dominated sediment transport, (q) over bar (s)similar to vertical bar u(infinity)vertical bar u(infinity), while phi(tau) = 90 degrees corresponds to total domination by the pressure gradients, (q) over bar similar to du(infinity)/dt. Using the optimal angle, phi = 51 degrees based on that data, good agreement is subsequently found with data that have strong influence from boundary layer streaming. Good agreement is also maintained with the large body of U-tube data simulating sine waves with superimposed currents and second-order Stokes waves, all of which have zero acceleration skewness. The recommended model can be applied to irregular waves with arbitrary shape as long as the assumption negligible time lag between forcing and sediment transport rate is valid. With respect to irregular waves, the model is much easier to apply than the competing wave-by-wave models. Issues for further model developments are identified through a comprehensive data review.