AIDA-Based Distributed File System

This paper describes a prototype implementation of a Distributed File System (DFS) based on the Adaptive Information Dispersal Algorithm (AIDA). Using AIDA, a file block is encoded and dispersed into smaller blocks stored on a number of DFS nodes distributed over a network. The implementation devises file creation, read, and write operations. In particular, when reading a file, the DFS accepts an optional timing constraint, which it uses to determine the level of redundancy needed for the read operation. The tighter the timing constraint, the more nodes in the DFS are queried for encoded blocks. Write operations update all blocks in all DFS nodes--with future implementations possibly including the use of read and write quorums. This work was conducted under the supervision of Professor Azer Bestavros (best@cs.bu.edu) in the Computer Science Department as part of Mohammad Makarechian's Master's project.

The application of distributed virtual machines for enterprise computer management: A two-tier network file system for image provisioning and management

In order to simplify computer management, several system administrators are adopting advanced techniques to manage software configuration of enterprise computer networks, but the tight coupling between hardware and software makes every PC an individual managed entity, lowering the scalability and increasing the costs to manage hundreds or thousands of PCs. Virtualization is an established technology, however its use is been more focused on server consolidation and virtual desktop infrastructure, not for managing distributed computers over a network. This paper discusses the feasibility of the Distributed Virtual Machine Environment, a new approach for enterprise computer management that combines virtualization and distributed system architecture as the basis of the management architecture. © 2008 IEEE.

Sincronizzazione e condivisione di file system tra sistemi operativi

La necessità di sincronizzare i propri dati si presenta in una moltitudine di situazioni, infatti il numero di dispositivi informatici a nostra disposizione è in continua crescita e, all' aumentare del loro numero, cresce l' esigenza di mantenere aggiornate le multiple copie dei dati in essi memorizzati. Vi sono diversi fattori che complicano tale situazione, tra questi la varietà sempre maggiore dei sistemi operativi utilizzati nei diversi dispositivi, si parla di Microsoft Windows, delle tante distribuzioni Linux, di Mac OS X, di Solaris o di altri sistemi operativi UNIX, senza contare i sistemi operativi più orientati al settore mobile come Android. Ogni sistema operativo ha inoltre un modo particolare di gestire i dati, si pensi alla differente gestione dei permessi dei file o alla sensibilità alle maiuscole. Bisogna anche considerare che se gli aggiornamenti dei dati avvenissero soltanto su di uno di questi dispositivi sarebbe richiesta una semplice copia dei dati aggiornati sugli altri dispositivi, ma che non è sempre possibile utilizzare tale approccio. Infatti i dati vengono spesso aggiornati in maniera indipendente in più di un dispositivo, magari nello stesso momento, è pertanto necessario che le applicazioni che si occupano di sincronizzare tali dati riconoscano le situazioni di conflitto, nelle quali gli stessi dati sono stati aggiornati in più di una copia ed in maniera differente, e permettano di risolverle, uniformando lo stato delle repliche. Considerando l' importanza e il valore che possono avere i dati, sia a livello lavorativo che personale, è necessario che tali applicazioni possano garantirne la sicurezza, evitando in ogni caso un loro danneggiamento, perchè sempre più spesso il valore di un dispositivo dipende più dai dati in esso contenuti che dal costo dello hardware. In questa tesi verranno illustrate alcune idee alternative su come possa aver luogo la condivisione e la sincronizzazione di dati tra sistemi operativi diversi, sia nel caso in cui siano installati nello stesso dispositivo che tra dispositivi differenti. La prima parte della tesi descriverà nel dettaglio l' applicativo Unison. Tale applicazione, consente di mantenere sincronizzate tra di loro repliche dei dati, memorizzate in diversi dispositivi che possono anche eseguire sistemi operativi differenti. Unison funziona a livello utente, analizzando separatamente lo stato delle repliche al momento dell' esecuzione, senza cioè mantenere traccia delle operazioni che sono state effettuate sui dati per modificarli dal loro stato precedente a quello attuale. Unison permette la sincronizzazione anche quando i dati siano stati modificati in maniera indipendente su più di un dispositivo, occupandosi di risolvere gli eventuali conflitti che possono verificarsi rispettando la volontà dell' utente. Verranno messe in evidenza le strategie utilizzate dai suoi ideatori per garantire la sicurezza dei dati ad esso affidati e come queste abbiano effetto nelle più diverse condizioni. Verrà poi fornita un' analisi dettagiata di come possa essere utilizzata l' applicazione, fornendo una descrizione accurata delle funzionalità e vari esempi per renderne più chiaro il funzionamento. Nella seconda parte della tesi si descriverà invece come condividere file system tra sistemi operativi diversi all' interno della stessa macchina, si tratta di un approccio diametralmente opposto al precedente, in cui al posto di avere una singola copia dei dati, si manteneva una replica per ogni dispositivo coinvolto. Concentrando l' attenzione sui sistemi operativi Linux e Microsoft Windows verranno descritti approfonditamente gli strumenti utilizzati e illustrate le caratteristiche tecniche sottostanti.

FS.js, una libreria file system per Node.js e HTML5

La tesi illustra le funzionalita e l'architettura di Node.js elencando e analizzando le caratteristiche che lo rendono un framework vincente nella sfida che il web attuale pone. La tesi comprende l'analisi e la descrizione del lavoro svolto per creare una libreria HTTP/ File system, integrata nel sistema di sviluppo cloud proprietario: Instant Developer , funzionante sia su Node.JS che sui browser che supportano appieno le nuove API File system di HTML 5. Particolare attenzione viene riservata per la descrizione della struttura della libreria, pensata per permettere all'utente dell'IDE di utilizzarla indifferentemente su server/browser senza preoccuparsi di chiamare metodi diversi. Fs.js permette di operare con file/cartelle, richieste HTTP in modo semplificato rispetto alle API Ufficiali dei rispettivi ambienti.

A FIREWALL MODEL OF FILE SYSTEM SECURITY

File system security is fundamental to the security of UNIX and Linux systems since in these systems almost everything is in the form of a file. To protect the system files and other sensitive user files from unauthorized accesses, certain security schemes are chosen and used by different organizations in their computer systems. A file system security model provides a formal description of a protection system. Each security model is associated with specified security policies which focus on one or more of the security principles: confidentiality, integrity and availability. The security policy is not only about “who” can access an object, but also about “how” a subject can access an object. To enforce the security policies, each access request is checked against the specified policies to decide whether it is allowed or rejected. The current protection schemes in UNIX/Linux systems focus on the access control. Besides the basic access control scheme of the system itself, which includes permission bits, setuid and seteuid mechanism and the root, there are other protection models, such as Capabilities, Domain Type Enforcement (DTE) and Role-Based Access Control (RBAC), supported and used in certain organizations. These models protect the confidentiality of the data directly. The integrity of the data is protected indirectly by only allowing trusted users to operate on the objects. The access control decisions of these models depend on either the identity of the user or the attributes of the process the user can execute, and the attributes of the objects. Adoption of these sophisticated models has been slow; this is likely due to the enormous complexity of specifying controls over a large file system and the need for system administrators to learn a new paradigm for file protection. We propose a new security model: file system firewall. It is an adoption of the familiar network firewall protection model, used to control the data that flows between networked computers, toward file system protection. This model can support decisions of access control based on any system generated attributes about the access requests, e.g., time of day. The access control decisions are not on one entity, such as the account in traditional discretionary access control or the domain name in DTE. In file system firewall, the access decisions are made upon situations on multiple entities. A situation is programmable with predicates on the attributes of subject, object and the system. File system firewall specifies the appropriate actions on these situations. We implemented the prototype of file system firewall on SUSE Linux. Preliminary results of performance tests on the prototype indicate that the runtime overhead is acceptable. We compared file system firewall with TE in SELinux to show that firewall model can accommodate many other access control models. Finally, we show the ease of use of firewall model. When firewall system is restricted to specified part of the system, all the other resources are not affected. This enables a relatively smooth adoption. This fact and that it is a familiar model to system administrators will facilitate adoption and correct use. The user study we conducted on traditional UNIX access control, SELinux and file system firewall confirmed that. The beginner users found it easier to use and faster to learn then traditional UNIX access control scheme and SELinux.

Geographic base file system--uses, maintenance, problem solving : conference proceedings, November 16-17, 1971, Arlington, Tex.

Issued May 1972; revised November 1976.

The Design and Implementation of a SD Card File System

SD card (Secure Digital Memory Card) is widely used in portable storage medium. Currently, latest researches on SD card, are mainly SD card controller based on FPGA (Field Programmable Gate Array). Most of them are relying on API interface (Application Programming Interface), AHB bus (Advanced High performance Bus), etc. They are dedicated to the realization of ultra high speed communication between SD card and upper systems. Studies about SD card controller, really play a vital role in the field of high speed cameras and other sub-areas of expertise. This design of FPGA-based file systems and SD2.0 IP (Intellectual Property core) does not only exhibit a nice transmission rate, but also achieve the systematic management of files, while retaining a strong portability and practicality. The file system design and implementation on a SD card covers the main three IP innovation points. First, the combination and integration of file system and SD card controller, makes the overall system highly integrated and practical. The popular SD2.0 protocol is implemented for communication channels. Pure digital logic design based on VHDL (Very-High-Speed Integrated Circuit Hardware Description Language), integrates the SD card controller in hardware layer and the FAT32 file system for the entire system. Secondly, the document management system mechanism makes document processing more convenient and easy. Especially for small files in batch processing, it can ease the pressure of upper system to frequently access and process them, thereby enhancing the overall efficiency of systems. Finally, digital design ensures the superior performance. For transmission security, CRC (Cyclic Redundancy Check) algorithm is for data transmission protection. Design of each module is platform-independent of macro cells, and keeps a better portability. Custom integrated instructions and interfaces may facilitate easily to use. Finally, the actual test went through multi-platform method, Xilinx and Altera FPGA developing platforms. The timing simulation and debugging of each module was covered. Finally, Test results show that the designed FPGA-based file system IP on SD card can support SD card, TF card and Micro SD with 2.0 protocols, and the successful implementation of systematic management for stored files, and supports SD bus mode. Data read and write rates in Kingston class10 card is approximately 24.27MB/s and 16.94MB/s.

Understanding file access mechanisms for embedded ubicomp collaboration interfaces

This paper explores the nature of interfaces to support people in accessing their files at tabletop displays embedded in the environment. To do this, we designed a study comparing people's interaction with two very different classes of file system access interface: Focus, explicitly designed for tabletops, and the familiar hierarchical Windows Explorer. In our within-subjects double-crossover study, participants collaborated on 4 planning tasks. Based on video, logs, questionnaires and interviews, we conclude that both classes of interface have a place. Notably, Focus contributed to improved collaboration and more efficient use of the workspace than with Explorer. Our results inform a set of recommendations for future interfaces enabling this important class of interaction -- supporting access to files for collaboration at tabletop devices embedded in an ubicomp environment.

Discovery of Application Workloads from Network File Traces

An understanding of application I/O access patterns is useful in several situations. First, gaining insight into what applications are doing with their data at a semantic level helps in designing efficient storage systems. Second, it helps create benchmarks that mimic realistic application behavior closely. Third, it enables autonomic systems as the information obtained can be used to adapt the system in a closed loop.All these use cases require the ability to extract the application-level semantics of I/O operations. Methods such as modifying application code to associate I/O operations with semantic tags are intrusive. It is well known that network file system traces are an important source of information that can be obtained non-intrusively and analyzed either online or offline. These traces are a sequence of primitive file system operations and their parameters. Simple counting, statistical analysis or deterministic search techniques are inadequate for discovering application-level semantics in the general case, because of the inherent variation and noise in realistic traces.In this paper, we describe a trace analysis methodology based on Profile Hidden Markov Models. We show that the methodology has powerful discriminatory capabilities that enable it to recognize applications based on the patterns in the traces, and to mark out regions in a long trace that encapsulate sets of primitive operations that represent higher-level application actions. It is robust enough that it can work around discrepancies between training and target traces such as in length and interleaving with other operations. We demonstrate the feasibility of recognizing patterns based on a small sampling of the trace, enabling faster trace analysis. Preliminary experiments show that the method is capable of learning accurate profile models on live traces in an online setting. We present a detailed evaluation of this methodology in a UNIX environment using NFS traces of selected commonly used applications such as compilations as well as on industrial strength benchmarks such as TPC-C and Postmark, and discuss its capabilities and limitations in the context of the use cases mentioned above.

Managing groups of files in a rule oriented data management system (iRODS)

The iRODS system, created by the San Diego Supercomputing Centre, is a rule oriented data management system that allows the user to create sets of rules to define how the data is to be managed. Each rule corresponds to a particular action or operation (such as checksumming a file) and the system is flexible enough to allow the user to create new rules for new types of operations. The iRODS system can interface to any storage system (provided an iRODS driver is built for that system) and relies on its’ metadata catalogue to provide a virtual file-system that can handle files of any size and type. However, some storage systems (such as tape systems) do not handle small files efficiently and prefer small files to be packaged up (or “bundled”) into larger units. We have developed a system that can bundle small data files of any type into larger units - mounted collections. The system can create collection families and contains its’ own extensible metadata, including metadata on which family the collection belongs to. The mounted collection system can work standalone and is being incorporated into the iRODS system to enhance the systems flexibility to handle small files. In this paper we describe the motivation for creating a mounted collection system, its’ architecture and how it has been incorporated into the iRODS system. We describe different technologies used to create the mounted collection system and provide some performance numbers.

LoneStar : Design and evaluation of an energy-efficient, disk-based archival storage system

Bandlaufwerke waren bisher die vorherrschende Technologie, um die anfallenden Datenmengen in Archivsystemen zu speichern. Mit Zugriffsmustern, die immer aktiver werden, und Speichermedien wie Festplatten die kostenmäßig aufholen, muss die Architektur vor Speichersystemen zur Archivierung neu überdacht werden. Zuverlässigkeit, Integrität und Haltbarkeit sind die Haupteigenschaften der digitalen Archivierung. Allerdings nimmt auch die Zugriffsgeschwindigkeit einen erhöhten Stellenwert ein, wenn aktive Archive ihre gesamten Inhalte für den direkten Zugriff bereitstellen. Ein band-basiertes System kann die hierfür benötigte Parallelität, Latenz und Durchsatz nicht liefern, was in der Regel durch festplattenbasierte Systeme als Zwischenspeicher kompensiert wird.rnIn dieser Arbeit untersuchen wir die Herausforderungen und Möglichkeiten ein festplattenbasiertes Speichersystem zu entwickeln, das auf eine hohe Zuverlässigkeit und Energieeffizienz zielt und das sich sowohl für aktive als auch für kalte Archivumgebungen eignet. Zuerst analysieren wir die Speichersysteme und Zugriffsmuster eines großen digitalen Archivs und präsentieren damit ein mögliches Einsatzgebiet für unsere Architektur. Daraufhin stellen wir Mechanismen vor um die Zuverlässigkeit einer einzelnen Festplatte zu verbessern und präsentieren sowie evaluieren einen neuen, energieeffizienten, zwei- dimensionalen RAID Ansatz der für „Schreibe ein Mal, lese mehrfach“ Zugriffe optimiert ist. Letztlich stellen wir Protokollierungs- und Zwischenspeichermechanismen vor, die die zugrundeliegenden Ziele unterstützen und evaluieren das RAID System in einer Dateisystemumgebung.