1 resultado para DATA STORAGE
em DigitalCommons@University of Nebraska - Lincoln
Filtro por publicador
- JISC Information Environment Repository (2)
- Aberdeen University (7)
- Academic Archive On-line (Stockholm University; Sweden) (1)
- AMS Tesi di Dottorato - Alm@DL - Università di Bologna (8)
- AMS Tesi di Laurea - Alm@DL - Università di Bologna (8)
- Aquatic Commons (6)
- ArchiMeD - Elektronische Publikationen der Universität Mainz - Alemanha (6)
- Archimer: Archive de l'Institut francais de recherche pour l'exploitation de la mer (3)
- Archivo Digital para la Docencia y la Investigación - Repositorio Institucional de la Universidad del País Vasco (4)
- Aston University Research Archive (16)
- B-Digital - Universidade Fernando Pessoa - Portugal (1)
- Biblioteca de Teses e Dissertações da USP (2)
- Biblioteca Digital da Produção Intelectual da Universidade de São Paulo (10)
- Biblioteca Digital da Produção Intelectual da Universidade de São Paulo (BDPI/USP) (6)
- Biblioteca Digital de Teses e Dissertações Eletrônicas da UERJ (1)
- Bioline International (1)
- BORIS: Bern Open Repository and Information System - Berna - Suiça (7)
- Boston University Digital Common (3)
- Bucknell University Digital Commons - Pensilvania - USA (1)
- Bulgarian Digital Mathematics Library at IMI-BAS (2)
- CaltechTHESIS (2)
- Cambridge University Engineering Department Publications Database (5)
- CentAUR: Central Archive University of Reading - UK (21)
- Chinese Academy of Sciences Institutional Repositories Grid Portal (46)
- Cochin University of Science & Technology (CUSAT), India (8)
- Coffee Science - Universidade Federal de Lavras (1)
- Collection Of Biostatistics Research Archive (1)
- Comissão Econômica para a América Latina e o Caribe (CEPAL) (1)
- CORA - Cork Open Research Archive - University College Cork - Ireland (3)
- CUNY Academic Works (1)
- Dalarna University College Electronic Archive (5)
- Deakin Research Online - Australia (47)
- Department of Computer Science E-Repository - King's College London, Strand, London (1)
- Digital Commons - Michigan Tech (2)
- Digital Commons @ Winthrop University (1)
- Digital Commons at Florida International University (19)
- Digital Peer Publishing (1)
- DigitalCommons@The Texas Medical Center (6)
- DigitalCommons@University of Nebraska - Lincoln (1)
- Doria (National Library of Finland DSpace Services) - National Library of Finland, Finland (3)
- DRUM (Digital Repository at the University of Maryland) (1)
- Duke University (2)
- eResearch Archive - Queensland Department of Agriculture; Fisheries and Forestry (2)
- Glasgow Theses Service (1)
- Greenwich Academic Literature Archive - UK (3)
- Helda - Digital Repository of University of Helsinki (7)
- Indian Institute of Science - Bangalore - Índia (39)
- Instituto Politécnico de Castelo Branco - Portugal (1)
- Instituto Politécnico do Porto, Portugal (1)
- Lume - Repositório Digital da Universidade Federal do Rio Grande do Sul (1)
- Martin Luther Universitat Halle Wittenberg, Germany (1)
- Massachusetts Institute of Technology (5)
- Memoria Académica - FaHCE, UNLP - Argentina (3)
- National Center for Biotechnology Information - NCBI (5)
- Open University Netherlands (1)
- Plymouth Marine Science Electronic Archive (PlyMSEA) (4)
- Publishing Network for Geoscientific & Environmental Data (37)
- QSpace: Queen's University - Canada (2)
- QUB Research Portal - Research Directory and Institutional Repository for Queen's University Belfast (20)
- Queensland University of Technology - ePrints Archive (407)
- Repositório Científico do Instituto Politécnico de Lisboa - Portugal (1)
- Repositório digital da Fundação Getúlio Vargas - FGV (3)
- Repositório Digital da UNIVERSIDADE DA MADEIRA - Portugal (1)
- Repositório Institucional da Universidade de Aveiro - Portugal (1)
- Repositório Institucional da Universidade Estadual de São Paulo - UNESP (1)
- Repositório Institucional UNESP - Universidade Estadual Paulista "Julio de Mesquita Filho" (40)
- RUN (Repositório da Universidade Nova de Lisboa) - FCT (Faculdade de Cienecias e Technologia), Universidade Nova de Lisboa (UNL), Portugal (1)
- SAPIENTIA - Universidade do Algarve - Portugal (2)
- Universidad de Alicante (2)
- Universidad Politécnica de Madrid (27)
- Universidade Federal do Pará (1)
- Universidade Federal do Rio Grande do Norte (UFRN) (3)
- Universidade Metodista de São Paulo (1)
- Universitat de Girona, Spain (1)
- Universitätsbibliothek Kassel, Universität Kassel, Germany (4)
- Université de Lausanne, Switzerland (2)
- Université de Montréal, Canada (2)
- Université Laval Mémoires et thèses électroniques (1)
- University of Connecticut - USA (1)
- University of Michigan (19)
- University of Queensland eSpace - Australia (15)
- University of Southampton, United Kingdom (3)
- University of Washington (4)
- WestminsterResearch - UK (3)
Resumo:
Hundreds of Terabytes of CMS (Compact Muon Solenoid) data are being accumulated for storage day by day at the University of Nebraska-Lincoln, which is one of the eight US CMS Tier-2 sites. Managing this data includes retaining useful CMS data sets and clearing storage space for newly arriving data by deleting less useful data sets. This is an important task that is currently being done manually and it requires a large amount of time. The overall objective of this study was to develop a methodology to help identify the data sets to be deleted when there is a requirement for storage space. CMS data is stored using HDFS (Hadoop Distributed File System). HDFS logs give information regarding file access operations. Hadoop MapReduce was used to feed information in these logs to Support Vector Machines (SVMs), a machine learning algorithm applicable to classification and regression which is used in this Thesis to develop a classifier. Time elapsed in data set classification by this method is dependent on the size of the input HDFS log file since the algorithmic complexities of Hadoop MapReduce algorithms here are O(n). The SVM methodology produces a list of data sets for deletion along with their respective sizes. This methodology was also compared with a heuristic called Retention Cost which was calculated using size of the data set and the time since its last access to help decide how useful a data set is. Accuracies of both were compared by calculating the percentage of data sets predicted for deletion which were accessed at a later instance of time. Our methodology using SVMs proved to be more accurate than using the Retention Cost heuristic. This methodology could be used to solve similar problems involving other large data sets.