發(fā)布時(shí)間：2018-06-13 13:20

  本文選題：HDFS + BT��； 參考：《吉林大學(xué)》2013年碩士論文

【摘要】：隨著計(jì)算機(jī)與網(wǎng)絡(luò)的迅猛發(fā)展，數(shù)據(jù)量也與日激增，據(jù)思科在其《全球云數(shù)據(jù)報(bào)告中》指出，由于用戶和企業(yè)的不受限制地訪問及應(yīng)用數(shù)據(jù)的需求，在2010至2015年期間，全球云數(shù)據(jù)流量將以每年66%的速度增長(zhǎng)。傳統(tǒng)的分布式高性能處理平臺(tái)處理數(shù)據(jù)的能力已經(jīng)滿足不了井噴式增長(zhǎng)的數(shù)據(jù)處理請(qǐng)求。應(yīng)運(yùn)而生的云計(jì)算、云存儲(chǔ)則滿足此類數(shù)據(jù)密集性的服務(wù)請(qǐng)求。 本文研究?jī)?nèi)容是基于Hadoop的云系列服務(wù)系統(tǒng)的文件系統(tǒng)HDFS在其讀寫過程中傳輸策略的研究。旨在通過對(duì)HDFS讀寫等基本操作的效率提高及容錯(cuò)來實(shí)現(xiàn)更高效率及高度容錯(cuò)的文件系統(tǒng)。云存儲(chǔ)中的每個(gè)操作都離不開對(duì)文件系統(tǒng)的調(diào)用，對(duì)文件系統(tǒng)HDFS中最基本、最常用的讀寫操作的傳輸過程進(jìn)行研究并改進(jìn)，實(shí)現(xiàn)并行讀寫及高度容錯(cuò)，這會(huì)極大程度提高云存儲(chǔ)服務(wù)中數(shù)據(jù)的訪問速度及可用性問題。 本文首先介紹了云存儲(chǔ)的相關(guān)理論和技術(shù)，闡釋了云存儲(chǔ)的定義，并對(duì)其應(yīng)用場(chǎng)合加以描述。緊跟著又對(duì)HDFS進(jìn)行了透徹的剖析，并對(duì)其相關(guān)技術(shù)進(jìn)行了解析和比較。隨后對(duì)HDFS讀寫容錯(cuò)需要的技術(shù)進(jìn)行詳盡的描述，為接下來的研究提供的良好的技術(shù)保障。最后在通過研究HDFS讀寫機(jī)制進(jìn)行分析之后，對(duì)HDFS的讀寫機(jī)制中文件傳輸進(jìn)行改進(jìn)，實(shí)現(xiàn)文件數(shù)據(jù)塊級(jí)的并行傳輸功能。從而為云存儲(chǔ)的高延遲及副本安全性問題提供一個(gè)可靠的解決方案。 本文主要實(shí)現(xiàn)了云存儲(chǔ)文件系統(tǒng)的文件系統(tǒng)HDFS中文件讀寫過程中的并行傳輸策略，及改進(jìn)的副本自動(dòng)復(fù)制策略，提高了讀寫效率，降低延遲時(shí)間，為云存儲(chǔ)用戶提供高效并穩(wěn)定的服務(wù)。在本文的改進(jìn)策略下，充分利用了副本的存在，分散了網(wǎng)絡(luò)的負(fù)載，，數(shù)據(jù)讀的效率可提高160%，數(shù)據(jù)副本的復(fù)制效率也大大提高。
[Abstract]:With the rapid development of computers and networks, the amount of data is also increasing, according to Cisco in its Global Cloud data report, due to the unrestricted access and application data needs of users and enterprises, between 2010 and 2015, Global cloud data traffic will grow at an annual rate of 66%. The traditional distributed high performance processing platform can not meet the data processing requirements of blowout growth. Cloud computing comes into being and cloud storage meets such data-intensive service requests. This paper focuses on the research of HDFS file system based on Hadoop in the process of reading and writing. The purpose of this paper is to achieve a more efficient and highly fault-tolerant file system by improving the efficiency and fault tolerance of basic operations such as reading and writing HDFS. Every operation in cloud storage can not be separated from the call of file system. The transmission process of the most basic and commonly used read and write operations in file system HDFS is studied and improved to realize parallel reading and writing and highly fault-tolerant. This greatly improves the speed and availability of data access in cloud storage services. This paper first introduces the theory and technology of cloud storage, explains the definition of cloud storage, and describes its application. Then the HDFS is analyzed thoroughly, and the related technology is analyzed and compared. Then the technology needed for reading and writing fault tolerance in HDFS is described in detail, which provides a good technical guarantee for the following research. Finally, after analyzing the reading and writing mechanism of HDFS, the paper improves the file transmission in HDFS reading and writing mechanism, and realizes the parallel transfer function of file data block level. This provides a reliable solution for high latency and replica security problems in cloud storage. In this paper, the parallel transmission strategy in HDFS file system and the improved replica automatic replication strategy are implemented, which can improve the efficiency of reading and writing and reduce the delay time. Provide efficient and stable services for cloud storage users. Under the improved strategy of this paper, we make full use of the existence of replicas and spread the load of the network. The efficiency of data reading can be improved by 160%, and the efficiency of replication of data replicas can also be greatly improved.
【學(xué)位授予單位】：吉林大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333;TP309

【參考文獻(xiàn)】

相關(guān)期刊論文 前3條

1 樊景超;周國(guó)民;;基于Lucene的“農(nóng)搜”并行索引技術(shù)研究[J];農(nóng)業(yè)網(wǎng)絡(luò)信息;2009年08期

2 吳晟;蘇慶堂;羅斌;趙莉楠;蔡燦民;;基于Socket和多線程技術(shù)的并發(fā)服務(wù)器的研究[J];昆明理工大學(xué)學(xué)報(bào)(理工版);2006年04期

3 張樹剛;張遂南;黃士坦;;CRC校驗(yàn)碼并行計(jì)算的FPGA實(shí)現(xiàn)[J];計(jì)算機(jī)技術(shù)與發(fā)展;2007年02期

相關(guān)博士學(xué)位論文 前1條

1 宋瑋;分布式存儲(chǔ)系統(tǒng)中的節(jié)點(diǎn)自主性問題研究[D];華南理工大學(xué);2010年

相關(guān)碩士學(xué)位論文 前2條

1 張倩;基于Linux操作系統(tǒng)的嵌入式視頻監(jiān)控系統(tǒng)的研究與開發(fā)[D];天津工業(yè)大學(xué);2007年

2 陳華;Linux下BT客戶端的設(shè)計(jì)與實(shí)現(xiàn)[D];蘇州大學(xué);2009年

本文編號(hào)：2014151