發(fā)布時(shí)間：2018-06-10 19:00

  本文選題：分布式存儲系統(tǒng) + 數(shù)據(jù)一致性��； 參考：《電子科技大學(xué)》2013年碩士論文

【摘要】：隨著國家對下一代寬帶無線移動網(wǎng)絡(luò)建設(shè)的推進(jìn)，能夠滿足各種電信業(yè)務(wù)數(shù)據(jù)要求的移動控制網(wǎng)絡(luò)對后臺的存儲支持提出了越來越高的要求。傳統(tǒng)的存儲解決方案往往不能滿足系統(tǒng)和用戶對可擴(kuò)展性和可用性的需求，在面對海量用戶和海量文件數(shù)據(jù)的情況下更是捉襟見肘：系統(tǒng)規(guī)模無法適應(yīng)用戶與應(yīng)用規(guī)模的增長，應(yīng)對海量數(shù)據(jù)的處理與訪問也顯得力不從心。 本文首先對國內(nèi)外幾種主流的分布式文件系統(tǒng)進(jìn)行了調(diào)研與介紹，面對海量用戶與海量文件的應(yīng)用場景，設(shè)計(jì)并實(shí)現(xiàn)了分布式存儲系統(tǒng)CSTORE中的資源定位規(guī)則和數(shù)據(jù)存儲模塊。 CSTORE采用純分布式的架構(gòu)，將元數(shù)據(jù)與數(shù)據(jù)的分別存儲在不同的集群中，并實(shí)現(xiàn)了一種基于兩級映射的資源定位規(guī)則，結(jié)合負(fù)載均衡與數(shù)據(jù)恢復(fù)策略，能夠提供較好的可擴(kuò)展能力。元數(shù)據(jù)和數(shù)據(jù)的分離使得客戶端對兩種數(shù)據(jù)的訪問分開，便于它們各自的管理與性能的優(yōu)化；而集群的利用進(jìn)一步促進(jìn)了數(shù)據(jù)的分布式管理，避免了單點(diǎn)失效帶來的危險(xiǎn)。 資源定位規(guī)則建立了資源與存儲數(shù)據(jù)的服務(wù)器之間的映射關(guān)系，它為客戶端提供了一種可靠的尋址方法，系統(tǒng)中任何節(jié)點(diǎn)都可以使用它獨(dú)立計(jì)算出資源的位置，既充分利用了客戶端的資源，又減輕了服務(wù)器端的壓力；它的偽隨機(jī)特性和映射變換同時(shí)也為系統(tǒng)提供了較好的負(fù)載均衡和可擴(kuò)展性。 數(shù)據(jù)存儲模塊為用戶提供數(shù)據(jù)存儲與訪問服務(wù)，存儲到系統(tǒng)中的數(shù)據(jù)按資源定位規(guī)則被分為了多個不同的資源管理單元，資源管理單元的多個副本之間兩兩建立心跳連接以推送數(shù)據(jù)和交換時(shí)序，在一定程度上保證了數(shù)據(jù)的可用性的同時(shí)也保證了一致性；容錯方面，冗余機(jī)制保證了系統(tǒng)可以容忍在一定范圍內(nèi)的失效，數(shù)據(jù)恢復(fù)機(jī)制可以使得失效的模塊能夠盡快的對客戶端服務(wù)。在數(shù)據(jù)存儲模塊中所有文件片都使用全局唯一ID對所存儲的數(shù)據(jù)進(jìn)行了去重處理。數(shù)據(jù)去重可以節(jié)省存儲和管理成本，提高物理存儲空間利用率。 本文最后對存儲系統(tǒng)的功能和性能進(jìn)行了測試，證明了它能正確執(zhí)行所需功能并保證了其高效性。
[Abstract]:With the development of the next generation broadband wireless mobile network, the mobile control network, which can meet the data requirements of various telecommunication services, has put forward higher and higher requirements for the storage and support of the background. The traditional storage solutions often can not meet the scalability and usability requirements of the system and users, especially in the face of massive users and massive file data: the scale of the system can not adapt to the growth of users and applications. It is not enough to deal with the massive data processing and accessing. Firstly, this paper investigates and introduces several mainstream distributed file systems at home and abroad, facing the application scenarios of massive users and massive files. The resource location rules and data storage modules in CSTORE are designed and implemented. CSTORE uses a pure distributed architecture to store metadata and data in different clusters. A resource location rule based on two-level mapping is implemented, which combines load balancing and data recovery strategy to provide better scalability. The separation of metadata and data enables clients to access the two kinds of data separately to facilitate their respective management and performance optimization, while the utilization of clusters further promotes the distributed management of data. The resource location rules establish a mapping relationship between the resource and the server that stores data, which provides a reliable addressing method for the client. Any node in the system can use it to calculate the location of the resource independently, which not only makes full use of the resources of the client, but also reduces the pressure of the server. Its pseudorandom characteristics and mapping transformation also provide better load balancing and extensibility for the system. Data storage module provides users with data storage and access services. The data stored in the system is divided into a plurality of different resource management units according to the resource location rules. A heartbeat connection is established between a plurality of copies of the resource management unit to push the data and exchange time series. To a certain extent, the availability of the data is guaranteed, and the consistency is ensured. In the fault-tolerant aspect, the redundancy mechanism ensures that the system can tolerate the failure in a certain range. The data recovery mechanism can enable the invalid module to serve the client as soon as possible. In the data storage module, all the files are de-reprocessed using the global unique ID. Data removal can save storage and management costs and improve the utilization of physical storage space. Finally, the function and performance of the storage system are tested, which proves that it can correctly perform the required functions and ensure its high efficiency.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前5條

1 袁培森;皮德常;;用于內(nèi)存數(shù)據(jù)庫的Hash索引的設(shè)計(jì)與實(shí)現(xiàn)[J];計(jì)算機(jī)工程;2007年18期

2 吳蔚;網(wǎng)絡(luò)站點(diǎn)的負(fù)載均衡技術(shù)探索[J];計(jì)算機(jī)時(shí)代;2004年07期

3 韓德志;陳磊;;存儲網(wǎng)絡(luò)高可用系統(tǒng)設(shè)計(jì)關(guān)鍵技術(shù)研究[J];計(jì)算機(jī)應(yīng)用研究;2007年08期

4 敖莉;舒繼武;李明強(qiáng);;重復(fù)數(shù)據(jù)刪除技術(shù)[J];軟件學(xué)報(bào);2010年05期

5 薛軍,李增智,王云嵐;負(fù)載均衡技術(shù)的發(fā)展[J];小型微型計(jì)算機(jī)系統(tǒng);2003年12期

，

本文編號：2004264