分布式云計(jì)算資源配置技術(shù)研究
發(fā)布時(shí)間：2018-08-05 17:00
【摘要】：采用令用戶滿意的高效資源配置機(jī)制是云服務(wù)提供商取得成功的關(guān)鍵因素之一。在基礎(chǔ)設(shè)施即服務(wù)(Infrastructure act as Service,Iaa S)模型中,云服務(wù)提供商主要以虛擬機(jī)的形式對(duì)外提供服務(wù),因此資源配置問題也就歸結(jié)為在滿足業(yè)務(wù)需求前提下的虛擬機(jī)部署問題。本文首先依據(jù)虛擬機(jī)部署順序,采用自頂向下的方法,將分布式云計(jì)算中虛擬機(jī)部署分為三個(gè)階段,即云網(wǎng)絡(luò)選擇階段,數(shù)據(jù)中心選擇階段和服務(wù)器選擇階段。然后結(jié)合各階段中具體業(yè)務(wù)的不同需求,給出了針對(duì)性的資源配置策略。在云網(wǎng)絡(luò)選擇階段,云經(jīng)紀(jì)人(cloud broker)會(huì)利用長期資源預(yù)留的價(jià)格優(yōu)勢(shì)和資源統(tǒng)計(jì)復(fù)用帶來的增益,通過租賃公有云服務(wù)提供商的基礎(chǔ)設(shè)施來降低自己的成本。由于云服務(wù)提供商會(huì)提供多種不同的虛擬機(jī)預(yù)留實(shí)例周期(virtual machines reserved instance terms),而不同周期的實(shí)例價(jià)格不同,云經(jīng)紀(jì)人必須從多種實(shí)例周期中做出適當(dāng)選擇,以便采用最低成本資源滿足用戶的實(shí)時(shí)動(dòng)態(tài)需求。針對(duì)此目標(biāo),本文提出了最長實(shí)例周期優(yōu)選的逐層預(yù)留啟發(fā)式算法和基于集合覆蓋的近似算法來解決離線情況下的資源配置問題,并提出了基于歷史資源使用信息的在線資源配置算法。實(shí)驗(yàn)表明啟發(fā)式離線算法的運(yùn)行速度幾乎是近似離線算法的兩倍,相比全部使用實(shí)時(shí)虛擬機(jī)的方法,兩種離線算法均能節(jié)省大約27%的成本,在線算法能節(jié)省大約14%的成本。與已有的只考慮一種預(yù)留實(shí)例的算法相比,本文提出的算法更具有實(shí)用性。在數(shù)據(jù)中心選擇階段,本文研究了虛擬機(jī)部署時(shí)基于聚類的數(shù)據(jù)中心選擇和大數(shù)據(jù)向云端遷移時(shí)數(shù)據(jù)中心選擇兩個(gè)問題。第一個(gè)問題的目標(biāo)是優(yōu)化數(shù)據(jù)中心間通信時(shí)延和帶寬。為某大型任務(wù)或者某組織服務(wù)的一組相互通信的虛擬機(jī)可能會(huì)跨越多個(gè)數(shù)據(jù)中心,所以需要極小化最大的數(shù)據(jù)中心間的距離,以便極小化通信時(shí)延,同時(shí)節(jié)省昂貴的數(shù)據(jù)中心間的長途帶寬。本文首先提出了基于密度聚類的2-近似數(shù)據(jù)中心選擇算法。相比已有的只考慮數(shù)據(jù)中心間距離的方法,該算法能充分利用數(shù)據(jù)中心的組網(wǎng)拓?fù)鋵傩�、稠密屬性以及容量信�?效率提升了約15%～72%。然后建立了虛擬機(jī)半通信模型,并提出了基于該模型的虛擬機(jī)劃分算法將虛擬機(jī)分組并對(duì)應(yīng)到已選定的數(shù)據(jù)中心。實(shí)驗(yàn)表明,該算法不僅能進(jìn)一步減少數(shù)據(jù)中心間的帶寬消耗,而且運(yùn)行速度比已有算法快2倍左右。所有算法均適用于虛擬機(jī)同構(gòu)或異構(gòu)的場(chǎng)景,克服了已有算法只適用于虛擬機(jī)同構(gòu)場(chǎng)景的局限性。第二個(gè)問題的目標(biāo)是為分布式大數(shù)據(jù)選擇合適的數(shù)據(jù)中心,將數(shù)據(jù)遷移到云端,既可保證快速的本地?cái)?shù)據(jù)接入又能實(shí)現(xiàn)低成本數(shù)據(jù)遷移和處理。本文首先分析了大數(shù)據(jù)遷移時(shí)的四種目標(biāo):即公平數(shù)據(jù)放置、優(yōu)選數(shù)據(jù)放置、傳輸成本最小化數(shù)據(jù)放置和總成本最小化數(shù)據(jù)放置。然后采用二分圖對(duì)問題進(jìn)行了建模。針對(duì)前兩種目標(biāo),提出了一種基于組合優(yōu)化的緊的3-近似算法。針對(duì)后兩種目標(biāo),提出了一種最近數(shù)據(jù)中心優(yōu)先的啟發(fā)式算法。所給兩種算法能夠降低接入時(shí)延和成本,解決因法規(guī)限制或者用戶偏好引起的部分?jǐn)?shù)據(jù)中心不可用的問題,彌補(bǔ)了已有方法默認(rèn)全部數(shù)據(jù)中心均可用的不足。在服務(wù)器選擇階段,需要實(shí)現(xiàn)服務(wù)質(zhì)量可感知的低成本服務(wù)器選擇。服務(wù)器和組網(wǎng)占了數(shù)據(jù)中心總成本的60%左右。如何有效部署虛擬機(jī)以盡可能地節(jié)省成本,并保證服務(wù)質(zhì)量,對(duì)于提高云服務(wù)提供商的競爭力起著舉足輕重的作用�？紤]異構(gòu)服務(wù)器以及虛擬機(jī)多種資源需求的隨機(jī)性,該問題被形式化為一個(gè)多目標(biāo)非線性規(guī)劃。通過利用數(shù)據(jù)中心的組網(wǎng)拓?fù)湫畔?具有更大通信量的虛擬機(jī)被盡可能地部署在一起,減少了通信時(shí)延并節(jié)省了服務(wù)器之間帶寬消耗。同時(shí),統(tǒng)計(jì)復(fù)用和新定義的“相似性”方法被用來整合虛擬機(jī),資源容量違約被盡可能地限定在指定的極小概率內(nèi)。這樣的部署方案既不會(huì)影響服務(wù)質(zhì)量,又可以節(jié)省服務(wù)器和網(wǎng)絡(luò)成本。本文分別針對(duì)離線和在線場(chǎng)景給出了啟發(fā)式算法。與已有算法比較,實(shí)驗(yàn)顯示本文所給算法能更多地節(jié)省包括服務(wù)器,帶寬在內(nèi)的多種資源成本,而且運(yùn)行更快。本文最后考慮了跨數(shù)據(jù)中心大數(shù)據(jù)處理時(shí)數(shù)據(jù)中心和服務(wù)器的協(xié)同選擇問題�？紤]到將跨全球分布的數(shù)據(jù)全部遷移到同一個(gè)數(shù)據(jù)中心進(jìn)行分析不一定可行,而已廣為接受的大數(shù)據(jù)分析框架Hadoop卻只能處理數(shù)據(jù)中心內(nèi)部的數(shù)據(jù),因此有必要研究跨數(shù)據(jù)中心的大數(shù)據(jù)處理機(jī)制。本文提出了一種新的跨數(shù)據(jù)中心大數(shù)據(jù)處理架構(gòu)和一種基于鍵值(key value)的解決方案,該方案能盡量遵守傳統(tǒng)Hadoop的數(shù)據(jù)本地性原則,并能以更低的成本實(shí)現(xiàn)跨數(shù)據(jù)中心大數(shù)據(jù)處理。方案采用二層規(guī)劃對(duì)問題進(jìn)行建模,用定制的二層分組遺傳算法求解。實(shí)驗(yàn)驗(yàn)證了該方案的有效性,相比采用傳統(tǒng)Hadoop思想的方案和已有的最新方案,本文所提方案能分別節(jié)省成本大約49%和40%。
[Abstract]:It is one of the key factors for the success of the cloud service provider to adopt the efficient resource allocation mechanism which makes the user satisfied. In the Infrastructure act as Service (Iaa S) model, the cloud service provider provides the service mainly in the form of virtual machine, so the problem of resource allocation is also attributed to meeting the business needs. Under the premise of the virtual machine deployment problem, this paper, based on the virtual machine deployment order, uses the top-down method to divide the virtual machine deployment in distributed cloud computing into three stages, namely, the cloud network selection stage, the data center selection stage and the server selection stage, and then combines the different requirements of the specific services in each stage and gives the needle. The resource allocation strategy for sex. In the cloud network selection stage, cloud broker will reduce its cost by leasing the infrastructure of the Gong Youyun service provider to reduce its cost by using the price advantage reserved for long-term resources and the gain brought by the reuse of resource statistics. The instance cycle (virtual machines reserved instance terms), and the price of different instances of different cycles, the cloud broker must make the appropriate selection from a variety of instance cycles so that the minimum cost resources can be used to satisfy the user's real-time dynamic demand. An approximate algorithm based on set coverage is used to solve the problem of resource allocation under off-line conditions, and an online resource allocation algorithm based on the use of historical resources is proposed. The experiment shows that the running speed of the heuristic offline algorithm is almost two times as much as that of the off-line algorithm. Compared with the method of using real time virtual machines in the whole part, two off-line algorithms are used. The algorithm can save about 27% of the cost, and the online algorithm can save about 14% of the cost. Compared with the existing algorithm only considering a reserved instance, the proposed algorithm is more practical. In the data center selection stage, this paper studies the data center selection based on clustering and the migration of large data to the cloud at the data center selection stage. The center selects two problems. The first problem is to optimize the communication delay and bandwidth between data centers. A group of communication virtual machines for a large task or organization may cross multiple data centers, so it is necessary to minimize the distance between the largest data centers, so as to minimize communication delay and save a lot of time. In this paper, the 2- approximate data center selection algorithm based on density clustering is proposed in this paper. Compared with the existing methods that only consider the distance between data centers, the algorithm can make full use of the network topology attributes, dense attributes and capacity information of the data center, and the efficiency is raised about 15% to 72%. and then established. The virtual machine semi communication model is proposed, and a virtual machine partition algorithm based on this model is proposed to group the virtual machine and correspond to the selected data center. The experiment shows that the algorithm can not only reduce the bandwidth consumption between data centers, but also speed up to about 2 times faster than the existing algorithms. All algorithms are applicable to the isomorphism of virtual machines. The objective of the second problem is to select the appropriate data center for distributed large data and transfer the data to the cloud, which can not only guarantee fast local data access but also realize low cost data migration and processing. This paper first analyzes large data migration. The four objectives of time shift: fair data placement, optimal data placement, data placement and total cost minimization of data placement. Then a two division graph is used to model the problem. A compact 3- approximation algorithm based on combination optimization is proposed for the first two targets. A new kind of nearest two targets is proposed for the last two targets. The two algorithms can reduce the time delay and cost of access and solve the problems that are not available due to regulatory restrictions or user preferences, which make up for the shortcomings of the default all data centers available in the existing methods. A low cost server selection. Servers and networking account for about 60% of the total cost of the data center. How to effectively deploy virtual machines to save costs as much as possible and guarantee the quality of service plays an important role in improving the competitiveness of cloud service providers. The problem is formalized into a multi-objective nonlinear programming. By using the network topology information of the data center, the virtual machines with greater traffic are deployed as much as possible, reducing the communication delay and saving the bandwidth consumption between the servers. The default of resource capacity is limited to the specified minimum probability as much as possible. Such a deployment scheme does not affect the quality of service, but also saves the cost of the server and the network. This paper gives a heuristic algorithm for offline and online scenarios. Compared with the existing algorithms, experiments show that the proposed algorithm can save more. It includes a variety of resource costs, including the server and bandwidth, and runs faster. Finally, this paper considers the collaborative selection of data centers and servers during large data processing across data centers. Considering that all the data transmigrated across the global distribution to the same data center is not necessarily feasible and is widely accepted The analysis framework Hadoop can only deal with the data inside the data center, so it is necessary to study the large data processing mechanism across the data center. In this paper, a new cross data center large data processing architecture and a solution based on the key value (key value) are proposed, which can comply with the data locality principle of the traditional Hadoop, and The scheme can be used to deal with large data processing across data centers at a lower cost. The scheme uses two layers of programming to model the problem and uses a customized two layer packet genetic algorithm. The experiment proves the effectiveness of the scheme. Compared with the traditional Hadoop thought scheme and the existing latest scheme, the proposed scheme can save about 49% of the cost respectively. 40%.
【學(xué)位授予單位】：哈爾濱工業(yè)大學(xué)
【學(xué)位級(jí)別】：博士
【學(xué)位授予年份】：2016
【分類號(hào)】：TP301.6
[Abstract]:It is one of the key factors for the success of the cloud service provider to adopt the efficient resource allocation mechanism which makes the user satisfied. In the Infrastructure act as Service (Iaa S) model, the cloud service provider provides the service mainly in the form of virtual machine, so the problem of resource allocation is also attributed to meeting the business needs. Under the premise of the virtual machine deployment problem, this paper, based on the virtual machine deployment order, uses the top-down method to divide the virtual machine deployment in distributed cloud computing into three stages, namely, the cloud network selection stage, the data center selection stage and the server selection stage, and then combines the different requirements of the specific services in each stage and gives the needle. The resource allocation strategy for sex. In the cloud network selection stage, cloud broker will reduce its cost by leasing the infrastructure of the Gong Youyun service provider to reduce its cost by using the price advantage reserved for long-term resources and the gain brought by the reuse of resource statistics. The instance cycle (virtual machines reserved instance terms), and the price of different instances of different cycles, the cloud broker must make the appropriate selection from a variety of instance cycles so that the minimum cost resources can be used to satisfy the user's real-time dynamic demand. An approximate algorithm based on set coverage is used to solve the problem of resource allocation under off-line conditions, and an online resource allocation algorithm based on the use of historical resources is proposed. The experiment shows that the running speed of the heuristic offline algorithm is almost two times as much as that of the off-line algorithm. Compared with the method of using real time virtual machines in the whole part, two off-line algorithms are used. The algorithm can save about 27% of the cost, and the online algorithm can save about 14% of the cost. Compared with the existing algorithm only considering a reserved instance, the proposed algorithm is more practical. In the data center selection stage, this paper studies the data center selection based on clustering and the migration of large data to the cloud at the data center selection stage. The center selects two problems. The first problem is to optimize the communication delay and bandwidth between data centers. A group of communication virtual machines for a large task or organization may cross multiple data centers, so it is necessary to minimize the distance between the largest data centers, so as to minimize communication delay and save a lot of time. In this paper, the 2- approximate data center selection algorithm based on density clustering is proposed in this paper. Compared with the existing methods that only consider the distance between data centers, the algorithm can make full use of the network topology attributes, dense attributes and capacity information of the data center, and the efficiency is raised about 15% to 72%. and then established. The virtual machine semi communication model is proposed, and a virtual machine partition algorithm based on this model is proposed to group the virtual machine and correspond to the selected data center. The experiment shows that the algorithm can not only reduce the bandwidth consumption between data centers, but also speed up to about 2 times faster than the existing algorithms. All algorithms are applicable to the isomorphism of virtual machines. The objective of the second problem is to select the appropriate data center for distributed large data and transfer the data to the cloud, which can not only guarantee fast local data access but also realize low cost data migration and processing. This paper first analyzes large data migration. The four objectives of time shift: fair data placement, optimal data placement, data placement and total cost minimization of data placement. Then a two division graph is used to model the problem. A compact 3- approximation algorithm based on combination optimization is proposed for the first two targets. A new kind of nearest two targets is proposed for the last two targets. The two algorithms can reduce the time delay and cost of access and solve the problems that are not available due to regulatory restrictions or user preferences, which make up for the shortcomings of the default all data centers available in the existing methods. A low cost server selection. Servers and networking account for about 60% of the total cost of the data center. How to effectively deploy virtual machines to save costs as much as possible and guarantee the quality of service plays an important role in improving the competitiveness of cloud service providers. The problem is formalized into a multi-objective nonlinear programming. By using the network topology information of the data center, the virtual machines with greater traffic are deployed as much as possible, reducing the communication delay and saving the bandwidth consumption between the servers. The default of resource capacity is limited to the specified minimum probability as much as possible. Such a deployment scheme does not affect the quality of service, but also saves the cost of the server and the network. This paper gives a heuristic algorithm for offline and online scenarios. Compared with the existing algorithms, experiments show that the proposed algorithm can save more. It includes a variety of resource costs, including the server and bandwidth, and runs faster. Finally, this paper considers the collaborative selection of data centers and servers during large data processing across data centers. Considering that all the data transmigrated across the global distribution to the same data center is not necessarily feasible and is widely accepted The analysis framework Hadoop can only deal with the data inside the data center, so it is necessary to study the large data processing mechanism across the data center. In this paper, a new cross data center large data processing architecture and a solution based on the key value (key value) are proposed, which can comply with the data locality principle of the traditional Hadoop, and The scheme can be used to deal with large data processing across data centers at a lower cost. The scheme uses two layers of programming to model the problem and uses a customized two layer packet genetic algorithm. The experiment proves the effectiveness of the scheme. Compared with the traditional Hadoop thought scheme and the existing latest scheme, the proposed scheme can save about 49% of the cost respectively. 40%.
【學(xué)位授予單位】：哈爾濱工業(yè)大學(xué)
【學(xué)位級(jí)別】：博士
【學(xué)位授予年份】：2016
【分類號(hào)】：TP301.6
【相似文獻(xiàn)】
相關(guān)期刊論文 前10條
1 江南;數(shù)據(jù)中心如何應(yīng)付管理挑戰(zhàn)[J];互聯(lián)網(wǎng)周刊;2001年40期
2 ;簡化管理挑戰(zhàn)——惠普推實(shí)用數(shù)據(jù)中心解決方案[J];每周電腦報(bào);2001年67期
3 李慶莉;去數(shù)據(jù)中心看一看——中國銀行華北信息中心計(jì)劃處處長云恩善談數(shù)據(jù)中心運(yùn)行、管理[J];中國金融電腦;2002年12期
4 馬天蔚;;數(shù)據(jù)中心按需造[J];每周電腦報(bào);2002年25期
5 戚麗,蔣東興,武海平,馮珂;校園數(shù)據(jù)中心建設(shè)與管理方法的探索[J];教育信息化;2002年S1期
6 何俊山;您企業(yè)的數(shù)據(jù)中心2003了嗎?[J];微電腦世界;2003年17期
7 ;挖潛數(shù)據(jù)中心[J];金融電子化;2004年07期
8 王琨月;;數(shù)據(jù)中心業(yè)務(wù)就緒[J];每周電腦報(bào);2004年21期
9 包東智;新熱點(diǎn):創(chuàng)建下一代數(shù)據(jù)中心[J];上海信息化;2005年10期
10 ;把握數(shù)據(jù)中心建設(shè)五大看點(diǎn)[J];中國計(jì)算機(jī)用戶;2005年10期
相關(guān)會(huì)議論文 前10條
1 姚，
本文編號(hào)：2166416
本文鏈接：http://www.lk138.cn/shoufeilunwen/xxkjbs/2166416.html
最近更新
教材專著
