English  |  正體中文  |  简体中文  |  全文筆數/總筆數 : 62830/95882 (66%)
造訪人次 : 4049901      線上人數 : 998
RC Version 7.0 © Powered By DSPACE, MIT. Enhanced by NTU Library & TKU Library IR team.
搜尋範圍 查詢小技巧:
  • 您可在西文檢索詞彙前後加上"雙引號",以獲取較精準的檢索結果
  • 若欲以作者姓名搜尋,建議至進階搜尋限定作者欄位,可獲得較完整資料
  • 進階搜尋
    請使用永久網址來引用或連結此文件: https://tkuir.lib.tku.edu.tw/dspace/handle/987654321/102397


    題名: 在Hadoop架構中需求共享及區域感知的排程研究
    其他題名: Shared-scan and locality-aware scheduling algorithm in Hadoop architecture
    作者: 許哲瑋;Hsu, Che-Wei
    貢獻者: 淡江大學資訊管理學系碩士班
    衛信文;Wei, Hsin-Wen
    關鍵詞: MapReduce;Shared-scan;Location-aware;scheduling;Hadoop
    日期: 2014
    上傳時間: 2015-05-04 09:54:45 (UTC+8)
    摘要: 在Hadoop分散式運算架構底下,根據系統所使用排程策略的不同,將會直接影響到整體的系統效能。Hadoop架構中系統所預設的排程策略為先進先出(FIFO),但先進先出排程策略並沒有考慮到不同任務間可能會需要相同的檔案,或是檔案過大時使用網路傳輸檔案導致系統效能降低的影響。本研究提出了FSSL排程策略,以先進先出排程策略為基礎,再加上考慮需求共享與區域感知的因素並在演算法中加入所需要的調整參數,並以此演算法制定新的排程策略進行任務排程以減少網路的負載。實驗結果顯示,我們所提出的FSSL排程策略相較於FIFO排程策略,在多數任務擁有相同需求檔案或是需求檔案較大的執行環境下能夠進一步地改善系統效能,平均系統效能的改善比率約為65%。
    Using different scheduling polices can affect the system performance in Hadoop architecture. In Hadoop architecture, the default scheduling policy is First-In-First-Out (FIFO). However, the FIFO scheduler simply schedule jobs according to their arrival time and does not consider any other factors that may have great impact on system performance. As a result, using FIFO cannot achieve good enough performance in Hadoop.
    In this paper, we propose a novel scheduling algorithm, called FSSL (FIFO with Shared-Scan and Locality-aware). FSSL is a scheduling policy based on FIFO and take locality of required data and data sharing probability between jobs into account. Such that the jobs which need the same data can be gathered and easily batch processed, and thus reduce the overhead of transferring data between data nodes and computations nodes. The results show that FSSL scheduling polity can improve system performance about 65% compared to FIFO scheduling policy.
    顯示於類別:[資訊管理學系暨研究所] 學位論文

    文件中的檔案:

    檔案 大小格式瀏覽次數
    index.html0KbHTML242檢視/開啟

    在機構典藏中所有的資料項目都受到原著作權保護.

    TAIR相關文章

    DSpace Software Copyright © 2002-2004  MIT &  Hewlett-Packard  /   Enhanced by   NTU Library & TKU Library IR teams. Copyright ©   - 回饋