English  |  正體中文  |  简体中文  |  Items with full text/Total items : 64198/96992 (66%)
Visitors : 7992467      Online Users : 2732
RC Version 7.0 © Powered By DSPACE, MIT. Enhanced by NTU Library & TKU Library IR team.
Scope Tips:
  • please add "double quotation mark" for query phrases to get precise results
  • please goto advance search for comprehansive author search
  • Adv. Search
    HomeLoginUploadHelpAboutAdminister Goto mobile version
    Please use this identifier to cite or link to this item: https://tkuir.lib.tku.edu.tw/dspace/handle/987654321/102397


    Title: 在Hadoop架構中需求共享及區域感知的排程研究
    Other Titles: Shared-scan and locality-aware scheduling algorithm in Hadoop architecture
    Authors: 許哲瑋;Hsu, Che-Wei
    Contributors: 淡江大學資訊管理學系碩士班
    衛信文;Wei, Hsin-Wen
    Keywords: MapReduce;Shared-scan;Location-aware;scheduling;Hadoop
    Date: 2014
    Issue Date: 2015-05-04 09:54:45 (UTC+8)
    Abstract: 在Hadoop分散式運算架構底下,根據系統所使用排程策略的不同,將會直接影響到整體的系統效能。Hadoop架構中系統所預設的排程策略為先進先出(FIFO),但先進先出排程策略並沒有考慮到不同任務間可能會需要相同的檔案,或是檔案過大時使用網路傳輸檔案導致系統效能降低的影響。本研究提出了FSSL排程策略,以先進先出排程策略為基礎,再加上考慮需求共享與區域感知的因素並在演算法中加入所需要的調整參數,並以此演算法制定新的排程策略進行任務排程以減少網路的負載。實驗結果顯示,我們所提出的FSSL排程策略相較於FIFO排程策略,在多數任務擁有相同需求檔案或是需求檔案較大的執行環境下能夠進一步地改善系統效能,平均系統效能的改善比率約為65%。
    Using different scheduling polices can affect the system performance in Hadoop architecture. In Hadoop architecture, the default scheduling policy is First-In-First-Out (FIFO). However, the FIFO scheduler simply schedule jobs according to their arrival time and does not consider any other factors that may have great impact on system performance. As a result, using FIFO cannot achieve good enough performance in Hadoop.
    In this paper, we propose a novel scheduling algorithm, called FSSL (FIFO with Shared-Scan and Locality-aware). FSSL is a scheduling policy based on FIFO and take locality of required data and data sharing probability between jobs into account. Such that the jobs which need the same data can be gathered and easily batch processed, and thus reduce the overhead of transferring data between data nodes and computations nodes. The results show that FSSL scheduling polity can improve system performance about 65% compared to FIFO scheduling policy.
    Appears in Collections:[資訊管理學系暨研究所] 學位論文

    Files in This Item:

    File SizeFormat
    index.html0KbHTML268View/Open

    All items in 機構典藏 are protected by copyright, with all rights reserved.


    DSpace Software Copyright © 2002-2004  MIT &  Hewlett-Packard  /   Enhanced by   NTU Library & TKU Library IR teams. Copyright ©   - Feedback