首页 | 本学科首页   官方微博 | 高级检索  
   检索      


Parallel Clustering Algorithm for Large-Scale Biological Data Sets
Authors:Minchao Wang  Wu Zhang  Wang Ding  Dongbo Dai  Huiran Zhang  Hao Xie  Luonan Chen  Yike Guo  Jiang Xie
Institution:1. School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China.; 2. High Performance Computing Center, Shanghai University, Shanghai, P.R.China.; 3. College of Stomatology, Wuhan University, Wuhan, P.R.China.; 4. Key Laboratory of Systems Biology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, P.R.China.; 5. Department of Computing, Imperial College London, London, United Kingdom.; Semmelweis University, Hungary,
Abstract:

Backgrounds

Recent explosion of biological data brings a great challenge for the traditional clustering algorithms. With increasing scale of data sets, much larger memory and longer runtime are required for the cluster identification problems. The affinity propagation algorithm outperforms many other classical clustering algorithms and is widely applied into the biological researches. However, the time and space complexity become a great bottleneck when handling the large-scale data sets. Moreover, the similarity matrix, whose constructing procedure takes long runtime, is required before running the affinity propagation algorithm, since the algorithm clusters data sets based on the similarities between data pairs.

Methods

Two types of parallel architectures are proposed in this paper to accelerate the similarity matrix constructing procedure and the affinity propagation algorithm. The memory-shared architecture is used to construct the similarity matrix, and the distributed system is taken for the affinity propagation algorithm, because of its large memory size and great computing capacity. An appropriate way of data partition and reduction is designed in our method, in order to minimize the global communication cost among processes.

Result

A speedup of 100 is gained with 128 cores. The runtime is reduced from serval hours to a few seconds, which indicates that parallel algorithm is capable of handling large-scale data sets effectively. The parallel affinity propagation also achieves a good performance when clustering large-scale gene data (microarray) and detecting families in large protein superfamilies.
Keywords:
设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号