A Cluster Operating System Supporting Parallel Computing

The single factor limiting the harnessing of the enormous computing power of clusters for parallel computing is the lack of appropriate software. Present cluster operating systems are not built to support parallel computing – they do not provide services to manage parallelism. The cluster operating environments that are used to assist the execution of parallel applications do not provide support for both Message Passing (MP) or Distributed Shared Memory (DSM) paradigms. They are only offered as separate components implemented at the user level as library and independent servers. Due to poor operating systems users must deal with computers of a cluster rather than to see this cluster as a single powerful computer. A Single System Image of the cluster is not offered to users. There is a need for an operating system for clusters. We claim and demonstrate that it is possible to develop a cluster operating system that is able to efficiently manage parallelism, support Message Passing and DSM and offer the Single System Image. In order to substantiate the claim the first version of a cluster operating system, called GENESIS, that manages parallelism and offers the Single System Image has been developed.     

A case study of a shared/buy-in computing ecosystem

Many research institutions are deploying computing clusters based on a shared/buy-in paradigm. Such clusters combine shared computers, which are free to be used by all users, and buy-in computers, which are computers purchased by users for semi-exclusive use. The purpose of this paper is to characterize the typical behavior and performance of a shared/buy-in computing cluster, using data traces from the Shared Computing Cluster (SCC) at Boston University that runs under this paradigm as a case study. Among our main findings, we show that the semi-exclusive policy, which allows any SCC user to use idle buy-in resources for a limited time, increases the utilization of buy-in resources by 17.4%, thus significantly improving the performance of the system as a whole. We find that jobs allowed to run on idle buy-in resources arrive more frequently and run for a shorter time than other jobs. Finally, we identify the run time limit (i.e., the maximum time during which a job is allowed to use resources) and the type of parallel environment as two factors that have a significant impact on the different performance experienced by shared and buy-in jobs.     

A heterogeneity-aware approach to load balancing of computational tasks: a theoretical and simulation study

One of the distinct characteristics of computing platforms shared by multiple users such as a cluster and a computational grid is heterogeneity on each computer and/or among computers. Temporal heterogeneity refers to variation, along the time dimension, of computing power available for a task on a computer, and spatial heterogeneity represents the variation among computers. In minimizing the average parallel execution time of a target task on a spatially heterogeneous computing system, it is not optimal to distribute the target task linearly proportional to the average computing powers available on computers. In this paper, effects of the temporal and spatial heterogeneity on performance of a target task have been analyzed in terms of the mean and standard deviation of parallel execution time. Based on the analysis results, an approach to load balancing for minimizing the average parallel execution time of a target task is described. The proposed approach whose validity has been verified through simulation considers temporal and spatial heterogeneities in addition to the average computing power on each computer.

Load balancing and optimization of distributed proximity effect correction on a temporally heterogeneous cluster

Grayscale electron-beam lithography is a technique widely used in transferring three-dimensional structures onto the resist layer or substrate. The proximity effect caused by electron scattering in the resist imposes a severe limitation on the ultimate spatial resolution attainable by e-beam lithography. Therefore, correction of the proximity effect is essential particularly for the fine-feature, high-density circuit patterns. However, the proximity effect correction is very time-consuming due to the intensive computation required in the correction procedure and a large size of circuit data to be processed. Hence, it is an ideal candidate for distributed computing where the otherwise-unused CPU cycles of a number of computers on a network (cluster) can be efficiently utilized. One of the characteristics of such a cluster is its heterogeneity, i.e., the available computing power varies with computer and/or time. This variation may degrade the performance of distributed computing significantly. In this paper, efficient distributed implementations of grayscale proximity effect correction on a temporally heterogeneous cluster are described with the main emphasis on static and dynamic load balancing schemes and their optimization through effective task partitioning methods. The experimental results obtained on a cluster of Sun workstations shared by multiple users are presented with detailed discussion.     

Cluster optimization algorithm based on CPU and GPU hybrid architecture

With the rapid development of network technology and parallel computing, clusters formed by connecting a large number of PCs with high-speed networks have gradually replaced the status of supercomputers in scientific research and production and high-performance computing with cost-effective advantages. The research purpose of this paper is to integrate the Kriging proxy model method and energy efficiency modeling method into a cluster optimization algorithm of CPU and GPU hybrid architecture. This paper proposes a parallel computing model for large-scale CPU/GPU heterogeneous high-performance computing systems, which can effectively describe the computing capabilities and various communication behaviors of CPU/GPU heterogeneous systems, and finally provide algorithm optimization for CPU/GPU heterogeneous clusters. According to the GPU architecture, an efficient method of constructing a Kriging proxy model and an optimized search algorithm are designed. The experimental results in this paper show that the construction of the Kriging proxy model can obtain a 220 times speedup ratio, and the search algorithm can reach an 8 times speedup ratio. It can be seen that this heterogeneous cluster optimization algorithm has high feasibility.

     

Optimizing transport protocol parameters for large scale PC cluster and its evaluation with parallel data mining

Recently, PC clusters have come to be studied intensively for large scale parallel computers of the next generation. ATM technology is a strong candidate as a de facto standard of high speed communication networks. Therefore, an ATM-connected PC cluster is a promising platform from the cost/performance point of view, as a future high performance computing environment. Data intensive applications, such as data mining and ad hoc query processing in databases, are considered very important for massively parallel processors, as well as for conventional scientific calculations. Thus, investigating the feasibility of applications on an ATM-connected PC cluster is meaningful. In this paper, an ATM-connected PC cluster consisting of 100 PCs is reported, and characteristics of a transport layer protocol for the PC cluster are evaluated. Point-to-point communication performance is measured and discussed, when a TCP window size parameter is changed. Parallel data mining is implemented and evaluated on the cluster. Retransmission caused by cell loss at the ATM switch is analyzed, and parameters of retransmission mechanism suitable for parallel processing on the large scale PC cluster are clarified. Default TCP protocol cannot provide good performance, since a lot of collisions happen during all-to-all multicasting executed on the large scale PC cluster. Using TCP parameters with the proposed optimization, performance improvement is achieved for parallel data mining on 100 PCs. This revised version was published online in July 2006 with corrections to the Cover Date.     

JXP4BIGI: a generalized, Java XML-based approach for biological information gathering and integration

MOTIVATION: In the post-genomic era, biologists interested in systems biology often need to import data from public databases and construct their own system-specific or subject-oriented databases to support their complex analysis and knowledge discovery. To facilitate the analysis and data processing, customized and centralized databases are often created by extracting and integrating heterogeneous data retrieved from public databases. A generalized methodology for accessing, extracting, transforming and integrating the heterogeneous data is needed. RESULTS: This paper presents a new data integration approach named JXP4BIGI (Java XML Page for Biological Information Gathering and Integration). The approach provides a system-independent framework, which generalizes and streamlines the steps of accessing, extracting, transforming and integrating the data retrieved from heterogeneous data sources to build a customized data warehouse. It allows the data integrator of a biological database to define the desired bio-entities in XML templates (or Java XML pages), and use embedded extended SQL statements to extract structured, semi-structured and unstructured data from public databases. By running the templates in the JXP4BIGI framework and using a number of generalized wrappers, the required data from public databases can be efficiently extracted and integrated to construct the bio-entities in the XML format without having to hard-code the extraction logics for different data sources. The constructed XML bio-entities can then be imported into either a relational database system or a native XML database system to build a biological data warehouse. AVAILABILITY: JXP4BIGI has been integrated and tested in conjunction with the IKBAR system (http://www.ikbar.org/) in two integration efforts to collect and integrate data for about 200 human genes related to cell death from HUGO, Ensembl, and SWISS-PROT (Bairoch and Apweiler, 2000), and about 700 Drosophila genes from FlyBase (FlyBase Consortium, 2002). The integrated data has been used in comparative genomic analysis of x-ray induced cell death. Also, as explained later, JXP4BIGI is a middleware and framework to be integrated with biological database applications, and cannot run as a stand-alone software for end users. For demonstration purposes, a demonstration version is accessible at (http://www.ikbar.org/jxp4bigi/demo.html).     

The big data analytics and applications of the surveillance system using video structured description technology

Recently, the video data has very huge volume, taking one city for example, thousands of cameras are built of which each collects high-definition video over 24–48 GB every day with the rapidly growth; secondly, data collected includes variety of formats involving multimedia, images and other unstructured data; furthermore the valuable information contains in only a few frames called key frames of massive video data; and the last problem caused is how to improve the processing velocity of a large amount of original video with computers, so as to enhance the crime prediction and detection effectiveness of police and users. In this paper, we conclude a novel architecture for next generation public security system, and the “front + back” pattern is adopted to address the problems brought by the redundant construction of current public security information systems which realizes the resource consolidation of multiple IT resources, and provides unified computing and storage environment for more complex data analysis and applications such as data mining and semantic reasoning. Under the architecture, we introduce cloud computing technologies such as distributed storage and computing, data retrieval of huge and heterogeneous data, provide multiple optimized strategies to enhance the utilization of resources and efficiency of tasks. This paper also presents a novel strategy to generate a super-resolution image via multi-stage dictionaries which are trained by a cascade training process. Extensive experiments on image super-resolution validate that the proposed solution can get much better results than some state-of-the-arts ones.     

Secure outsourcing of modular exponentiations in cloud and cluster computing

Cloud computing and cluster computing are user-centric computing services. The shared software and hardware resources and information can be provided to the computers and other equipments according to the demands of users. A majority of services are deployed through outsourcing. Outsourcing computation allows resource-constrained clients to outsource their complex computation workloads to a powerful server which is rich of computation resources. Modular exponentiation is one of the most complex computations in public key based cryptographic schemes. It is useful to reduce the computation cost of the clients by using outsourcing computation. In this paper, we propose a novel outsourcing algorithm for modular exponentiation based on the new mathematical division under the setting of two non-colluding cloud servers. The base and the power of the outsourced data can be kept private and the efficiency is improved compared with former works.     

DNA计算机的分子生物学研究进展

DNA(脱氧核糖核酸)计算机研究是一个新领域。从字面上看,它既包含DNA研究也包含计算机的研究,因而也包含DNA技术与计算机技术如何交融的研究。1994年,Adleman在Science上报道了首例DNA计算的研究结果;2001年,Benenson等在Nature报道了一种由DNA分子和相应的酶分子构成的、有图灵机功能的可程序试管型DNA计算机,标志着DNA计算机研究的重大进展。DNA计算机最大的特点是超大规模的并行运算能力和潜在的巨大的数据储存能力。目前DNA计算机研究已涉及许多领域,包括生物学、数学、物理、化学、计算机科学和自动化工程等具体应用,是计算概念上的一次革命。DNA计算机的研究大大促进了DNA分子操作技术尤其是在纳米尺度下操作DNA分子的研究速度。从DNA计算机的基本原理、应用形式、与基因组学研究的重要关系等方面总结和评述了相关研究进展。     

Developing an integrated cloud-based spatial-temporal system for monitoring phenology

Geospatial cloud computing offers computing infrastructure, software and data services that enable rapid integration of ecological data from various resources. The objectives of this study were to utilize readily-available and low-cost technology (e.g., GPS–enabled cameras, Cloud photo storage, Google Drive) to create a cloud-based spatial-temporal inventory of plant (including flowering phenology) and other relevant information. An interactive ArcGIS Online Map of Lake Issaqueena, SC with sampling locations of flowering plants allows users to obtain additional information (plant, soil, weather data) by selecting sampling locations or soil polygons. The contents of the map can be filtered using any of the attributes (e.g., growth form) in the data tables by selecting specific information. Plant information can be viewed at custom time intervals using the settings in ArcGIS Online. Spatial patterns (e.g., clustering) in the plant data can be viewed using the ArcGIS Online heat map view. The map can be easily queried and viewed on both computers and hand-held devices. Services from multiple cloud infrastructures can be integrated for use by various species monitoring programs, improving workflow and assessment capabilities.     

FFED: a novel strategy based on fast entropy to detect attacks against trust computing in cloud

Cluster Computing - Trust management systems give way to trustworthy interactions in cloud computing. However, malicious cloud users can intentionally provide unfair ratings to benefit or reduce a...     

Techniques for integrating ‐omics data

The challenge for -omics research is to tackle the problem of fragmentation of knowledge by integrating several sources of heterogeneous information into a coherent entity. It is widely recognized that successful data integration is one of the keys to improve productivity for stored data. Through proper data integration tools and algorithms, researchers may correlate relationships that enable them to make better and faster decisions. The need for data integration is essential for present ‐omics community, because ‐omics data is currently spread world wide in wide variety of formats. These formats can be integrated and migrated across platforms through different techniques and one of the important techniques often used is XML. XML is used to provide a document markup language that is easier to learn, retrieve, store and transmit. It is semantically richer than HTML. Here, we describe bio warehousing, database federation, controlled vocabularies and highlighting the XML application to store, migrate and validate -omics data.     

Software Architecture for Processing Clusters Based on I2O

Mainstream computing equipment and the advent of affordable multi-Gigabit communication technology permit us to address data acquisition and processing problems with clusters of COTS machinery. Such networks typically contain heterogeneous platforms, real-time partitions and even custom devices. Vital overall system requirements are high efficiency and flexibility. In preceding projects we experienced the difficulties to meet both requirements at once. Intelligent I/O (I₂O) is an industry specification that defines a uniform messaging format and execution environment for hardware and operating system independent device drivers in systems with processor based communication equipment. Mapping this concept to a distributed computing environment and encapsulating the details of the specification into an application-programming framework allow us to provide architectural support for (i) efficient and (ii) extensible cluster operation. This paper portrays our view of applying I₂O to high-performance clusters. We demonstrate the feasibility of this approach and report on the efficiency of our XDAQ software framework for distributed data acquisition systems.     

A parallel programming interface for out-of-core cluster applications

Clusters of workstations are a practical approach to parallel computing that provide high performance at a low cost for many scientific and engineering applications. In order to handle problems with increasing data sets, methods supporting parallel out-of-core computations must be investigated. Since writing an out-of-core version of a program is a difficult task and virtual memory systems do not perform well in some cases, we have developed a parallel programming interface and the support library to provide efficient and convenient access to the out-of-core data. This paper focuses on how these components extend the range of problem sizes that can be solved on the cluster of workstations. Execution time of Jacobi iteration when using our interface, virtual memory and PVFS are compared to characterize the performance for various problem sizes, and it is concluded that our new interface significantly increases the sizes of problems that can be efficiently solved. Jianqi Tang received B.Sc. and M.Sc. from Harbin Institute of Technology in 1997 and 1999 respectively, both in computer application. Currently, she is a Ph.D. candidate at the Department of Computer Science and engineering, Harbin Institute of Technology. She has participated in several National research projects. Her research interests include parallel computing, parallel I/O and grid computing. Binxing Fang received M.Sc. in 1984 from Tsinghua University and Ph.D. from Harbin Institute of Technology in 1989, both in computer science. From 1990 to 1993 he was with National University of Defense Technology as a postdoctor. Since 1984, he is a faculty member at the Department of Computer Science and engineering of Harbin Institute of Technology, where he is presently a Professor. He is a Member of the National Information Expert Consultant Group and a Standing Member of the Council of Chinese Society of Communications. His research efforts focus on parallel computing, computer network and information security. Professor Fang has implemented over 30 projects from the state and ministry/province. Mingzeng Hu was born in 1935. He has been with the Department of Computer Science and engineering in Harbin Institute of Technology since 1958, where he is currently a Professor. He was a visiting scholar in the Siemens Company, Germany from 1978 to 1979, a visiting associate professor in Chiba University, Japan from 1984 to 1985, and a visiting professor in York University, Canada from 1989 to 1995. He is the Director of the National Key Laboratory of Computer Information Content Security. He is also a Member of 3rd Academic Degree Committee under the State Council of China. Professor Hu’s research interests include high performance computer architecture and parallel processing technology, fault tolerant computing, network system, VL design, and computer system security technology. He has implemented many projects from the state and ministry/province and has won several Ministry Science and Technology Progress Awards. He published over 100 papers in core journals home and abroad and one book. Professor Hu has supervised over 20 doctoral students. Hongli Zhang received M.Sc in computer system software in 1996 and Ph.D. in computer architecture in 1999 from Harbin Institute of Technology. Currently, she is an Associate Professor at the Department of Computer Science and engineering, Harbin Institute of Technology. Her research interests include computer network security and parallel computing.     

基于高通量RNA测序数据分析的弹性云平台

高通量RNA测序（RNA-seq）技术为研究人员提供了海量数据,如何对这些数据进行快速有效的分析,并为后续转录组、基因表达等研究提供支持,是生物信息学领域的热点方向。本文讨论了当前RNA-seq数据分析的发展水平和常用软件、算法,并设计了一系列数据处理模块和分析流程。同时,为了给用户提供更好的使用环境,我们设计了基于弹性资源管理系统的生物云平台BioCloud。该平台集成了丰富的软件,采用高灵活度、高扩展性的体系架构,在给用户提供低成本、高性能计算服务的同时,还提供个性化的流程定制服务。     

Service platform and monitoring architecture for network function virtualization (NFV)

With the advent of cloud and virtualization technologies and the integration of various computer communication technologies, today’s computing environments can provide virtualized high quality services. The network traffic has also continuously increased with remarkable growth. Software defined networking/network function virtualization (SDN/NFV) enhancing the infrastructure agility, thus network operators and service providers are able to program their own network functions on vendor independent hardware substrate. However, in order for the SDN/NFV to realize a profit, it must provide a new resource sharing and monitoring procedures among the regionally distributed and virtualized computers. In this paper, we proposes a NFV monitoring architecture based practical measuring framework for network performance measurement. We also proposes a end-to-end connectivity support platform across a whole SDN/NFV networks has not been fully addressed.     

Dynamic service deployment in a distributed heterogeneous cluster based router (DHCR)

This paper presents the design, implementation and evaluation of an extensible, scalable and distributed heterogeneous cluster based programmable router, called DHCR (Distributed Heterogeneous Cluster based Router), capable of supporting and deploying network services at run time. DHCR is a software IP router relying on heterogeneous cluster composed of separated computers with different hardware and software architecture capabilities, running different operating systems and interconnected through a high speed network connection. The DHCR ensures dynamic deployment of services and distributed control of router components (forwarding and routing elements) over heterogeneous system environments. The DHCR combines the IETF ForCES (Forwarding and Control Element Separation) architecture with software component technologies to meet the requirements of the next generation software routers. To ensure reliable and transparent communication between separated, decentralized and heterogeneous router components, the CORBA based middleware technology is used to support the DHCR internal communication. The paper also explores the use of the CORBA Component Model (CCM) to design and implement a modular, distributed and heterogeneous forwarding path for the DHCR router architecture. The CCM based forwarding plane ensures dynamic reconfiguration of the data path topology needed for low-level service deployment. Results on achievable performance using the proposed DHCR router are reported.

Two-stage selection of distributed data centers based on deep reinforcement learning

Cluster Computing - Cloud computing is a computing service provided on demand through the Internet, which can provide sufficient computing resources for applications such as big data and the...