A BlueGene/L cabinet

Blue Gene is a computer architecture project designed to produce several supercomputers, designed to reach operating speeds in the PFLOPS (petaFLOPS) range, and currently reaching sustained speeds of nearly 500 TFLOPS (teraFLOPS). It is a cooperative project among IBM (particularly the Thomas J. Watson Research Center), the Lawrence Livermore National Laboratory, the United States Department of Energy (which is partially funding the project), and academia. There are four Blue Gene projects in development: BlueGene/L, BlueGene/C, BlueGene/P, and BlueGene/Q. For other uses, see Supercomputer (disambiguation). ... For the popular-music magazine, see Musician (magazine). ... Blue Gene Tyranny (born Robert Sheff in 1945 in San Antonio, Texas) is an avant-garde composer and pianist. ... Image File history File linksMetadata Download high-resolution version (1244x1906, 1804 KB) A BlueGene/L rack. ... Image File history File linksMetadata Download high-resolution version (1244x1906, 1804 KB) A BlueGene/L rack. ... A typical vision of a computer architecture as a series of abstraction layers: hardware, firmware, assembler, kernel, operating system and applications (see also Tanenbaum 79). ... For other uses, see Supercomputer (disambiguation). ... For other uses, see Flop. ... For other uses, see Flop. ... For other uses, see IBM (disambiguation) and Big Blue. ... The Thomas J. Watson Research Center is the headquarters for the IBM Research Division. ... Aerial view of the lab and surrounding area, facing NW. The Lawrence Livermore National Laboratory (LLNL) in Livermore, California is a United States Department of Energy (DOE) national laboratory, managed and operated by Lawrence Livermore National Security, LLC (LLNS), a limited liability consortium comprised of Bechtel National, the University of... The United States Department of Energy (DOE) is a Cabinet-level department of the United States government responsible for energy policy and nuclear safety. ... Academia is a collective term for the scientific and cultural community engaged in higher education and research, taken as a whole. ...

Blue Gene/L

The first computer in the Blue Gene series, Blue Gene/L, developed through a partnership with Lawrence Livermore National Laboratory (LLNL), originally had a theoretical peak performance of 360 TFLOPS, and scored over 280 TFLOPS sustained on the Linpack benchmark. During an upgrade in 2007 the performance increased to 478 TFLOPS sustained and 596 TFLOPS peak. Aerial view of the lab and surrounding area, facing NW. The Lawrence Livermore National Laboratory (LLNL) in Livermore, California is a United States Department of Energy (DOE) national laboratory, managed and operated by Lawrence Livermore National Security, LLC (LLNS), a limited liability consortium comprised of Bechtel National, the University of... For other uses, see Flop. ... LINPACK is a software library for performing numerical linear algebra on digital computers. ...

The term BlueGene/L sometimes refers to the computer installed at LLNL, and sometimes refers to the architecture of that computer. As of November 2006, there are 27 computers on the Top500 list using the Blue Gene/L architecture. All these computers are listed as having an architecture of eServer Blue Gene Solution. The TOP500 project ranks and details the 500 most powerful publicly-known computer systems in the world. ...

History

The block scheme of the BlueGene/L ASIC including dual PowerPC 440 cores.

In December 1999, IBM announced a $100 million research initiative for a five-year effort to build a massively parallel computer, to be applied to the study of biomolecular phenomena such as protein folding. The project has two main goals: to advance our understanding of the mechanisms behind protein folding via large-scale simulation, and to explore novel ideas in massively parallel machine architecture and software. This project should enable biomolecular simulations that are orders of magnitude larger than current technology permits. Major areas of investigation include: how to use this novel platform to effectively meet its scientific goals, how to make such massively parallel machines more usable, and how to achieve performance targets at a reasonable cost, through novel machine architectures. The design is built largely around the previous QCDSP and QCDOC supercomputers. Image File history File links Download high-resolution version (1362x1562, 81 KB) File links The following pages on the English Wikipedia link to this file (pages on other projects are not listed): Blue Gene ... Image File history File links Download high-resolution version (1362x1562, 81 KB) File links The following pages on the English Wikipedia link to this file (pages on other projects are not listed): Blue Gene ... Blue Gene/L Blue Gene is computer architecture project designed to produce several next generation super computers, operating in the PFLOPS range. ... The acronym ASIC, depending on context, may stand for: Application-specific integrated circuit ASIC programming language Australian Securities and Investments Commission This is a disambiguation page — a navigational aid which lists pages that might otherwise share the same title. ... Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain faster results. ... Protein before and after folding. ...

In November 2001, Lawrence Livermore National Laboratory joined IBM as a research partner for Blue Gene. Aerial view of the lab and surrounding area, facing NW. The Lawrence Livermore National Laboratory (LLNL) in Livermore, California is a United States Department of Energy (DOE) national laboratory, managed and operated by Lawrence Livermore National Security, LLC (LLNS), a limited liability consortium comprised of Bechtel National, the University of...

On September 29, 2004, IBM announced that a Blue Gene/L prototype at IBM Rochester (Minnesota) had overtaken NEC's Earth Simulator as the fastest computer in the world, with a speed of 36.01 TFLOPS on the Linpack benchmark, beating Earth Simulator's 35.86 TFLOPS. This was achieved with an 8-cabinet system, with each cabinet holding 1,024 compute nodes. Upon doubling this configuration to 16 cabinets, the machine reached a speed of 70.72 TFLOPS by November 2004 , taking first place in the Top500 list. is the 272nd day of the year (273rd in leap years) in the Gregorian calendar. ... Year 2004 (MMIV) was a leap year starting on Thursday of the Gregorian calendar. ... IBM Rochester is the facility of International Business Machines in Rochester, Minnesota. ... Capital Saint Paul Largest city Minneapolis Largest metro area Minneapolis-St. ... The Earth Simulator (ES) was the fastest supercomputer in the world from 2002 to 2004. ... LINPACK is a software library for performing numerical linear algebra on digital computers. ... The TOP500 project ranks and details the 500 most powerful publicly-known computer systems in the world. ...

On March 24, 2005, the US Department of Energy announced that the Blue Gene/L installation at LLNL broke its speed record, reaching 135.5 TFLOPS. This feat was possible because of doubling the number of cabinets to 32. is the 83rd day of the year (84th in leap years) in the Gregorian calendar. ... Year 2005 (MMV) was a common year starting on Saturday (link displays full calendar) of the Gregorian calendar. ...

On the June 2006 Top500 list, Blue Gene/L installations across several sites world-wide took 3 out of the 10 top positions, and 13 out of the top 64. Three racks of BlueGene/L are housed at the San Diego Supercomputer Center and are available for academic research. The San Diego Supercomputer Center (SDSC) is an organized research unit of the University of California, San Diego (UCSD). ... This article is about the concept. ...

On October 27, 2005, LLNL and IBM announced that Blue Gene/L had once again broken its speed record, reaching 280.6 TFLOPS on Linpack, upon reaching its final configuration of 65,536 "compute nodes" (i.e., 2¹⁶ nodes) and an additional 1024 "I/O nodes" in 64 air-cooled cabinets. The LLNL Blue Gene/L uses Lustre and GPFS to access a 900TB filesystem. is the 300th day of the year (301st in leap years) in the Gregorian calendar. ... Year 2005 (MMV) was a common year starting on Saturday (link displays full calendar) of the Gregorian calendar. ... LINPACK is a software library for performing numerical linear algebra on digital computers. ... Lustre is an Open Source file system for Network-attached storage, generally used for large scale cluster computing. ... [GPFS]http://www-03. ...

BlueGene/L is also the first supercomputer ever to run over 100 TFLOPS sustained on a real world application, namely a three-dimensional molecular dynamics code (ddcMD), simulating solidification (nucleation and growth processes) of molten metal under high pressure and temperature conditions. This won the 2005 Gordon Bell Prize. For other uses, see Flop. ... // Gordon Bell Prizes The Gordon Bell Prizes are a set of awards that were established in 1987. ...

On June 22, 2006, NNSA and IBM announced that Blue Gene/L has achieved 207.3 TFLOPS on a quantum chemical application (Qbox). [1] is the 173rd day of the year (174th in leap years) in the Gregorian calendar. ... Year 2006 (MMVI) was a common year starting on Sunday of the Gregorian calendar. ... The United States National Nuclear Security Administration (NNSA) is part of the United States Department of Energy. ...

On Nov 14, 2006, at Supercomputing 2006 SC06, Blue Gene/L has been awarded the winning prize in all HPC Challenge Classes of awards. [2] November 14 is the 318th day of the year (319th in leap years) in the Gregorian Calendar, with 47 days remaining. ... Year 2006 (MMVI) was a common year starting on Sunday of the Gregorian calendar. ...

On Apr 27, 2007, a team from the IBM Almaden Research Lab and the University of Nevada ran an artificial neural network almost half as complex as the brain of a mouse for the equivalent of ten seconds. [3] April 27 is the 117th day of the year (118th in leap years) in the Gregorian Calendar, with 248 days remaining. ... Year 2007 (MMVII) was a common year starting on Monday of the Gregorian calendar in the 21st century. ... An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ...

In November 2007, the LLNL BlueGene/L remained at the number one spot as the world's fastest supercomputer. It had been upgraded since the last measurement, and was now almost three times as fast as the second fastest, a BlueGene/P system.

Major features

The Blue Gene/L supercomputer is unique in the following aspects:

• Trading the speed of processors for lower power consumption.
• Dual processors per node with two working modes: co-processor (1 user process/node: computation and communication work is shared by two processors) and virtual node (2 user processes/node)
• System-on-a-chip design
• A large number of nodes (scalable in increments of 1024 up to at least 65,536)
• Three-dimensional torus interconnect with auxiliary networks for global communications, I/O, and management
• Lightweight OS per node for minimum system overhead (computational noise)

^[1]

Architecture

One BlueGene/L node card

Each Compute or I/O node is a single ASIC with associated DRAM memory chips. The ASIC integrates two 700 MHz PowerPC 440 embedded processors, each with a double-pipeline-double-precision Floating Point Unit (FPU), a cache sub-system with built-in DRAM controller and the logic to support multiple communication sub-systems. The dual FPUs give each BlueGene/L node a theoretical peak performance of 5.6 GFLOPS (gigaFLOPS). Node CPUs are not cache coherent with one another. Image File history File linksMetadata Download high-resolution version (985x678, 660 KB) A BlueGene/L rack. ... Image File history File linksMetadata Download high-resolution version (985x678, 660 KB) A BlueGene/L rack. ... This article does not cite any references or sources. ... Dynamic random access memory (DRAM) is a type of random access memory that stores each bit of data in a separate capacitor within an integrated circuit. ... The PowerPC 400 family is a line of 32-bit embedded RISC-processor cores built using Power Architecture technology. ... A floating point unit (FPU) is a part of a computer system specially designed to carry out operations on floating point numbers. ... For other uses, see cache (disambiguation). ... For other uses, see Flop. ... Cache coherence refers to the integrity of data stored in local caches of a shared resource. ...

Compute nodes are packaged two per compute card, with 16 compute cards plus up to 2 I/O nodes per node board. There are 32 node boards per cabinet/rack ^[2]. By integration of all essential sub-systems on a single chip, each Compute or I/O node dissipates low power (about 17 watts, including DRAMs). This allows very aggressive packaging of up to 1024 compute nodes plus additional I/O nodes in the standard 19" cabinet, within reasonable limits of electrical power supply and air cooling. The performance metrics in terms of FLOPS per watt, FLOPS per m² of floorspace and FLOPS per unit cost allow scaling up to very high performance.

Each Blue Gene/L node is attached to three parallel communications networks: a 3D toroidal network for peer-to-peer communication between compute nodes, a collective network for collective communication, and a global interrupt network for fast barriers. The I/O nodes, which run the Linux operating system, provide communication with the world via an Ethernet network. The I/O nodes also handle the filesystem operations on behalf of the compute nodes. Finally, a separate and private Ethernet network provides access to any node for configuration, booting and diagnostics. 2-dimensional renderings (ie. ... This article is about operating systems that use the Linux kernel. ... An operating system (OS) is a software that manages computer resources and provides programmers with an interface used to access those resources. ... Ethernet is a large, diverse family of frame-based computer networking technologies that operate at many speeds for local area networks (LANs). ... Ethernet is a large, diverse family of frame-based computer networking technologies that operate at many speeds for local area networks (LANs). ... In computing, booting (booting up) is a bootstrapping process that starts operating systems when the user turns on a computer system. ...

Blue Gene/L compute nodes use a minimal operating system supporting a single user program. Only a subset of POSIX calls are supported, and only one process may be run at a time. Programmers need to implement green threads in order to simulate local concurrency. An operating system (OS) is a software that manages computer resources and provides programmers with an interface used to access those resources. ... POSIX or Portable Operating System Interface[1] is the collective name of a family of related standards specified by the IEEE to define the application programming interface (API) for software compatible with variants of the Unix operating system. ... A thread in computer science is short for a thread of execution. ...

Application development is usually performed in C, C++, or Fortran using MPI for communication. However, some scripting languages such as Ruby have been ported to the compute nodes. Message Passing Interface (MPI) is both a computer specification and is an implementation that allows many computers to communicate with one another. ... Ruby is a reflective, dynamic, object-oriented programming language. ...

To allow multiple programs to run concurrently, a Blue Gene/L system can be partitioned into electronically isolated sets of nodes. The number of nodes in a partition must be a positive integer power of 2, and must contain at least 2⁵ = 32 nodes. The maximum partition is all nodes in the computer. To run a program on Blue Gene/L, a partition of the computer must first be reserved. The program is then run on all the nodes within the partition, and no other program may access nodes within the partition while it is in use. Upon completion, the partition nodes are released for future programs to use. Not to be confused with Natural number. ...

With so many nodes, component failures are inevitable. The system is able to electrically isolate faulty hardware to allow the machine to continue to run.

Plan 9 support

A team composed of members from Bell-Labs, IBM Research, Sandia National Labs, and Vita Nuova have completed a port of Plan 9 to Blue Gene/L. Plan 9 kernels are running on both the compute nodes and the I/O nodes. The Ethernet, Torus, Collective Network, Barrier Network, and Management networks are all supported.[4] [5] Plan 9 from Bell Labs is a distributed operating system, primarily used as a research vehicle. ...

Cyclops64 (BlueGene/C)

Main article: Cyclops64

BlueGene/C (now renamed to Cyclops64) is a sister-project to BlueGene/L. It is a massively parallel, supercomputer-on-a-chip cellular architecture. It was slated for release in early 2007 but has been delayed. Blue Gene/L Blue Gene is computer architecture project designed to produce several next generation super computers, operating in the PFLOPS range. ... The Cyclops64 architecture will contain many hundreds of computing nodes This is about the computer architecture. ...

Blue Gene/P

A BlueGene/P node card

On June 26, 2007, IBM unveiled Blue Gene/P, the second generation of the Blue Gene supercomputer. Designed to run continuously at 1 PFLOPS (petaFLOPS), it can be configured to reach speeds in excess of 3 PFLOPS. Furthermore, it is at least seven times more energy efficient than any other supercomputer, accomplished by using many small, low-power chips connected through five specialized networks. Four 850 MHz PowerPC 450 processors are integrated on each Blue Gene/P chip. The 1-PFLOPS Blue Gene/P configuration is a 294,912-processor, 72-rack system harnessed to a high-speed, optical network. Blue Gene/P can be scaled to an 884,736-processor, 216-rack cluster to achieve 3-PFLOPS performance. A standard Blue Gene/P configuration will house 4,096 processors per rack.[6] Image File history File linksMetadata Download high-resolution version (984x920, 786 KB) A BlueGene/P rack. ... Image File history File linksMetadata Download high-resolution version (984x920, 786 KB) A BlueGene/P rack. ... is the 177th day of the year (178th in leap years) in the Gregorian calendar. ... Year 2007 (MMVII) was a common year starting on Monday of the Gregorian calendar in the 21st century. ... For other uses, see Flop. ... The PowerPC 400 family is a line of 32-bit embedded RISC-processor cores built using Power Architecture technology. ...

On November 12, 2007, the first system, JUGENE, with 65536 processors is running in the Jülich Research Centre in Germany with a performance of 167 TFLOPS.^[3] It is the fastest supercomputer in Europe and the second fastest in the world. The first laboratory in the United States to receive the Blue Gene/P will be Argonne National Laboratory. The first racks of the Blue Gene/P has been shipped in fall 2007. The first installment will be a 111-teraflop system, which has approximately 32,000 processors, will be operational for the US research community in spring 2008. ^[4] is the 316th day of the year (317th in leap years) in the Gregorian calendar. ... Year 2007 (MMVII) was a common year starting on Monday of the Gregorian calendar in the 21st century. ... Position of Jülich Research Centre in Germany The Jülich Research Centre (German: Forschungszentrum Jülich, short FZJ) is based near Jülich, North Rhine-Westphalia, Germany. ... Aerial photo of the Advanced Photon Source at Argonne National Laboratory. ...

Blue Gene/P as a generic Web-scale platform

A team from IBM Research has ported Linux to the compute nodes and demonstrated generic Web 2.0 workloads running at scale on Blue Gene/P. Their paper ^[5] published in the ACM Operating Systems Review describes a kernel driver that tunnels Ethernet over the tree network, which results in all-to-all TCP/IP connectivity. They describe running standard Linux software like MySQL, and report performance results on SpecJBB that rank among the highest on record.

Blue Gene/Q

The last known supercomputer in the Blue Gene series, Blue Gene/Q is aimed to reach 10 PFLOPS in the 2010-2012 time frame. It will continue to expand and enhance the Blue Gene/L and /P architectures with higher frequency at similar performance/watt. Blue Gene/Q will have a similar number of nodes but many more cores per node. [7] For other uses, see Flop. ...

See also

• IBM Roadrunner

Roadrunner is the name given to a next-generation supercomputer to be built at the Los Alamos National Laboratory in New Mexico. ...

References

Year 2007 (MMVII) was a common year starting on Monday of the Gregorian calendar in the 21st century. ... is the 316th day of the year (317th in leap years) in the Gregorian calendar. ... Chicago Life is a magazine included every other month in the Sunday edition of the New York Times in the Chicago area. ... Year 2007 (MMVII) was a common year starting on Monday of the Gregorian calendar in the 21st century. ... is the 224th day of the year (225th in leap years) in the Gregorian calendar. ...

External links and sources

• Blue Gene: A Vision for Protein Science using a Petaflop Supercomputer IBM Systems Journal, Special Issue on Deep Computing for the Life Sciences, Vol. 40, No. 2
• IBM Journal of Research and Development, Blue Gene, Vol. 49, No. 2/3
• IBM Research: Massive parallelism for power and performance efficiency
• IBM Research: Blue Gene
• CNET story: Blue Gene/L tops its own supercomputer record
• CNET story: It's Linux for IBM supercomputer project
• More detailed presentation of Blue Gene architecture (in pdf format)
• Blue Gene application power and performance characterization (pdf)
• Lawrence Livermore National Laboratory: BlueGene/L
• A technical description of the Cyclops64 architecture and system software (Gzipped postscript file)
• TOP500 List of supercomputers as of November 2007
• IBM Triples Performance of World's Fastest, Most Energy-Efficient Supercomputer
• IBM Research: Holistic Aggregate Resource Environment (about Plan 9 on Blue Gene)
• Petascale Plan 9 on Blue Gene (pdf)
• Next generation supercomputers - Blue Gene/P overview (pdf)