Americanmachinist 618 77324linuxwindo00000049645

Linux and Windows: the right platform for high performance computing and product design

Dec. 21, 2007
By David Morton High-performance computing (HPC) is an essential component of the manufacturing and design processes. Today, it is almost unthinkable to develop the key components of a car, airplane, or even many consumer products without ...

By David Morton

High-performance computing (HPC) is an essential component of the manufacturing and design processes.

Today, it is almost unthinkable to develop the key components of a car, airplane, or even many consumer products without computer-assisted fluid dynamics, structural or impact analysis powered by an highperformance computing system.

Much of the credit for this belongs to the rise of high-performance Linux clusters.

These open source systems have delivered greater performance at dramatically lower costs than outdated proprietary systems and brought new users to high-performance computing. Perhaps attracted by the growing market, Microsoft recently launched Windows Compute Cluster Server and declared its mission to grow market share in high-performance computing.

According IDC, a service that tracks supercomputing trends, 60 percent of the high-performance computing market utilizes Linux-based cluster systems. Projections are that by 2010 that penetration will reach more than 80 percent.

That percentage is even higher at the high end of the market with the Top500, a high-end scoreboard for supercomputers that shows a 77 percent share for these systems. Microsoft is gaining ground, especially in the growing entry level and midrange high-performance computing markets —occupied by scientists, engineers and information technology (IT) staffs trained in Microsoft systems—which is increasingly turning to high-performance computing clusters to crunch data, run simulations and perform other tasks that require a larger amount of compute power.

Microsoft prospects appear favorable. While many consider Windows to be bug-ridden and insecure, almost everyone uses some version of it. That makes it a comfortable platform for new high-performance computing users. In addition, Linux clusters have a reputation for being complex and difficult for non-specialists to manage.

Microsoft also has another strong advantage entering the marketplace: The Windows Compute Cluster Server allows users to submit jobs from within many of the Independent Software Vendor (ISV) applications. The market size and current dominance of the desktop environment has enabled Microsoft to form close partnerships with many of the software vendors that supply commercial applications to the highperformance computing marketplace.

Though Web-based Linux tools now allow users to submit jobs more easily than in previous systems, the complete advantage of in-application job submissions still does not exist in the Linux world. Most of these current web-based job submission tools have been tied to a particular scheduling software tool (such as LSF or PBS Pro) and, as such, have not been broadly adopted. As an aside, Linux Networx will be announcing this quarter a Web-based job submission tool with interfaces to all of the major scheduling tools.

At the same time, Linux and open source software is increasingly moving away from the fringes and into the mainstream of the computing world. Many people now use some version of Web-browsers that are open source software-based, such as Firefox from Mozilla. Increasingly, Linux-based desktop operating systems like the new, well-reviewed Ubuntu ( offer a Windows-like experience with the Linux advantages users have come to expect: scalability, security, fewer bugs and quicker fixes.

In these same areas, Linux-based systems still hold an advantage over Microsoft.

A Microsoft high-performance computing system must run the complete Windows Server 2003, whereas a Linux system can be scaled to the needs of the user, eliminating potential difficulties by using only the necessary elements of the operating system. In addition, bugs and security concerns are often more quickly resolved by the Linux community or by system vendors, while Microsoft users must wait for the seemingly endless procession of service packs.

Linux has been very prevalent in the high-performance computing community over the past decade and dominant over the past five years. As a result, Linux has built up considerable software assets in terms of system management, performance tools, compilers, libraries, global shared file systems, applications, etc.

There are large and clear advantages in all of these areas and it will require many years for the Microsoft ecosystem to catch up in all of these areas. There is also a real community that exists in the online world that can assist new users if they want to access it.

A more user-friendly experience can now also be found in the Linux-based high-performance computing market. For example, the new Clusterworx Advanced cluster management solution from Linux Networx offers a more streamlined user interface for high-performance computing.

This means that organizations don’t need a Linux IT staff to utilize the advantages of Linux-based highperformance computing. In addition, convenient features such as an ‘undo’ button allows users to back out of provisioning errors, one action at a time, without having to restart the entire provisioning process.

In all, Microsoft is bringing the advantages of Windows to the highperformance computing market, including the ability to execute a job from within an application and the familiar, user-friendly Windows experience.

These features will continue to be attractive to potential users in the lower mid-market range especially. Linux systems, however, still dominate the market and are scalable and secure, and increasingly user friendly, to a degree that is still more attractive to many who require the maximum amount of customized compute capability. As the market space continues to grow, both approaches to high-performance computing will continue to adapt and grow with it, finding new niches and solving old problems. In addition, Microsoft’s entry into high-performance computing is good for the space as a whole because of the vendor’s clout and visibility. The company’s entry will undoubtedly raise the visibility of highperformance computing and help to bring high-performance computing to a wider audience.