Ready-to-deploy deep learning solutions

Accelerate your deep learning project deployments with Radeon Instinct™ powered solutions Deep learning adoption is lagging as companies struggle with how to make it work. Now a new ecosystem is rising to deliver the integrated pieces that ultimately will be part of one turnkey system for deep learning. Automation has proved its worth in meeting IT and business objectives. Even so, efficiencies in automation and work augmentation software can be greatly enhanced with deep learning. Yet deep learning adoption rates are low. That’s in part because the tech is difficult, and the talent pool is thin. The good news is that an ecosystem is forming and already beginning to resolve some of these issues as it continues to grow towards becoming a single turnkey system. Why it takes an ecosystem A Deloitte report found that fewer than 10% of the companies surveyed across 17 countries invested in machine learning. The chief reasons for the adoption gap is a lack of understanding on how to use the technology, an insufficient amount of data to train it with, and a shortage of talent who could make it all work. Translated in the simplest of terms, deep learning is perceived by some to be too hard to deploy for practical use. The solution for that dilemma is what it has always been for any new technology requiring esoteric skill sets and faced with a talent shortage – build an easy-to-use, turnkey system. That is, of course, easier said than done. “The ongoing digital revolution, which has been reducing frictional, transactional costs for years, has accelerated recently with tremendous increases in electronic data, the ubiquity of mobile interfaces, and the growing power of artificial intelligence (AI),” according to a McKinsey & Company report. “Together, these forces are reshaping customer expectations and creating the potential for virtually every sector with a distribution component to have its borders redrawn or redefined, at a more rapid pace than we have previously experienced.” That’s why today’s sophisticated and complex systems are commonly constructed not by a single vendor but by a strong and diverse ecosystem capable of delivering the many moving parts needed to make a single turnkey system. Especially when said systems must be equally workable for companies across industries and with diverse needs. As a result, ecosystems are growing at breathtaking speeds. McKinsey & Company analysts predict that new ecosystems are likely to entirely replace many traditional industries by 2025. Such an ecosystem is forming for machine learning. It’s seeded with four recently launched, ready-to-deploy solutions. They center on AMD’s Radeon Instinct training accelerator for machine learning, and its ROCm Open eCosystem (ROCm), an open source HPC/Hyperscale-class platform for GPU computing. AMD takes open source all the way down to the graphics card level. Open source is key to successfully wrangling machine learning systems as it leverages the skills and coding work from entire communities and makes an ecosystem functional across technologies and applications. The ROCm open ecosystem This newly forming ecosystem is optimal for beginning or expanding your deep learning efforts whether you are the IT person looking to get pre-configured deep learning technologies in place, or the scientist who just needs access to HPC systems with one of the frameworks loaded. Either way, users can quickly get to work with their data. Developers also have full and open access to the hardware and software which speeds their work in developing frameworks. Everything AMD develops for its Radeon Instinct system is open source and available on GitHub. The company also has docker containers for easier installs of ROCm drivers and frameworks which can be found on the ROCm site for Docker.  Caffe and TensorFlow machine learning frameworks are offered now, with more to follow soon. A deep learning solutions page has gone live, which features the four systems that service as the bud of the blooming ecosystem rooted in AMD technologies. The frameworks docker containers will be listed there as well. This budding machine learning ecosystem is already bearing fruit for organizations looking to launch machine learning training and applications with a minimum of technical effort and expertise by combining: Fast and easy server deployments ROCm Open eCosystem and infrastructure Deep learning framework docker containers Optimized MIOpen framework libraries The four systems forming the ecosystem center “Data science is a mix of art and science—and digital grunt work. The reality is that as much as 80 percent of the work on which data scientists spend their time can be fully or partially automated,” according to a Deloitte report. This newly forming ecosystem is focused on automating much of the machine learning processes. While complicated to achieve, the end results are far easier for organizations to use. Deloitte identified five key vectors of progress that should help foster significantly greater adoption of machine learning by making it more accessible. “Three of these advancements—automation, data reduction, and training acceleration—make machine learning easier, cheaper, and/or faster. The others—model interpretability and local machine learning—open up applications in new areas,” according to the Deloitte report. There are four prebuilt systems shaping this ecosystem early on. Each is provided by an independent partner and built on or for AMD’s Radeon Instinct and ROCm platforms, but their initial presentations are at varying levels of integration. While more partners will join the ecosystem over time, these four provide a solid bedrock for organizations looking to get started in machine learning now. 1) AMAX is providing systems with preloaded ROCm drivers and a choice of framework, either TensorFlow or Café, for machine learning, advanced rendering and HPC applications. 2) Exxact is similarly providing multi-GPU Radeon Instinct-based systems with preloaded ROCm drivers and frameworks for deep learning and HPC-class deployments, where performance per watt is important. 3) Inventec provides optimized high performance systems designed with AMD EPYC™ processors and Radeon Instinct compute technologies capable of delivering up to 100 teraflops of FP16 compute performance for deep learning and HPC workloads. 4) Supermicro is providing SuperServers supporting Radeon Instinct machine learning accelerators for AI, big data analytics, HPC, and business intelligence applications. The payoff from leveraging the technologies in a machine learning ecosystem potentially comes in many forms. “A growing number of tools and techniques for data science automation, some offered by established companies and others by venture-backed start-ups, can help reduce the time required to execute a machine learning proof of concept from months to days. And, automating data science means augmenting data scientists’ productivity in the face of severe talent shortages,” say the Deloitte researchers.

https://www.hpcwire.com/2018/07/23/ready-to-deploy-deep-learning-solutions/

Radio Free HPC Looks at New USA Supercomputing Map

In this podcast, the @RadioFree #HPC team looks at the new interactive USA #Supercomputing Map from @Hyperion Research. “The mapped sites include government, academic and industrial HPC data centers, along with HPC vendors. This powerful tool can be used to identify the economic impact of HPC in a user-defined area (state, Congressional district, et al.) or for the United States as a whole, or to understand where HPC jobs are located, as well as who the Congressional district representatives are.” As part of the discussion, Rich recaps Hyperion’s recent HPC User Forum in Tucson. The event featured an extended session on Quantum Computing with presentations by D-Wave Systems, Google, IBM, Intel, Microsoft, NIST, and Rigetti Computing. You can watch them all right here on insideHPC.  After that, we do our Catch of the Week: Henry is concerned about about the privacy implications raised by near-relative DNA methods used to catch the Golden State Killer. Henry notes that Gartner’s claim that AI is going to deliver $3.9 Trillion dollars in Global Business Value by 2022. Shahin reminds us that this week marked the 25th anniversary of the first Mosaic browser. Dan notes that Amazon logged $1.6 Billion in profits for the first quarter, a big jump from last year thanks to substantial growth in AWS. Meanwhile, Microsoft Azure is up 93 percent this quarter as well.
https://insidehpc.com/2018/04/radio-free-hpc-looks-new-usa-supercomputing-map/

Affordable Optical Technology Needed Says HPE’s Daley

While not new, the challenges presented by computer cabling/PCB circuit routing design – cost, performance, space requirements, and power management – have coalesced into a major headache in advanced #HPC system design, said @JimmyDaley, head of HPC engineering for #HPE, at the HPC User Forum last week. What’s required, and sooner-rather-than-later, is broader adoption of optical cable technology as well as mid-board optics said Daley. Daley’s ~15-minute jaunt through the major copper versus optical issues was a good reminder of this persistent problematic area. He bulleted out three sets of challenges posed by cabling and reviewed briefly how optical solutions could help overcome them: Cost and Signaling Density and Egress Power and Thermals The slide below contrasts the size difference of copper versus optical cable. Like many, Daley is looking for cabling options and currently there aren’t many. His presentation was both a refresher and look-ahead at cable interconnect issues and opportunities. Copper cable, of course, predominates but it’s not cheap and has plenty of constraints. Active optical cable (AOC) is powerful but also on the order of 4-6X more expensive than copper; Daley called AOC HPC’s nemesis. Broader use of passive optical would solve many problems and be less expensive but there’s a fair amount of work to be done before that is practical. Today, AOC is the backbone for most very large supercomputers. “Even though these AOC cables struggle with costs and other things, the adoption of these cables is still very aggressive. We have to [use them] in order to get our science done,” he said. “At 50 gigs (see chart below), we as an industry pretty much only use them sparingly. As we get to 100 gigs speeds we are already at about 40 percent mix in active optical cables (AOC) and copper and as we move to 200 gigs and beyond, we are predicting that most of what we deploy is going to have to be active optical cable.” Daley pointed out it is not the fiber in AOC that’s so expensive; it’s actually less expensive than copper, but everything else required (various components and materials) for active optical cable that boosts its cost. “[Given] the cost of copper cable I’m not sure I am going to be able to do a meaningful 400 gig copper cable, which means I am now at a list price of $2,000-plus for AOC. Go build a fabric out of that and see how far the wallet goes,” said Daly. The big win would be in being able to deploy passive cable and efforts are ongoing to develop better passive optical cable solutions. Printed circuit boards present similar challenges as signaling demands rise. “Where we live today (PCIe Gen 3) I can get everything using fairly standard PCB material. As we start talking about PCIe Gen 4…[w]e are going to have to start looking at more and more exotic routing material. We have a lot of smart guys in HPE and industry and this box will start to shift up ever so slightly, but it will not shift up meaningfully. And my red box (chart below) here is just a lost cause,” he said. Clearly optical cable has many advantages – physical size and bandwidth are the obvious ones. The growth in systems has exacerbated pressure on interconnect throughout systems and Daley’s slide below illustrating the difference in copper cabling versus optical cabling required for 256-node makes the point nicely; the cross section of the copper cable is roughly the size of a CD while the optic cable counterpart is the size of a watch face. Moving to optical as well as “mid-board optics,” he said, is a necessary step for larger systems. He also argued for accelerated development of passive optical approaches, which are not only cheaper, but more suitable for modular systems. “If I move to passive optical cable that also frees us up. Forget the economics and physics of things. It frees you guys up to wire the datacenter one time, you put the wires in and the optics actually come with the thing that you are plugging in. Today we are very driven to making sure that within a rack that you hit the right switch radix because if you leave switch ports empty, those are costs that have to get amortized across all of the other nodes in the rack and that becomes very, very painful. “If I move to optical cable where distances aren’t [the issue] then I don’t have to have a top-of-rack switch, I have a mid-row switch and we just make sure that within that row we design and get the right things there. Then I can start to design to power constraints and thermal constraints besides switch radixes. The biggest thing in my mind is that it allows us to start to optimize our topologies for the workload we are trying to do as opposed to optimizing for “how do I get as many copper cables into this picture as I can because I can’t afford optical cables as we move forward.’”

https://www.hpcwire.com/2018/04/26/affordable-optical-technology-needed-says-hpes-daly/

Overcoming Space and Power Limitations in HPC Data Centers

In companies of all sizes, critical applications are being adopted to accelerate product development, make forecasts based on predictive models, enhance business operations, and improve customer engagements. As a result, there is a growing need for Big Data analytics in many businesses, more sophisticated and more granular modeling and simulation, wide-spread adoption of AI (and the need to train neural nets), and new applications such as the use of genomic analysis in clinical settings and personalized medicine. These applications generate workloads that overwhelm the capacity of most installed data center server systems. Simply put, today’s compute-intensive workloads require access to significant HPC resources. Challenges bring HPC to the Mainstream Many of today’s new and critical business applications are pushing the limits of traditional data centers. As a result, most companies that previously did not need HPC capabilities, now find such processing power is required to stay competitive. Unfortunately, several problems prevent this from happening. When attempting to upgrade infrastructure, most organizations face inherent data center limitations with space and power. Specifically, many data centers lack the physical space to increase compute capacity significantly. And all organizations incur high electricity costs to run and cool servers, while some data centers have power constraints that cannot be exceeded. Additionally, there is lack of internal HPC expertise. IT staff may not have the knowledge base to determine which HPC elements (including processors, memory, storage, power, and interconnects) are best for the organization’s workloads or the expertise to carry out HPC system integration and optimization. These skills have not been required in mainstream business applications until now. As a result, most organizations need help when selecting an HPC solution to ensure it is the right match for the organization’s compute requirements and budget constraints, and one that fits into an existing data center. Selecting the Right Technology Partner Traditional clusters consisting of commodity servers and storage will not run the compute-intensive workloads being introduced into many companies today. Fortunately, HPC systems can be assembled using the newest generation of processors, high-performance memory, high-speed interconnect technologies, and high-performance storage device like NVMe SSDs. However, to address data center space and power issues, an appropriate solution must deliver not just HPC capabilities, but the most compute power per watt in a densely packed enclosure. To achieve this, it makes sense to find a technology partner with deep HPC experience who can bring together optimized systems solutions with rack hardware integration and software solution engineering to deliver ultimate customer satisfaction. This is an area where Super Micro Computer, Inc. can help. Supermicro® has a wide-range of solutions to meet the varying HPC requirements found in today’s organizations. At the heart of its HPC offerings are the SuperBlade® and MicroBlade™ product lines, which are advanced high-performance, density-optimized, and energy-efficient solutions for scalable resource-saving HPC applications. Both lines offer industry-leading performance, density, and energy efficiency. They support the option of BBP® (Battery Backup Power modules), so the systems provide extra protection to the data centers when a power outage or UPS failure occurs. This feature is ideal for critical workloads, ensuring uptime in the most demanding situations. SuperBlade and MicroBlade solutions are offered in several form factors (8U, 6U, 4U, 3U) to meet the various compute requirements in different business environments. At the high end of the spectrum, there is the 8U SuperBlade: SBE-820C series enclosure supports 20x 2-socket (Intel® Xeon® Scalable processor) blade servers with 40 hot-plug NVMe SSDs or 10x 4-socket (Intel® Xeon® Scalable processor) blade servers with 80 hot-plug NVMe SSDs, 100Gbps EDR InfiniBand or 100Gbps Intel Omni-Path switch, and 2x 10GbE switches. This SKU is best for HPC, enterprise-class applications, cloud computing, and compute-intensive applications. SBE-820J series enclosure supports 20x 2-socket (Intel® Xeon® Scalable processor) blade servers with 40 hot-plug NVMe SSDs or 10x 4-socket (Intel® Xeon® Scalable processor) blade servers with 80 hot-plug NVMe SSDs, and 4x Ethernet switches (25GbE/10GbE). This SKU is similar to the SKU above, except it is built to operate at 25G/10G Ethernet instead of 100G InfiniBand or Omni-Path. This solution is most suitable for HPC workloads in IT environments that leverage Ethernet switches with 40G or 100G uplinks. The 8U SuperBlade offering includes the highest density x86 based servers that can support up to 205W Intel® Xeon® Scalable processor. One Supermicro customer at a leading semiconductor equipment company is using 8U SuperBlade systems for HPC applications with 120x 2-socket (Intel® Xeon® Scalable processor) blade servers per rack. This allows the company to save a significant amount of space and investment dollars in its data center. Supermicro solutions helped a Fortune 50 Company scale its processing capacity to support its rapidly growing compute requirements. To address space limitations and power consumption issues, the company deployed over 75,000 Supermicro MicroBlade disaggregated, Intel® Xeon® processor-based servers at its Silicon Valley data center. Both SuperBlade and MicroBlade are equipped with advanced airflow and thermal design and can support free-air cooling. As a result, this data center is one of the world’s most energy efficient with a Power Usage Effectiveness (PUE) of 1.06. Compared to a traditional data center running at 1.49 PUE, this new Silicon Valley data center powered by Supermicro blade servers achieves an 88 percent improvement in overall energy efficiency. When the build-out is complete at a 35 megawatt IT load power, the company is targeting $13.18M in savings per year in total energy costs across the entire data center.

https://www.hpcwire.com/2018/04/23/overcoming-space-and-power-limitations-in-hpc-data-centers/

Why HPC Matters: Understanding Our Universe

Searching for gravitational waves and exploring the physics of black holes and warped space-time is all in a day’s work for teams of researchers in Australia. This is the type of work conducted by the Australian Research Council’s Centre of Excellence for Gravitational Wave Discovery (OzGrav). And it’s the type of work that couldn’t be carried out without the extreme computational power of a supercomputer. In OzGrav’s case, the computational power is delivered via a new $4 million Dell EMC supercomputer launched at Swinburne University of Technology. The supercomputer, named OzSTAR, delivers an astounding peak performance of 1.2 petaflops, making it one of the most powerful supercomputers in Australia. What does that mean in layman’s terms? OzGrav’s director, Professor Matthew Bailes, explains it this way: “In one second, OzSTAR can perform 10,000 calculations for every one of the 100 billion stars in our galaxy.”1 That’s the kind of computational power it takes to shift through an unfathomable amount of data coming from giant telescopes to gain insights that help unlock the secrets of the universe. With this type of scientific research, high performance computing (HPC) is an absolute requirement. For example, researchers couldn’t begin to model the large-scale structure of the universe or simulate the formation and evolution of galaxies without HPC. “We will be looking for gravitational waves that help us learn more about supernovas, the formation of stars, intergalactic gases and more,” Professor Bailes says. “It’s exciting to think that we at OzGrav could make the next landmark discovery in gravitational wave astrophysics — and the Dell EMC supercomputer will allow us to capture, visualize and process the data to make those discoveries.”2 The OzSTAR supercomputer is based on Dell EMC PowerEdge™ R740 compute and data-crunching nodes; a Dell EMC H-Series Networking Fabric, which is powered by the Intel® Omni-Path Architecture (Intel OPA); and Dell EMC HPC Storage with a Lustre parallel file system. “This combination of Dell EMC technologies will deliver the incredibly high computing power required to move and analyze data sets that are literally astronomical in size,” says Andrew Underwood, Dell EMC’s Australia-New Zealand high performance computing lead, who collaborated with Swinburne on the supercomputer design.3 “Brilliant researchers are often only limited by advances in technology,” adds Chris Kelly, Vice President, Compute and Networking Solutions, Dell EMC Asia-Pacific. “With this new supercomputer, Swinburne and OzGrav will be able to embark on a new era of astronomy that could unlock answers to questions mankind has pondered for centuries. It’s an incredibly exciting time for astronomical research.”4 For a closer look at technologies used to create HPC solutions that deliver the extreme computational power needed to explore black holes and warped space-time, visit the Dell EMC Go Ahead. Dream Big. site.

https://www.cio.com/article/3269031/it-industry/why-hpc-matters-understanding-our-universe.html

The Future Of Manufacturing Technologies, 2018

These and many other insights are from a recent research study @Deloitte in collaboration with the Council on Competitiveness and Singularity University published this month titled Exponential Technologies in Manufacturing (PDF, 64 pp., no opt-in). The study’s goals include discovering the latest opportunities and barriers manufacturers face in evaluating and adopting technologies, and explores how global manufacturing companies can best capitalize on emerging technologies. The study defines exponential technologies as those that enable change at a rapidly accelerating, nonlinear pace facilitated by substantial progress and cost reduction in the areas of computing power, bandwidth, and data storage. Key takeaways from the study include the following: #IoT #Bkockchain #HPC #3DPrinting #AI

https://www.forbes.com/sites/louiscolumbus/2018/04/15/the-future-of-manufacturing-technologies-2018/#26220ed12995

Supermicro RSD 2.1 Pools All-Flash NVMe Composable Storage

Today #Supermicro announced their new Rack Scale Design (RSD) 2.1 with pooled all-flash #NVMe composable storage support for applications like high throughput ingest, #HPC, #dataanalytics, video streaming, CDN, and software-defined storage ( #SDS) environments. “Occupying just 1U of rack space, our all-flash NVMe storage systems support 32 hot-swap 2.5″ NVMe SSDs for a half petabyte of high-performance storage with Supermicro RSD 2.1 that can be shared by 12 hosts simultaneously,” said Charles Liang, president and CEO of Supermicro. “With dynamically composable server nodes, efficient storage utilization, and independently upgradeable compute resources, our RSD 2.1 solutions with advanced NVMe pooled storage are perfect for efficient and flexible hyperscale datacenters. In fact, we have already deployed these 32-drive systems running Hadoop workloads for a major automobile company.”  Supermicro RSD empowers cloud service providers, telecoms and Fortune 500 companies to build their own agile, efficient software-defined datacenters or expand existing ones. Supermicro RSD is based on Intel Rack Scale Design (Intel RSD), which is an industry-aligned datacenter architecture built on open standards. Intel RSD helps large datacenters support new data-centric applications that require large amounts of very fast storage to be shared while enabling increased speed, agility and automation. In an environment of explosive datacenter growth, Intel RSD provides an infrastructure blueprint with more flexibility and scalability, higher security and better control, while delivering substantial savings in capex and opex. Supermicro RSD manages racks of disaggregated servers, storage, and networking with industry standard Redfish Restful APIs that remain consistent across different vendors and multiple server generations. Supermicro RSD 2.1, supports high performance, high density, and disaggregated NVMe storage for dramatically improved datacenter efficiency, increased utilization and lower costs. Based on Intel RSD version 2.1 spec, the arrival of this technology marks the beginning of a paradigm shift to deploy truly disaggregated resource pools in large-scale data centers that dramatically improve datacenter efficiency, increase utilization and reduce costs. For optimal compatibility, Supermicro RSD 2.1 runs on all X11-generation server and storage systems supporting Intel Xeon Scalable processors as well as Supermicro X10-generation server and storage systems supporting the Intel Xeon processor E5-2600 v4 family. This cross-generational hardware compatibility offers Supermicro customers investment protection and the flexibility to either build new or expand their existing datacenters. In addition, Supermicro RSD 2.1 is tightly integrated with other datacenter management software layers such as OpenStack using the Restful Pod Manager APIs that enable end-to-end cloud infrastructure deployment. The traditional way of scaling up datacenter resources often involves adding server nodes with fixed computing, networking, and storage ratios. Due to the different life cycles of these resources, a wholesale upgrade of the entire set of server nodes will often lead to premature retirement of valuable investments and resource underutilization. That is why Supermicro’s resource-saving, disaggregated NVMe storage solutions like NVMe-oF (NVMe over Fabric) are so important to customers who want to build more efficient and flexible hyper-scale datacenters.

https://insidehpc.com/2018/04/supermicro-rsd-2-1-pools-flash-nvme-composable-storage/

Advanced Scale Forum Announces Sponsorship from Dell EMC, HPE, Microsoft and Others

SAN DIEGO, Calif., April 10, 2018 — Advanced Scale Forum 2018, a power summit designed to solve challenges in the build-out of scalable advanced enterprise computing solutions, today announced this year’s sponsor lineup. The summit, produced by Tabor Communications (publisher of HPCwire, EnterpriseTech, Datanami) and nGage Events, is scheduled for May 6-8, 2018 at the Hyatt Regency Lost Pines Resort & Spa in Austin, Texas, and will focus on #AI, #deeplearning and #machinelearning, #Blockchain, advanced-scale #cloudcomputing, #IoTedge to #IoTcore and high performance computing #HPC. Advanced Scale Forum (#ASF) is proud to announce the following sponsorships with key technology providers, who will be conducting private, board room briefings to leading enterprise executives: @Accenture, @Dell EMC, @HPE, @Microsoft, @NVIDIA, @Quantum, @Attunity, @CB Technologies, @HDF, @WekaIO, @AMD, @Arcadia Data, @Cray, @E8 Storage, @Intel, @Lawrence Livermore National Laboratory, @MapR, @NetApp, @Nimbix, @Penguin Computing, @Six Nines, @Striim, @System Fabric Works, @Talking Data, @Data Vortex, @Memsql and @PSSC Labs. Through these sponsorships, ASF brings attendees face-to-face with advanced-scale IT’s leading solution providers as they share the case studies that have driven the most value for their customers. This diversity of vendor perspectives allows for full view of industry direction, including: strategy, consulting, digital technology and operations, scale-out tiered storage, cloud and hybrid environments, deep learning and modern AI, and more. “Convergence around the high end of computing, data analytics and AI is transforming the business computing landscape, where modern IT solutions are engines for innovation and growth,” said Tiffany Trader, Managing Editor of HPCwire and Director of Editorial, Tabor Communications. “Advanced Scale Forum brings you face-to-face with these emerging technologies as vehicles for driving real-world results. Through leadership keynotes, case studies, one-on-one meetings, and breakout sessions, this three-day summit provides a tailored experience like no other. The information gained over the course of the summit is essential for staying ahead in our industry.” Helping to drive the sponsor lineup has been the announcement of this 2018’s Summit Agenda, which features thought leaders and industry experts on hot-button topics from Blockchain, data governance, artificial intelligence and machine learning, and the first ever demonstration of a Gulfstream aircraft simulation outside of NDA-restricted sales meetings. “Advanced Scale Forum is a one-of-a-kind executive summit,” said Tom Tabor, CEO of Tabor Communications. “Nowhere in the world is there an event that’s exclusive to leaders from the industrial end-user and vendor communities where mission-critical, production-level challenges are discussed with a focus of utilizing advanced-scale computing technologies as a cornerstone of the solution. “Each year we are honored to bring more vendors, thought leaders and decision makers to the table, and the activity surrounding our 2018 event has been unprecedented. We’re immensely proud of this year’s program as well as the breadth of in-depth sponsor participation, and cannot wait to share it with our attendees.”

https://www.hpcwire.com/off-the-wire/advanced-scale-forum-announces-sponsorship-from-hpe-microsoft-and-others/

The Future of Programming GPU Supercomputers

There are few people as visible in high performance computing programming circles as @MichaelWolfe—and fewer still with level of experience. With 20 years working on PGI compilers and another 20 years before that working on languages and #HPC compilers in industry, when he talks about the past, present and future of programming supercomputers, it is worthwhile to listen. In his early days at PGI (formerly known as The Portland Group) Wolfe focused on building out the company’s suite of Fortran, C, and C++ compilers for HPC, a role that changed after Nvidia Tesla GPUs came onto the scene and required stronger hooks for the growing supercomputing set in 2007–and certainly after. Even though Wolfe (who came to Nvidia in 2007 when the GPU maker acquired PGI) sees GPU computing as a major force that has shaped HPC, he says that when it comes to programming GPU supercomputers in the next decade, the more things will change the more they will stay the same—in some ways at least. During his talk today at the annual GPU Technology Conference (GTC), Wolfe made the argument that the way we write parallel codes in HPC will not change much from how it’s done now. In short, C++ and Fortran will still dominate but for GPU accelerated supercomputing, these simpler methods will overtake others as more creativity is put into exploiting the inherent parallelism of GPU and the sequential offload capabilities of CPUs (which Wolfe says will continue to be just as relevant). This might not be the sexy futurist answer folks were hoping to hear from Wolfe, but let’s be honest—HPC programming is about extending the lifespan of languages over many decades. “Some, especially those who are younger, are going to be bothered by seeing Fortran here. Think about this: 30 years ago when C++ was invented, Fortran was already 30 years old. Young people then were sneering as many of you are now, saying it’s too old and needs to be retired. Now C++ is 30 and I don’t hear any sneering there—in 30 years when C++ is 60, you’ll be ecstatic that the framework you worked on still works with C++,” Wolfe told the whippersnappers in the room. The way these languages adapt over time is going to take some creativity, as we will get to in a moment, but Wolfe says he believes that both Fortran and C++ will continue to be the future of HPC programming with GPUs with no extensions and no directives for most users—a big claim at a time when we see even sophisticated directives-based approaches like OpenMP and OpenACC working hard to keep up with the many changes in new GPU memory architectures (among other things) in architectures like the latest Volta generation GPUs.

https://www.google.com/amp/s/www.nextplatform.com/2018/03/26/the-future-of-programming-gpu-supercomputers/amp/#ampshare=https://www.nextplatform.com/2018/03/26/the-future-of-programming-gpu-supercomputers/

Google makes it easier to run high-performance workloads on its cloud platform

@Google makes it easier to run high-performance workloads on its cloud platform Frederic Lardinois @fredericl / Yesterday #Hyperscale #cloud platforms from the likes of @Amazon, @Microsoft and @Google are great for running the kind of high-performance computing ( #HPC) projects that scientists in academia and the industry need for their simulations and analyses. Many of the workloads they run are, after all, easily parallelized across hundreds or thousands of machines. Often, though, the challenge is about how to create these clusters and how to then manage the workloads that run on them. To make this easier for the HPC community, Google today announced that it is bringing support for the open source Slurm HPC workload manager to its cloud platform (which is different from this Slurm). That’s the same piece of software that the many of the users in the TOP500 supercomputer list use, including the world’s biggest and fastest cluster to date, including the Sunway TaihuLight with its over 10 million computing cores. For this project, Google teamed up with the experts at SchedMD, the company behind Slurm, to make it easier to run Slurm on Compute Engine. Using this integration, developers can easily launch an auto-scaling Slurm cluster on Compute Engine that runs based on the developers’ specifications. One interesting feature here is that users can also federate jobs from their on-premise cluster to the cloud when they need a bit of extra compute power. Compute Engine currently offers machines with up to 96 cores and 624 GB of memory, so if you have the need (and money), building a massive compute cluster on GCP just got a little bit easier. It’s worth noting that Microsoft, too, offers a template for deploying Slurm on Azure and that the tool has long supported AWS, too.

https://techcrunch.com/2018/03/23/google-makes-it-easier-to-run-high-performance-workloads-on-its-cloud-platform/

Alibaba Launches European Supercomputing Cloud Service, Quantum Computing Platform

Chinese-base tech giant @Alibaba is challenging American cloud providers in Europe with an #HPC service designed for users running a variety of compute-intensive and data-intensive workloads. The company also unveiled a new cloud-based quantum computing platform. The addition of the HPC service was revealed at the Mobile World Congress taking place this week in Barcelona Spain. The announcement actually encompassed eight new Alibaba Cloud products for Europe, one of which, the Super Computing Cluster (SCC) product, is aimed at traditional high performance computing workloads, as well as AI and other data analytics applications. SCC represents a sub-family of the company’s Elastic Compute Service (ECS) bare metal instances and are powered by Intel’s latest Xeon Scalable Processors and, optionally, NVIDIA P100 and V100 GPUs. Cluster connectivity is supplied by an RoCE v2 (RDMA over Converged Ethernet, version 2). The choice of Ethernet over InfiniBand was rationalized by noting that RoCE approaches InfiniBand in performance but supports more extensive Ethernet-based applications. The support for Ethernet rather than InfiniBand may be a calculation by Alibaba that their main customer base is probably going to be drawn from enterprise users – in traditional HPC, artificial intelligence, data analytics, and audio/video processing – which tend to be more comfortable with Ethernet-based networking According to the SCC webpage, those bare metal instances support up to 96 cores of Xeon processors and 512 GB of main memory.  As many as eight P100 or V100 GPU coprocessors can be attached as well. Two types of SCC instances have been described: scch5 and sccg5.   The scch5 instance provides 64 Xeon Gold 6149 (3.1 GHz) cores and 192 GB of memory, while the top-of-the-line sccg5 provides 96 Xeon Platinum 8163 (2.5 GHz) cores and 384 GB of memory. Those specs are based on the Alibaba Cloud’s instances webpage as of this writing.  However, the SCC configurations described there say GPUs are unavailable in these instances, and a configuration with 512 GB of memory is missing. Both of those instances are also described as “coming soon,” so the discrepancies might just be the result of webpages that are not yet synced up. Alibaba did not specify where the SCC service would be hosted. However, the announcement did note that there are two existing Alibaba Cloud availability zones in Europe, both of which are in Frankfurt, Germany. One can infer from that that the company will likely set up its expanded cloud offerings there. To jumpstart ecosystem support in Europe, Alibaba is establishing relationships with a number of partners, including the Met Office in the UK, Vodafone in Germany, and Station F in France Although Alibaba is basically starting from scratch in Europe, its entry into that market challenges more established cloud providers, including giants like Amazon Web Service (AWS), Microsoft Azure, and the Google Cloud. And even though Alibaba’s international market share is currently in the single digits, the company commands nearly half the cloud market in China. An article published this week in the South China Morning Post (which is owned by Alibaba) reports that the company’s expansion is Europe is the beginning of a new strategy to offer international products specifically designed for overseas markets, rather than just “internationalizing” the Chinese offerings. The article quotes Derek Wang, chief architect of Alibaba Cloud International, who says the US is a key focus for company. For that market, the strategy is to target Chinese companies doing business in the US as well as American multinationals using of the Alibaba Cloud in China, and who are thus potential users of the company’s offerings in the US.

https://www.top500.org/news/alibaba-launches-european-supercomputing-cloud-service-quantum-computing-platform/

HPE to Deliver Seven Supercomputers to Department of Defense

@Hewlett Packard Enterprise ( #HPE ) has been selected to supply seven new supercomputers for the US Department of Defense ( @DoD) High Performance Computing Modernization Program ( #HPCMP ). All of the systems are HPE SGI 8600 clusters. Four of them will be deployed at the Air Force Research Laboratory ( @AFRL) at Wright-Patterson Air Force Base, while the other three will be installed at the Navy’s DoD Supercomputing Resource Center. The HPE SGI 8600 clusters for the Air Force will be powered by 24-core Intel Xeon Scalable Processors, hooked together with Intel’s Omni-Path fabric. In aggregate, the four clusters will deliver more than 7.3 petaflops of peak performance, backed by over 12 petabytes DDN Lustre storage. The AFRL systems will be used to support research in hypersonics and computational modeling of Air Force weapon systems. The three Navy systems are almost identical, sporting the same 24-core Xeon processors and Omni-Path fabric, along with 12 petabytes of DDN storage. Together, the machines will supply 6.8 petaflops of computational performance. They will be used to support the Navy’s advanced weapons capabilities and global weather modeling. The HPE SGI 8600 is of fairly recent vintage, having been introduced in 2017. Although it doesn’t sport the Apollo brand, the 8600 is included under that HPC-centered portfolio. Rather than being based on the HPE Proliant server, the like the other Apollo products, the 8600 is a derivation of SGI’s ICE XA platform, which HPE upgraded and rebranded last year. Besides offering Intel’s latest Xeon silicon, the system can also be equipped with NVIDIA GPU coprocessors, although none of the Air Force or Navy systems appear to employ this particular option. Other large HPE SGI 8600 systems of note include TSUBAME 3.0, a 12.1-petaflop supercomputer installed at Tokyo Tech, and Electra, a 4.8-petaflop machine deployed at NASA’s Ames Research Center. The 8600 platform can support more than 10,000 nodes, so scaling into multi-petaflop territory is not a problem. The contract for the seven DoD systems is valued at $57 million, which includes the cost of the supercomputing machinery, plus five years of 24/7 support, on-site system administration, and access to applications support personnel from HPE.

https://www.top500.org/news/hpe-to-deliver-seven-supercomputers-to-department-of-defense/

4 steps to implementing high-performance computing for big data processing

In the #bigdata world, not every company needs high performance computing ( #HPC ), but nearly all who work with big data have adopted @Hadoop -style analytics computing. The difference between HPC and Hadoop can be hard to distinguish because it is possible to run Hadoop analytics jobs on HPC gear, although not vice versa. Both HPC and Hadoop analytics use parallel processing of data, but in a Hadoop/analytics environment, data is stored on commodity hardware and distributed across multiple nodes of this hardware. In HPC, where the size of data files is much greater, data storage is centralized. HPC, because of the sheer volume of its files, also requires more expensive networking communications, such as Infiniband, because the size of the files it processes require high throughput and low latency.

https://www.techrepublic.com/article/4-steps-to-implementing-high-performance-computing-for-big-data-processing/

Fluid HPC: How Extreme-Scale Computing Should Respond to Meltdown and Spectre

The #Meltdown and #Spectre vulnerabilities are proving difficult to fix, and initial experiments suggest security patches will cause significant performance penalties to #HPC applications. Even as these patches are rolled out to current HPC platforms, it might be helpful to explore how future HPC systems could be better insulated from CPU or operating system security flaws that could cause massive disruptions. Surprisingly, most of the core concepts to build #supercomputers that are resistant to a wide range of threats have already been invented and deployed in HPC systems over the past 20 years. Combining these technologies, concepts, and approaches not only would improve cybersecurity but also would have broader benefits for improving HPC performance, developing scientific software, adopting advanced hardware such as neuromorphic chips, and building easy-to-deploy data and analysis services. This new form of “Fluid HPC” would do more than solve current vulnerabilities. As an enabling technology, Fluid HPC would be transformative, dramatically improving extreme-scale code development in the same way that virtual machine and container technologies made cloud computing possible and built a new industry. In today’s extreme-scale platforms, compute nodes are essentially embedded computing devices that are given to a specific user during a job and then cleaned up and provided to the next user and job. This “space-sharing” model, where the supercomputer is divided up and shared by doling out whole nodes to users, has been common for decades. Several non-HPC research projects over the years have explored providing whole nodes, as raw hardware, to applications. In fact, the cloud computing industry uses software stacks to support this “bare-metal provisioning” model, and Ethernet switch vendors have also embraced the functionality required to support this model. Several classic supercomputers, such as the Cray T3D and the IBM Blue Gene/P, provided nodes to users in a lightweight and fluid manner. By carefully separating the management of compute node hardware from the software executed on those nodes, an out-of-band control system can provide many benefits, from improved cybersecurity to shorter Exascale Computing Project (ECP) software development cycles.

https://www.hpcwire.com/2018/02/15/fluid-hpc-extreme-scale-computing-respond-meltdown-spectre/

Sylabs launches Singularity Pro, a container platform for high-performance computing

@Sylabs, the commercial company behind the open source #Singularity #container engine, announced its first commercial product today, #SingularityPro. Sylabs was launched in 2015 to create a container platform specifically designed for scientific and high performance computing use cases, two areas that founder and CEO @Gregory Kurtzer, says were left behind in the containerization movement over the last several years. (For an explanation of containers, see this article.) @Docker emerged as the container of engine of choice for developers, but Kurtzer says the container solutions developed early on focused on #microservices. He says there’s nothing inherently wrong with that, but it left out some types of computing that relied on processing jobs instead of services, specifically high performance computing. Kurtzer, who didn’t exactly just fall off the open source turnip truck, had more than 20 years of experience as a high performance computing architect working at the US Department of Energy Lab, where he founded CentOS, an open source enterprise Linux project and Warewulf, which he says has become the most utilized stateless HPC cluster provisioner. He decided to shift his attention to containers when founded Sylabs and launched the first open source version of Singularity in April, 2016. Even then, he had a vision of creating a commercial version of the product. He saw Singularity as a Docker for HPC environments, and would run his company in a similar fashion to Docker, leading with the open source project, then building a commercial business on top of it — just as Docker had done. Kurtzer now wants to bring Singularity to the enterprise with a focus not just on the HPC commercial market, but other high performance computing workloads such as artificial intelligence, machine learning, deep learning and advanced analytics. “These applications carry data-intensive workloads that demand HPC-like resources, and as more companies leverage data to support their businesses, the need to properly containerize and support those workflows has grown substantially,” Kurtzer wrote in a blog post announcing the enterprise product. Even though Singularity is designed to handle different kinds of workloads, it still works with container orchestration tools, specifically Kubernetes and Mesos, and it is also compatible with Microsoft’s Azure Batch tool and other cloud tools. Kurtzer indicated Sylabs currently has 12 employees, and is operating on an undisclosed amount of seed money. It was funded by RStor, a startup itself currently operating in stealth mode.
https://techcrunch.com/2018/02/08/sylabs-launches-singularity-pro-a-container-platform-for-high-performance-computing/?ncid=mobilenavtrend

Exclusive Q&A: Dell sets out new IoT strategy

In an exclusive interview, Internet of Business talks to @Dell EMC ’s @Dermot O’Connell about the vendor’s new vision for the #IoT, and how companies should be developing their IoT strategy from both technology and business standpoints. Dell recently announced that it is investing $1billion over the next three years in a new IoT division. The unit will focus on developing next-gen products, research, and partnerships across everything from driverless cars to smart light bulbs, with a focus on edge computing and the distributed core. More, the company has just announced the launch of three new servers designed for software-defined environments, edge, and high-performance computing ( #HPC ) (the PowerEdge R6415, R7415, and R7425). IoB spoke to Dermot O’Connell, Dell VP and General Manager, OEM and IOT Solutions EMEA, about the company’s strategy and roadmap for the IoT, and how its customers should be following Dell on its journey. Internet of Business: What are the three developments you expect to have the biggest impact on the IoT during 2018? Dermott O’Connell Dermot O’Connell: “The move to a distributed core will have a huge impact in 2018. Managed well, this will result in valuable data coming into businesses from connected devices. Managed poorly, the value of the data could be completely lost in fragmentation, losing the potential advantages of the IoT in the process. “There will also be a change in what it means to have everything connected and working together. While previously a centralised cloud computing infrastructure brought everything together in theory, the future will consist of systems that are so interconnected and integrated they’ll become a single ecosystem working seamlessly. “Intelligence from the IoT today will feed the intelligence of the IoT tomorrow. As is the nature of AI and machine learning, data will only feed systems so that they become more intelligent, and ultimately generate more valuable data. This is why it’s so critical to have systems that work together and manage IoT data effectively. Any failure to establish that capability today means that the arrival of IoT services at scale will move much further into the distance.” Which sectors do you think will see significant growth in IoT deployments during 2018? “It’s hard to pinpoint any one industry that is reaping more of the rewards from the IoT than others, largely because we’re still at the tip of the iceberg. But if farmers can use data from IoT devices to help them grow crops or rear livestock more successfully, and if doctors can use IoT devices to monitor hundreds of patients simultaneously, then all industries have the potential to grow from harnessing the IoT. “But arguably one of the most interesting industries where gains are certainly being made is manufacturing. With sensors that can indicate faults or errors, as well as integrate with retail and stock management, the whole supply chain can be managed and made more efficient through the use of the IoT. • See IoB’s Internet of Manufacturing events in Munich (on now) and Chicago (June 2018). “If we can transform the supply chain behind the products bought by any number of industries, then the resulting savings – and the more intelligent systems that will be created as a result – could have a domino effect across other industries worldwide.” Can you explain the distinctions between edge, core, and cloud when it comes to IoT deployments, and the advantages of each? “It’s endpoint data that’s being processed at the edge, making it the most valuable in providing information on real-world scenarios; that’s the starting point. By using artificial intelligence to analyse this data, it can be cleansed, normalised, and used to make split-second decisions at the edge. “This can then feed into the more intelligence-driven stage of IoT, at the core. Data streams are brought together in the core, and then correlated, curated, and used by other IoT devices. “With all the data now in one place and in a valuable format, it can be transferred to the cloud for broader analysis to inform business decisions, and thus become part of an organisation’s intelligence over time. It’s here that the short-term gains from the IoT can be realised, and its long-term potential can be designed.” What role will AI play in the future of the IoT? “AI and the IoT are intrinsically linked; it’s a single ecosystem. There’s no point in having a connected device if it can’t feed into the bigger picture of generating insight for the business, which can then be acted on automatically. For example, with data coming from smart sensors, when patterns emerge or anomalies appear, the technology needs to be able to take appropriate action by harnessing AI.” Dell is investing $1billion over the next three years on a new division that targets connected technologies. What was the thinking behind the strategy? “The beauty of Dell Technologies is that, just like the IoT ecosystem itself, it is interconnected and interoperable. We understand how to make technologies work together and what’s required – from both a technical and strategic alliance standpoint – at each stage of the IoT process, from the edge to the core to the cloud. “The new IoT division will be devoted to this. From products, solutions, and labs to our partners, the IoT ecosystem will continue to evolve. And we’re dedicated to evolving with it.” What three key pieces of advice would you give to any organisation that is formulating a strategy for its first IoT project? “There are three critical pieces of the puzzle to have in place. First, the vision. An organisation needs to identify and prioritise its business use cases for IoT data, to ensure they know what success looks like. “Second, organisations need a team that knows exactly how to implement an IoT architecture and roadmap for implementation. This is a complex process involving a variety of technologies, and so teams need to be co-ordinated to make an IoT project a success. The importance of a planned deployment can’t be underestimated! “Finally, generating and using the analytics created. Never before have so many organisations had access to such credible data. “As a result, businesses need to architect a way to get this data back into the system and inform machines that can make a real-world difference using this previously untapped intelligence. Only with these three elements in place can organisations realise their digital future with the IoT.” Internet of Business says Dell’s simple description of the distinct roles that edge computing, the distributed core, and cloud platforms play in IoT deployments is valuable, with the edge environment being of particular importance for time-critical actions – such as an autonomous vehicle’s need to avoid hitting a pedestrian. Because of the drift of so much enterprise IT into the cloud, it’s easy for decision-makers to assume that cloud services will be fast enough to handle the IoT at every stage, and so Dell’s focus on the edge and the distributed core is particularly useful in challenging those perceptions. We wish the company luck with its new IoT division, and – separately – with its investment in a number of IoT startups, across areas such as IoT security and smart data analytics

https://internetofbusiness.com/exclusive-qa-dell-talks-iot-strategy/

Researchers Advance User-Level Container Solution for HPC

Most scientific computing facilities, such us #HPC or #gridinfrastructures, are shared among different research disciplines, and thus the system software environment needs to be generic enough to accommodate different user and applications profiles; they are multi-user environments. Because of managerial and technical constraints, such infrastructures cannot afford offering every research project a tailored environment in their machines. Therefore the interest of exploring the applicability of containers technology on such systems is rather evident from the end-user point of view. Researchers need then to customize their applications software to fit the computing center environment at the level of system software and batch system. #Containers provide a way to pack and deploy software including all the dependencies in a way that can be executed in a seamless way, independently of the underlying #Linux Operating System and environment. The main benefit of integrating the execution of containers in #HPC systems would then be to provide a way to execute applications homogeneously across different resource centers. The flagship container software, @Docker, cannot be used in a satisfactory way on HPC systems, grids and in general multi-user oriented infrastructures. Deploying Docker on such facilities presents a number of problems related to the fact that within the container, processes are executed with the root id. This raises security concerns among system managers, as the Docker root might be able to gain access to root privileges in the host machine. Also, when executed as root, the processes escape from the usual managerial limits on resource consumption or accounting, imposed on regular users at shared facilities. User-level tools The user-level tool udocker provides a layer for users to execute Docker containers, that by definition, does not require the intervention of the system administrators. Udocker combines the pulling, extraction and execution of Docker containers without requiring privileges. The Docker image is extracted on a user-space filesystem area, and from there on, it is executed in an chroot-like environment. udocker provides a command line interface that mimics Docker, providing a subset of its commands to be able to handle Docker images at the level of pulling, extracting and execute containers “á la Docker”. Processes are run without privileges under the regular user id, under the same process tree, thus facilitating the enforcement of the managerial limits imposed to regular users in HPC or grid resource centers. udocker provides several ways, depending on the application and host environment, to execute containerized applications. It is also possible to access specialized hardware like Infiniband for MPI jobs, or GPGPUs, making it adequate to execute containers in batch systems and HPC infrastructures. udocker enables the execution of Docker containers with different engines based on intercepting system calls. Depending on the application requirements the user may choose to run in one execution mode or another. For instance CPU-intensive applications may use udocker in the ptrace execution mode, to intercept and modify pathnames; if the application is I/O intensive the interception of system calls via library pre-loading using the Fakechroot execution mode is a more adequate way to run the container. All the tools and libraries required by udocker and its execution modes are provided with udocker itself. The udocker execution mode RunC employs the technology of user namespaces to run the containers in rootless mode. This feature can be used with modern Linux distributions with kernels from 3.9 on. However most HPC systems are conservative environments and it will take some time until they will be able to support this execution mode. Regarding impact in performance, in the figure presented below we have plotted the weak scaling performance of openQCD, a comprehensive software package to run Lattice QCD simulations (a CPU-intensive application) from 8 to 256 cores. As we see, the performance of the containerized version of openQCD is slightly higher than the one on the host itself. This is especially so when the execution takes place within a single node (the test machine has 24-core nodes). This behavior has been reported consistently by container users across different hardware and system software settings, and it is related to the better libraries available in the more advanced versions of the operating systems inside the container. Clearly this feature opens the door to container exploitation in HPC mainframes since there the software system is by necessity very conservative.  Figure Caption: Weak Scaling performance of openQCD with a local lattice of Volume=32^4. The tests have been performed on the Finisterrae-II HPC system at CESGA (Spain). Since its first release in June 2016 udocker expanded quickly in the open source community. It is being used in large international collaborations like the case of MasterCode, a leading particle physics phenomenology collaboration, which uses udocker to handle the library complexity of the set of codes included in the MasterCode. It has also been adopted by a number of software projects to complement Docker. Among them openmole, bioconda, Common Workflow Language or SCAR. System Administration level Beyond the user level, several solutions have been developed in recent times to support system administrators in deploying customized containers for their users. These solutions rely on the installation of system software by the system administrator, which also is in charge of preparing the containers that the users are authorized to run on the system. The most popular of these tools is Singularity. Singularity can be downloaded and installed from source or binaries, and must be installed by root for the software to have all the functionalities. Singularity binaries are therefore installed with SUID and need be deployed in a filesystem that allows SUID. Given the security concerns on network filesystems regarding SUID, Singularity is normally installed in a directory locally accessible to the users (i.e., not network-mounted). Singularity offers its own containers registry, the Singularity Hub, and its own specification to create containers, the Singularity Recipe (i.e., the Singularity equivalent of the Dockerfile specification). The default container format is squashfs, which is a compressed read-only Linux file system, where the images need to be created by root. It also supports a sandbox format, in which the container is deployed inside a standard Unix directory, much like udocker. In particular, executing udocker in Singularity execution mode will cause the container to be executed via Singularity if installed in the system. In order to do this udocker exploits the sandbox mode. The container building environment of Singularity belongs to root. Containers may be built either from a Singularity recipe, from a previous container coming from the Singularity Hub, or importing a container from the Docker repository. Notice that the Singularity format for containers is not compatible with Docker; therefore, in the latter case the container needs to be converted to the Singularity format. Once the container exists, it can be executed by a regular user in a way analogous to Docker. These containers can also be checked at the binary level, at the level of sensitive content of the filesystem for example, or even for particular features defined by the system administrator. The comparison of the most popular tools, udocker and Singularity, shows that they have a completely different scope, and the selection of one solution or another depends on the priorities at the user level and the computing center management policies. Singularity is a system administration level tool, to be installed at this level, giving the managers of the infrastructure full control of which containers are run into the system or not. Udocker however is a user tool that acts as a layer over different execution methods, enabling regular users to run containers in their own user space, much in the philosophy of the jailed systems.

https://www.hpcwire.com/2017/12/18/researchers-advance-user-level-container-solution-hpc/

Lenovo and Intel to Deliver Next-Generation Supercomputer to Leibniz Supercomputing Center

RESEARCH TRIANGLE PARK, N.C., Dec. 14, 2017 — @Lenovo (SEHK:0992) (Pink Sheets: #LNVGY ) Data Center Group and @Intel will deliver a next-generation #supercomputer to @Leibniz Supercomputing Centre (LRZ) of the Bavarian Academy of Sciences in Munich, Germany. One of the foremost European computing centers for professionals in the scientific, research and academic communities, LRZ is tasked with managing not only exponential amounts of big data, but processing and analyzing that data quickly to accelerate research initiatives around the world. For example, the LRZ recently completed the world’s largest simulation of earthquakes and resulting tsunami’s, such as the Sumatra-Andaman Earthquake. This research enables real-time scenarios planning that can help predict aftershocks and other seismic hazards. Upon its completion in late 2018, the new supercomputer (called SuperMUC-NG) will support LRZ in its groundbreaking research across a variety of complex scientific disciplines, such as astrophysics, fluid dynamics and life sciences, by offering highly available, secure and energy-efficient high-performance computing (HPC) services that leverage industry-leading technology optimized to address the a broad range of scientific computing applications. The LRZ installation will also feature the 20-millionth server shipped by Lenovo, a significant milestone in the company’s data center history. “Lenovo is committed to providing research institutions like LRZ with not only sheer computing power, but a true, end-to-end solution that can help effectively and efficiently solve critical humanitarian challenges. We’re pleased to be working on this next-generation project in partnership with Intel,” said Scott Tease, Executive Director, HPC and AI, Lenovo Data Center Group. “The new SuperMUC-NG installation will provide LRZ with greater compute power in a smaller data center footprint with drastically reduced energy usage through innovative water-cooling technology, offering researchers a comprehensive supercomputing solution that packs more performance than ever to accelerate critical research projects.” The SuperMUC-NG will deliver a staggering 26.7 petaflop compute capacity powered by nearly 6,500 nodes of Lenovo’s recently-announced, next-generation ThinkSystem SD650 servers, featuring Intel Xeon Platinum processors with Intel Advanced Vector Extensions (Intel AVX 512), and interconnected with Intel Omni-Path Architecture. The new system will also include the integration of Lenovo Intelligent Computing Orchestrator (LiCO), a powerful management suite with an intuitive GUI that helps accelerate development of HPC and AI applications, as well as cloud-based components to empower LRZ researchers with the freedom to virtualize, process the vast amount of data sets and expediently share results with colleagues. To address the often-astronomical operational expenses generated by high-performance computing (HPC) infrastructure, the new SuperMUC-NG supercomputer will benefit from Intel technical optimizations and also feature cutting-edge water cooling technology from Lenovo. In combination with Lenovo Energy Aware Run-Time (EAR) software, a technology that dynamically controls system infrastructure power while applications are still running, Lenovo’s comprehensive water-cooling technology delivers 45 percent greater electricity savings to LRZ as compared to a similar, standard air-cooled system. Together, these energy efficiency innovations will help further reduce the research center’s carbon footprint and total cost of ownership. “Global research leaders like LRZ are driving insights that address not only some of the most complex problems we face, but that also make meaningful improvements in all of our lives,” said Trish Damkroger, Vice President of Technical Computing at Intel. “Intel offers the technical foundation that, when combined with the solution expertise of Lenovo, delivers the efficient performance and ease of programing to help LRZ’s researchers drive more discoveries with deeper analytics than have ever been possible before.” Once operational, the LRZ SuperMUC-NG system is expected to place on the industry-wide TOP500 list.

https://www.hpcwire.com/off-the-wire/lenovo-intel-deliver-next-generation-supercomputer-leibniz-supercomputing-center/

Intel® Omni-Path Architecture and Intel® Xeon® Scalable Processor Family Enable Breakthrough Science on 13.7 petaFLOPS MareNostrum 4

In publicly and privately funded computational science research, dollars (or Euros in this case) follow FLOPS. And when you’re one of the leading computing centers in Europe with a reputation around the world of highly reliable, leading edge technology resources, you look for the best in supercomputing in order to continue supporting breakthrough research. Thus, Barcelona Supercomputing Center (BSC) is driven to build leading supercomputing clusters for its research clients in the public and private sectors.  MareNostrum 4 is nestled within the Torre Girona chapel “We have the privilege of users coming back to us each year to run their projects,” said Sergio Girona, BSC’s Operations Department Director. “They return because we reliably provide the technology and services they need year after year, and because our systems are of the highest level.” Supported by the Spanish and Catalan governments and funded by the Ministry of Economy and Competitiveness with €34 million in 2015, BSC sought to take its MareNostrum 3 system to the next-generation of computing capabilities. It specified multiple clusters for both general computational needs of ongoing research, and for development of next-generation codes based on emerging supercomputing technologies and tools for the Exascale computing era. It fell to IBM, who partnered with Fujitsu and Lenovo, to design and build MareNostrum 4. MareNostrum 4 is a multi-cluster system which main cluster and data storage are interconnected by the Intel® Omni-Path Architecture (Intel® OPA) fabric. A general-purpose compute cluster, with 3,456 nodes of Intel® Xeon® Scalable Processor Product Family will provide up to 11.1 petaFLOPS of computational capacity. A smaller cluster delivering up to 0.5 petaFLOPS is built on the Intel® Xeon Phi™ Processor 7250. A third small cluster up to 1.5 petaFLOPS will include Power9* and Nvidia GPUs, and a fourth one, made of ARM v8 processors, will provide other 0.5 petaFLOPS of performance. And an IBM storage array will round out the system. All systems are interconnected with the storage subsystem. MareNostrum 4 is designed to be twelve times faster than its predecessor. Spain’s 13.7 petaFLOPS supercomputer contributes to the Partnership for Advanced Computing in Europe (PRACE) and supports the Spanish Supercomputing Network (RES). “From my point of view,” stated Girona, “Intel had, at the time of the procurement, the best processor for general purpose systems. Intel is very good on specific domains, and they continue to innovate in other domains. That is why we chose Intel processors for the general-purpose cluster and Intel Xeon Phi Processor for one of the emerging technology clusters, on which we can explore new code development.” The system was in production by July 2017 and placed at number 13 in the June 2017 Top500 list and number 16 on the November 2017 list. “The Barcelona Supercomputing Center team is committed to maximizing MareNostrum in any way we can,” concluded Girona. “But MareNostrum is not about us. Our purpose at BSC is to help others. We are successful when the scientists and engineers using MareNostrum’s computing power get all the data they need to further their discoveries. It is always rewarding to know we help others to further cutting-edge scientific exploration.”

https://www.hpcwire.com/2017/12/11/intel-omni-path-architecture-intel-xeon-scalable-processor-family-enable-breakthrough-science-13-7-petaflops-marenostrum-4-2/

Supermicro Announces Scale-Up SuperServer Certified for SAP HANA

SAN JOSE, Calif., Dec. 7, 2017 — @Super Micro Computer, Inc. (NASDAQ: #SMCI), a global leader in #enterprisecomputing, #storage, #networkingsolutions, #greencomputing technology and an @SAP global technology partner, today announced that its latest 2U 4-Socket SuperServer (2049U-TR4) supporting the highest performance Intel Xeon Scalable processors, maximum memory and all-flash SSD storage has been certified for operating the SAP HANA platform. SuperServer 2049U-TR4 for SAP HANA supports customers by offering a unique scale-up single node system based on a well-defined hardware specification designed to meet the most demanding performance requirements of SAP HANA in-memory technology. “Combining our capabilities in delivering high-performance, high-efficiency server technology, innovation, end-to-end green computing solutions to the data center, and cloud computing with the in-memory computing capabilities of SAP HANA, Supermicro SuperServer 2049U-TR4 for SAP HANA offers customers a pre-assembled, pre-installed, pre-configured, standardized and highly optimized solution for mission-critical database and applications running on SAP HANA,” said Charles Liang, President and CEO of Supermicro. “The SAP HANA certification is a vital addition to our solution portfolio further enabling Supermicro to provision and service innovative new mission-critical solutions for the most demanding enterprise customer requirements.” Supermicro is collaborating with SAP to bring its rich portfolio of open cloud-scale computing solutions to enterprise customers looking to transition from traditional high-cost proprietary systems to open, cost-optimized, software-defined architectures. To support this collaboration, Supermicro has recently joined the SAP global technology partner program. SAP HANA combines database, data processing, and application platform capabilities in-memory. The platform provides libraries for predictive, planning, text processing, spatial and business analytics. By providing advanced capabilities, such as predictive text analytics, spatial processing and data virtualization on the same architecture, it further simplifies application development and processing across big-data sources and structures. This makes SAP HANA a highly suitable platform for building and deploying next-generation, real-time applications and analytics. The new SAP-certified solution complements existing solutions from Supermicro for SAP NetWeaver technology platform and helps support customers’ transition to SAP HANA and SAP S/4HANA. In fact, Supermicro has certified its complete portfolio of server and storage solutions to support the SAP NetWeaver technology platform running on Linux. Designed for enterprises that require the highest operational efficiency and maximum performance, all these Supermicro SuperServer solutions are ready for SAP applications based on the NetWeaver technology platform such as SAP ECC, SAP BW and SAP CRM, either as application or database server in a two- or three-tier SAP configuration. Supermicro plans to continue expanding its portfolio of SAP HANA certified systems including an 8-socket scale-up solution based on the SuperServer 7089P-TR4 and a 4-socket solution based on its SuperBlade in the first half of 2018.

https://www.hpcwire.com/off-the-wire/supermicro-announces-scale-superserver-certified-sap-hana/