The University of Melbourne is entering the final stages of an implementation of around 1.6 petabytes of distributed software-defined storage on commodity hardware for researchers across Victoria, using an open source platform called Gluster.
The large-scale storage refresh project kicked off in October, with the capacity to be made available to researchers across the state through the VicNode partnership with Monash University.
University of Melbourne principal advisor of research computation strategy Steven Manos told iTnews the university has progressively been moving away from high-end disk and NetApp solutions in favour of software-defined storage on commodity hardware using Gluster.
“For us at the University of Melbourne, software-defined networking is the way forward," Manos said.
"Having a platform on which we can deploy new types of storage services without having to necessarily purchase entirely new infrastructure is something that's very important to us."
University of Melbourne system administrators team leader Linh Vu said the storage upgrade is a response to growing file sizes used in academic research projects.
“In the past, VicNode had a limit of 100 terabytes per volume, but it became very limiting as some research projects progressed well past that 100 terabyte mark,” Vu said.
According to Vu, the large files meant a distributed file system optimised to handle sequential I/O on a large scale was required.
“In the past, if you wanted to buy a storage server, you’d buy a commodity box and install NFS, kernel daemon, or Samba daemon. That is OK if you have a small group of users, and everything is stored on that one box,” Vu said.
In the Gluster platform’s file system, known as GlusterFS, each server exports its storage via what are known as ‘bricks’. The platform then assembles these ‘bricks’ into a volume that spans across all the participating nodes, making it appear as if it’s a single storage device.
No comments:
Post a Comment