GPFS breaks file scanning record by 37x

Editor's note: Guest author Richard Freitas is a computer scientist in storage class memory at IBM Research - Almaden.

In our increasingly instrumented, interconnected and intelligent world, individuals and companies are struggling to deal with the explosive growth of data. According to a recent study there will be 1800 exabytes (EB) of digital data in 2011, up from 220 EB in 2007. Such growth places the data user under tremendous pressure to turn data into actionable insights quickly, while straining the world’s IT infrastructure to its limits. This forces the data user to manage the explosive growth of data and storage using tools designed for easier times.

Existing storage management solutions find it difficult to provide timely storage backup, migration to appropriate performance tiers, replication and distribution. In many cases, users go without the kind of daily backup that industry experts would expect of a large data store. As new applications emerge in industries from financial services to healthcare, traditional systems will be unable to process data on this scale, leaving users exposed to critical data loss.

Anticipating these storage challenges decades ago, Almaden researchers created GPFS, a highly scalable, clustered parallel file system. Already deployed in enterprise environments with one billion files to perform essential tasks such as file backup and data archiving, this technology’s unique approach overcomes the key challenge in managing unprecedented large file systems with the combination of multi-system parallelization and fast access to file system metadata stored on a solid-state storage appliance.

GPFS advanced algorithms make possible the full use of all processor cores on all of these machines in all phases of the task (data read, sorting, and rules evaluation). GPFS exploits the excellent random performance and high data transfer rates of the 6.5 TB solid-state metadata storage. The solid-state appliances sustainably perform hundreds of millions of IO operations, while GPFS continuously identifies, selects and sorts the right set of files from the 10 billion-file file system. Performing this selection in 43 minutes was achieved by using GPFS running on a cluster of ten 8-core systems and four Violin Memory solid-state memory arrays. This is 37 times the rate that was achieved in 2007 on a file system containing 1 Billion files.


You can read more about this IBM Research breakthrough here.

Community: Help through horses for autism

When Carolyn Wallace, IBM Research - Almaden Client Relationship Manager learned about IBM's Centennial Day of Service, she was quickly motivated to rally around a particular cause that affects her personally: autism. Carolyn's 10-year old grandson, Max, has autism, and in the spirit of IBM's 100th year, she decided to gather the efforts of her colleagues and friends in the area to donate their time to Gilroy's DreamPower Horsemanship Camp.

The DreamPower Horsemanship Camp offers a number of therapeutic services to individuals affected by physical and mental disabilities, ranging from equine facilitated psychotherapy, groups for cancer patients and confident rider clinics to couples communication workshops and therapeutic horsemanship lessons for children, teens and adults.

"Max struggles with speech and has been in speech therapy for several years," Carolyn said. "I don't know if it's my imagination or wishful thinking, but I believe his speech is much clearer when he is riding."

Founded by Licensed Marriage and Family Therapist Martha McNiel, the program was developed with a specific mission in mind: Building lives filled with dreams, and the power to make dreams come true. Martha is a NARHA Registered Therapeutic Riding Instructor and a Certified Equine Interaction Professional in Mental Health. Along with her staff, she hosts groups at the camp ranging from juvenile delinquents to individuals with cerebral palsy - all with the intent to provide emotional and physical healing for their clients.

With an IBM cash grant of $5,000 used to support the week-long DreamPower Horsemanship Camp for Children on the Autism Spectrum, IBMers and DreamPower staff worked together to create 15 hours worth of horse riding, grooming, arts and crafts and science projects for the children to participate in. Ranging in age from 4 - 13, 16 camp participants from San Jose and Morgan Hill learned about things like horse safety and care, horseriding commands and coordination activities, and also played interactive games like giant Jenga. On the last day of camp, the group got to decorate a mini horse with paint and glitter.

"We have had Max in therapeutic horseback training since November of 2010," Carolyn said. "He is very enthusiastic about his riding classes on Monday evening - he gets ready and stands by the door when it gets close to time to depart for class."

In order to qualify for the grant, which was proposed as part of IBM's Centennial service commitment, the program had to incorporate a science and technology component. By enlisting the help of Schmahl Science - in essence, a science lab on wheels - the kids had an opportunity to explore science, math and engineering through projects tailored specifically to their needs.

Mayor Steve Tate, City of Morgan Hill, coincidentally a 38-year IBMer, supports the philanthropic efforts of IBM, and is proud to have the affiliation attached to his resume: "they're community builders. And that's what we're all about here in Morgan Hill," he said.

In fact, IBMers in the Silicon Valley have donated time directly to the South County in a number of different ways. Community teams have partnered with Solorsano Middle School in Gilroy to put on Lego Robotics Workshops and to invite students to IBM's Annual Technology Camps. Around 25 IBMers volunteer for Rebekah's Children's Services by cleaning up, painting, organizing or planting at various locations. Overall, IBM has donated $182,494 in cash and equipment to the City of Morgan Hill and $10,208 to the City of Gilroy since 2006.

IBM is committed to serving local communities - in fact, IBMers volunteer more than any other company in the world. Since 2003, more than 11 million hours of service has been recorded. In the Silicon Valley alone, IBMers are committed to donating 32,000 hours of service in 2011.



Read about IBM's involvement in the DreamPower Horsemanship Camp in the Morgan Hill Times.

Photo captions, top to bottom:

A camp participant straps on her helmet with the help of a volunteer in anticipation of a therapeutic ride through the arena.

IBM Fellow, CTO and VP, Technical Strategy and Senior Location Executive for the Silicon Valley, Jai Menon, visits the camp to support the nearly 100 IBM volunteers and to express his praise for a program dedicated to an extremely relevant cause.