Social Business Research at Almaden

Ten years ago there was a significant shift in the way people interacted with each other: the web came to the workplace and became a serious business tool for organizations in industries of every kind. The evolution continues with the coming of age of Social Business as social computing, analytics, policies, governance and cultures are integrated into enterprise design and organizations are focused on socially-enabling business processes.

Jeff Pierce, IBM Research - Almaden
Mobile Computing Research Lead

In a recent media event at IBM Research - Almaden focused on how Social Business is helping organizations around the world embrace a new culture of transparency and collaboration, IBM GM for Social Business Alistair Rennie spoke to about 6 reporters and 4 analysts about IBM's strategy and IBM mobile computing research lead Jeff Pierce led a discussion about user studies and mobile devices in the Social Business world.

The attendees had access to a number of mobile/social Research demonstrations, including the crowdsourcing mobile phone app CreekWatch, IBM Connections and a few Twitter analytics projects developed out of the Social Media Analytics & Engagement area of IBM Research - Almaden. Much of the research is developed out of behavioral studies - including the interesting result that even as early as 2007 some users were already managing their activities across an average of 6 devices - many of which were published in the last 3 years:

Smart Phone Use by Non-Mobile Business Users
Presented at Mobile HCI 2011

Understanding Information Preview in Mobile Email Processing

Presented at Mobile HCI 2011

No Smart Phone Is an Island: The Impact of Places, Situations, and Other Devices on Smart Phone Use

IBM Research Report

Triage and Capture: Rethinking Mobile Email

IBM Research Report

Also! ReadWriteWeb's article "IBM Rethinking Mobile Email" from December 2011

"It's on my other computer!": Computing with Multiple Devices

Presented at CHI 2008

Following the event, influential Forbes blogger Haydn Shaughnessy reported on the evolution of social media to social business highlighting IBM's embrace of social technologies across the organization. The article discusses IBM's evolution into a social business and cites IBM as a category leader in social software. 

Jennifer Okimoto, Social Business Consultant from IBM's Global Business Services division, and speaker at this event, shared her thoughts on "Social Business Game Changers," which include behaviors, skills and attitudes: 

You can learn more about IBM's Social Business here. 

IBM storage gets more Hollywood hardware

In October, we announced that IBM's Linear Tape File Storage project earned a spot at the Emmy Award Ceremony in Hollywood, where the team received a golden statuette for "improving the ability of media companies to capture, manage and exploit content in digital form, fundamentally changing the way that audio and video content is manager and stored" (as stated by the Academy of Television Arts & Sciences).

Since then, IBM was awarded the HPA Engineering Excellence Award, presented by the 

Hollywood Post Alliance (HPA) for the invention of the Linear Tape File System, providing "a simple, cost efficient way to access and manage massive archives of data and digital assets." Arnon Amir, an Almaden researcher, received the award on behalf of IBM at the ceremony, at the Skirball

Cultural Center, Los Angeles.

Arnon shared some thoughts about the work he's done with his team to bring LTFS to this level of excellence. 

IBM has been working, for a number of years, on developing innovative storage solutions for the M&E 

market. LTFS is a prime example of IBM's ability to think out-of-the-box, develop a new technology and work with our partners and customers to lead a transformation through the industry. This collaboration, across multiple geographies, markets and technologies, demonstrates IBM's global leadership in storage systems.

When we started, the team was very small, however we only had a couple of shared tape drives in our lab to work with. One of core LTFS developers, Lucas Villa Real, worked remotely from IBM Brazil.  We set for him a webcam in our lab, showing the front of the tape drive 24 hours a day. Looking at the live video he could see the tape drive loading and unloading the tape as he tested his software. This must have been one of the most boring live videos ever streamed. :-) 

Later on, as the technology was hardened and transferred to product, the development team grew and span developers at multiple IBM locations Asia, Europe and America, with plenty of tape drives at all locations.

Seeing LTFS forming up from an idea to a product, now being adopted by many of our partners and customers in such a short time is truly fascinating. It is a great privilege to work with our excellent teams worldwide and to collaborate with so many IBM partners and customers. I feel very fortunate to have a part in this project.

IBM was also recognized as a Visionary M&E Company at this year's Storage Vision Conference (held in conjunction with the CES show). Also, December's issue of Storage Magazine Online listed LTFS as one of the six hot storage technologies for 2012, and was featured in Datamation's, "How Open Source Could Drive a Tape Storage Comeback" article. 

More coverage and analyst reviews: 

Computer Technology Review: LTO-5 and LTFS: Shaking the Pillars of Heaven

Backup Central: LTFS: Crazy Like a Fox

Clipper Group: Dealing with Cool and Cold Data — and Getting It 'Just Right' 

B&L Blog: LTO-5: Changing the Way You Look at Tape

Dave Ferrucci on Watson: How it all began and what's next

In his first west coast appearance, the IBM Watson creator gave a talk at the Computer History Museum in Mountain View, CA, followed by an exhibition Jeopardy! match between Watson and two brave volunteers with comedic host, IBMer Eric Brown. 

Computer History Museum
president John Hollar

In front of what Computer History Museum president John Hollar called "the largest crowd for a Revolutionaries lecture" that he's ever seen, IBM Watson principle investigator Dave Ferrucci sat with Financial Times' Richard Waters on November 15th for a conversation about "A Computer Called Watson." To the audience of about 450 Silicon Valley techies, influencers, teenagers and inspired engineers, Dave kicked off the conversation by explaining how Watson came about; and it began with the notion of natural language processing, namely, contextual aspects of language.


"At our house, I'd always call the kids down to see something 'interesting' that I'd done - some type of experiment or science-related thing," Dave said. "After enough of these demonstrations, my daughter started to associate the word 'interesting' with 'boring' - so there's a little about language context."

IBM Watson principle investigator David Ferrucci (left)
with Financial Times' Richard Waters

It turns out Dave was headed toward a career in medicine and was pursuing an M.D. rather than a Ph.D. But the biology major quickly developed a fascination with artificial intelligence, and a passion for programming. "I thought it was incredible that you could tell the computer what to do - and that it would do it," he said.

After obtaining his BS in biology from Manhattan College, he pursued computer science with an emphasis in knowledge representation and reasoning at Renesslaer Polytechnic Institute, completing his Ph.D. in 1994.

Since joining IBM in 1995, Dave has contributed largely to the Research function as a computer scientist. But in 2007, when IBM executive Charles Lickel challenged Dave and his team to revolutionize Deep QA and put an IBM computer against Jeopardy!'s human champions, he was off to the races.

"I had to get funding," Dave explains. "I told the executives I could do this in 3-5 years. I kind of just guessed."

The executives bought it, and Dave had a huge task ahead. By assembling a team of eventually 28 researchers in the areas of natural language processing, software architecture, information retrieval, machine learning and knowledge representation and reasoning, Dave created Watson - a computer system, that, using a combination of sophisticated hardware and software, could understand natural language and deliver a single, precise answer with confidence and evidence for its decision.

At the end of the conversation, Dave told the crowd about Watson's new job in the medical field: "We want Watson to enable better judgement by humans in decision-making, whether it be in medicine, law, finance or services," Dave said. "While the human is the ultimate decision-maker, Watson will provide evidence and confidence by scouring millions of sources of related information in a short amount of time."

GigaOm's Stacey Higginbotham takes on IBM's Watson
and Sierra Ventures' Robert Walker in an exhibition match

In an exhibition Jeopardy! game following the talk, IBM's Eric Brown played host to GigaOm's Stacey Higginbotham, Sierra Ventures' Robert Walker, and "oh yeah, our third contestant, Watson, from Yorktown Heights, New York, built by a few computer scientists," an introduction met with laughter that would continue throughout the game.

The humans playfully 'teamed' up against the computer,
high-fiving and fist bumping on each correct answer

The animated human contestants instantly won over the crowd after trailing Watson through the first part of the game. In fact, when Stacey buzzed in with the first correct question for the humans, the crowd went wild.

The night continued in that way, and the human contestants even found themselves getting answers from the crowd, to which host Eric Brown responded: "Watson can't hear you, so humans have an advantage!" As it turned out, the trick was buzzing in before Watson - hard to do unless you're a seasoned Jeopardy! vet like Ken Jennings or Brad Rutter.

As the exciting match wound down, and all three contestants answered the Final Jeopardy! question correctly, Watson came away with the win, but left the auditorium with tremendous enthusiasm for this computer and its impact on the future of technology.

Check back here for a video clip of this event and the IBM Research news blog for more news about Watson.

More from the event:

• GigaOm: Misconceptions in AI: Or why Watson can't talk to Siri
• Computer History Museum Watson Event Photos on Facebook
• IBM Research Watson Event Photos on Flickr
• www.ibmwatson.com

Pre-game (post-practice game) thoughts from contestants*:




*apologies for the incorrect orientation

Meet Almaden's Master Inventors

Each year, IBM selects a new field of Master Inventors as one way of recognizing IBMers who have mastered the patent process, provided broad mentoring, added value to IBM's portfolio, and demonstrated sustained innovation leadership and service. Once selected, a Master Inventor is expected to apply his or her mastery of patent knowledge by actively serving as a:

*leader in the invention community

*mentor to a broad community of inventors

*resource to Intellectual Property Legal office

This year, IBM Research has named forty-nine Master Inventors from its global community of researchers, including five from Almaden. Jim Hafner, Ray Strong, Alshakim Nelson, Tanveer Syeda-Mahmood and Tyrone Grandison, shared some thoughts about their motivations, their inspirations, and what it means to be part of an inventor community that has led the world in patents for 18 years.

Jim's current job is to "do whatever is needed for my project to succeed," but is known around IBM as a stellar mathematician in the Storage Systems department. Ever enthusiastic about the daily challenges - and opportunities to master stubborn ones - Jim credits QBIC (query by image processing) as the project he learned the most from, while his most inventive burst came from working on advanced RAID technologies.

By 6th grade, Jim knew that he was born to be a mathematician -- "it's in my genes," he says. "But I trained for and pursued an academic career for 6 years before coming to IBM." While he says he never aimed for IBM Research, the opportunity came up and "it was irresistible."


Thoughts from Jim on patent leadership and the freedom to explore:

It's exciting working for a corporation (or any institution) that regards invention, innovation and creativity as highly as IBM does. IBM supports the patent process and acknowledges special contributions to that process in order to keep the innovation train moving forward. As a company it needs new ideas to stay competitive.  But I think IBM looks beyond just its own profit margins in its vision.  From the old e-business initiatives to the current Smarter Planet initiative, it's about setting the agenda for world wide growth, improving the lives of people across the globe. That vision takes invention, innovation and creativity.  It's good to be a part of this; to ride this train. 

Favorite hobby: running and scuba diving, both physically challenging but with great rewards.

Ray Strong is a Services Research participant in three projects that involve "designing the future of work; transforming service cost cases into service delivery staffing plans; and providing a new approach to cost forecasting and predictive cost models for services." Driven by anticipation for discovering new ways to have an impact on the business, Ray is also a natural when it comes to math and science. "Growing up, I appreciated the fact that there were often objectively correct answers," he says. "And I enjoyed finding new ways to get at them. In grade school, I missed the coverage of 'subtraction,' so I invented my own (superior) method."

His favorite IBM project so far was a services-based solution offering called Impact of Future Technology, in which he served as the Research Technical Lead.

Ray shares with us his thoughts on invention:

Research means asking questions beyond those required to find good engineering solutions to problems. Almaden houses an outstanding group of research scientists and engineers who have the fun of inventing new stuff that can actually be used by the company. The invention disclosure process provides an early way to record these things and protect IBM's freedom of action at the same time. Our successful combination of science and business is an excellent motivator. What could be better than working with other enthusiastic inventors to explore areas where it is possible to have a big effect and being paid to do it?

Favorite hobby: Off trail mountain hiking and exploring.

Alshakim Nelson, a research chemist in the advanced organic materials group gets to work on technology at a fundamental scientific level, like working on magnetic particles with the tape storage group at Almaden.

Favorite hobby: Spending time with my daughter.

Tanveer Syeda-Mahmood has a very dynamic role as a manager in Almaden's computer science department - her job typically begins with an idea and typically ends with closing a deal. In between, she leads a team of researchers, develops algorithms, codes and presents. When asked what makes her excited to get out of bed and go to work in the morning, she answered, "The possibility of advancing the state-of-the-art technologies and developments in my field, building systems that others can use and change their ways of practicing medicine and freedom of action."

That doesn't stray too far from her passion for math and science that began as a child as well: "I was interested in the possibility of solving hard problems and excited about the discovery of new concepts and theories," Tanveer said. "I was inspired by famous scientists as role models."

The lead researcher behind AALIM, she credits that as her favorite project at IBM. "Motivated by personal incident, I conceived AALIM as a diagnostic aid to clinicians to help in their decision making," she said. "Seeing it used by clinicians around the world is still my dream."

Favorite hobby: Watching movies. 

Tyrone Grandison is currently the Program Manager for IBM's Core Healthcare Services, but in a previous role at IBM Research developed the SoundIndex project for the British Broadcasting Corporation - his favorite project to date.

Asked why he pursued math and science in his youth, Tyrone answered, "I loved the elegance and simplicity of Mathematics, which could be used to model everything around me and in my estimation is the universal language that helps us get a better handle on the world."

As a strong contributer to the success of IBM IP, I asked Tyrone what it meant to him that IBM continuously breaks the worldwide record for patents in a year.

To me, it signifies IBM's commitment to innovation and its recognition of the importance of research in charting the future of the business. It also highlights the fact that IBM has an amazing collection of intelligent, creative and savvy employees who have the freedom to "dream" and make those dreams reality. IBM's patent leadership, worldwide for the past 18 years, is a testament to the corporation's consistent focus on long-term impact; both on society and business. Leadership over such a long period is only possible if innovation and impact are a part of one's DNA.


Personally, intellectual curiosity is the primary driving force behind my activities. The freedom to tackle the hard problems and invent the future is the main reason I am an IBMer. The opportunity to have a positive impact on the world and influence communities, both scientific and commercial, is one that would be hard to have anywhere else.

Favorite hobby: Writing poetry and photography.


Congratulations to Almaden's new class of Master Inventors.

Happy Halloween!

Spirited IBMers at enjoy a warm Halloween day at Almaden

One Million Healthy Children

Last Thursday, IBM Research announced a collaborative research effort with Georgia Tech that aims to improve pediatric healthcare from a payment and policy level. The project, called One Million Healthy Children, will apply advanced modeling and analytics to thousands children's healthcare records to better understand the inefficiencies that make up today's model, which encourages fee for service, rather than prevention and precise diagnoses. By examining factors far beyond the realm of medicine - the region's literary make-up, transportation hubs and access, healthy food stores and socioeconomic status of families - IBM and Georgia Tech hope to provide doctors, policymakers and patients a better idea of how to approach disease. The team will first look at diabetes, which accounts for over $174B in costs in the U.S. per year. 


The modeling technology used in this initiative was born out of IBM Research - Almaden, by a services research team led by Paul Maglio. The Smarter Planet Platform for the Analysis and Simulation of Health is a tool that uses a plug-and-play type of format to insert factors making up a population's health - the systems of systems so to say. IBM researcher on the project, Cheryl Kieliszewski tells us a little more about the project:

We are thrilled to partner with Georgia Tech on the One Million Healthy Children project.  Together, we aim to tackle a difficult, multi-dimensional problem in health – kids experiencing preventable chronic non-communicable diseases, such as obesity and diabetes, at ever younger ages, and which will have a major influence on overall health and well being throughout their lifespan.  To do this, we will explore a number of ways to create and use complex composite models to examine what-if scenarios to improve children's health.  

The partnership brings together Bill Rouse and his team at Georgia Tech, which has expertise in complex adaptive systems, in particular within healthcare modeling, and our team, which has expertise in composite model assembly. On the one hand, it provides the team an opportunity to help understand a difficult societal challenge – keeping our kids healthy – and on the other, it forces the team to confront technical issues, such as how to semi-automatically couple models from different domains, and also social and process issues to support complex decision making. We’re very excited about the potential of this project and for continued work with the Georgia Tech team.

IBM Press Release
"What Is or What If" - IBM Research news blog guest post by Professor William Rouse, Georgia Institute of Technology's Tennenbaum Institute Executive Director, co-chair of the National Academies Healthy America Initiative and member of the National Academy of Engineering
eWeek
HealthITNews
Information Week

We won an Emmy!

Earlier today it was announced that IBM was presented a unique award. Together with FOX, an IBM Research project born out of Almaden won an Engineering Emmy award for Innovation from the Academy of Television Arts and Sciences. According to the Academy, by improving the ability of media companies to capture, manage and exploit content in digital form, IBM and Fox have fundamentally changed the way that audio and video content is managed and stored.

The Linear Tape File System (LTFS), invented by IBM's lauded Research Division, enabled major improvements in digital workflow and dramatic reductions in the costs associated with capturing, storing and repurposing media content while providing dramatic improvements in transfer rates, storage density, automated workflow, meta-data capture and content availability.  Combining digital broadcast and IT standards in a broadcast environment, the LTFS has enabled real-time content recording and high-speed recovery of content to be a broadly-supported, multi-industry solution.

Michael Richmond, Brian Biskeborn, David Pease, Arnon Amir (Almaden Research), and Shinobu Fujihara (Yamato) at the 2010 NAB show where LTFS was announced and released.

In a blog post earlier this year, IBMer Tony Pearson, Master Inventor and Senior Managing Consultant for the IBM System Storage product line, wrote:

"With the capabilities of LTFS, IBM has introduced an entirely new role for tape, as an attractive high capacity, easy to use, low cost and shareable storage media. LTFS can make tape usable in a fashion like removable external disk, a giant alternative to floppy diskettes, DVD-RW and USB memory sticks with directory tree access and file-level drag-and-drop capability. LTFS can allow the for passing of information around from one system or employee to another. And as for high video storage capacity, a 1.5TB LTO-5 cartridge can hold about 50 hours of XDCAM HD video!"

Lead researcher on the project David Pease is a long time storage research expert at the Almaden lab in San Jose, CA. Pioneering many of the tape and disk storage technologies out of IBM Research over the last decade, David recalls a significant factor in deciding to pursue this project the way he did. "We really needed to make the first version open source," David said. "The idea of a file system that was cross-platform and interoperable was key; we wanted people to have an interface they were familiar with, similar to disk with file folders, drag and drop and double-click, but we also wanted to make sure it wasn't tied to only Windows or only Unix. The real future for acceptance for just about any kind of storage technology is interoperability and that people aren't tied to a platform."

David and his team developed LTFS from concept to fruition in just less than 3 years. An impressive feat in the research world, he shares some thoughts about winning an Emmy for his work:


First, I am truly stunned.  This recognition is more than we ever expected so early in the project, and hopefully it reflects the importance of what we've done.  When we started this work, we said that our goal was to change the tape industry and the Media and Entertainment business; it seems that we are well on the way to realizing these goals.

I have to point out that an idea and project like this are never the work of an individual.  From Ed Childers and the other tape experts in Tucson, to the folks at Almaden who encouraged me to get involved with tape (again), to the team of great researchers and developers who worked on this in my group, to the tape specialists in the Yamato Lab who joined my team or worked to support it, I have to say that we couldn't have gotten here without the efforts of each of you.  Thank you all for making this possible!

Fun fact: This past February, David Pease completed a 41-day motorcyle ride from San Jose, CA to Panama City, Panama with 3 companions on different stages of the trip. You can read about his travels through California, Northern, Central and Southeast Mexico, Guatemala, Honduras, Nicaragua, Costa Rica and finally Panama at his blog here.


*****UPDATE Monday, October 31*****

Here's a picture of David with the Emmy. You can see more pictures of the team from the Awards Ceremony here.

IBM Research - Almaden in Councilwoman Nancy Pyle's Newsletter

 

 

 

In 2005, Nancy began her first term on the San Jose City Council as the representative for District 10. A retired teacher of over 25 years and a former Community College Board Trustee, Nancy represents approximately 100,000 residents from Almaden Valley and Blossom Valley.

As District 10's City Councilmember, Nancy takes an innovative approach to solving today's challenges through creativity and collaboration. She has focused city government on the basics, maintaining our streets and parks, keeping our neighborhoods safe, investing resources in youth and senior programs and making city government more accountable.

Councilmember Pyle is the Chair of the Neighborhood Services and Education Committee and a member of the Airport Competitiveness Committee. She serves as the Council liaison to the Parks and Recreation Commission and the Disability Advisory Commission. Councilmember Pyle also serves on the Santa Clara County Emergency Preparedness Council.
 
Nancy Pyle is a graduate of LeMoyne College in Syracuse, New York, where she majored in French (she’s fluent!), and earned a Masters Degree in Educational Administration from the US International University in San Diego. In 1960, Nancy moved to San Jose with her family where she served as a teacher, Community Relations Manager, and Legislative Analyst for San Jose Unified School District. 

Best Paper Award for "An Optimal Algorithm for the Distinct Elements Problem"

Last month, the winners of IBM's 2010 Pat Goldberg Memorial Best Paper competition in computer science, electrical engineering and math were announced. IBM Research - Almaden computer scientist David Woodruff co-authored one of the winning papers, titled "An Optimal Algorithm for the Distinct Elements Problem" with Daniel M. Kane (Harvard University) and Jelani Nelson (MIT) for PODS 2010 (ACM Symposium on Principles of Database Systems).

The Professional Interest Communities at IBM Research (PICs) reviewed a total of close to 120 papers submitted by IBM Research authors and nominated 34 for best paper consideration. A worldwide Research team reviewed the nominated papers and selected four outstanding papers as the award winners.

All of the submitted papers represent IBM Researchers advancing our field and are indicative of our commitment to long-term, exploratory work that can change the way we look at the world.

Below, David explains a bit about his paper, what it might mean for the advancement of mathematical discovery and what it's like working for IBM.

Estimating the number of distinct attributes is a fundamental practical and theoretical problem in database applications dating back to the 1970s. It arises in trying to optimize a query sequence, where keeping the number of distinct elements small at intermediate stages in the sequence ensures the overall running time is low. It is also useful for comparing two data sets, e.g., how many new items did we get by putting the two datasets together? 

The techniques in this paper will be useful for improving the memory and time complexity of a number of fundamental problems in the data stream literature, related to estimating the number of distinct elements, such as estimating rarity, similarity, union sizes of databases, etc. 

They can also be used for estimating the number of distinct elements in the distributed model, sliding window model, and time-decayed model. In the distributed model, there are multiple servers, each holding a database, and we want to estimate the number of distinct elements in the union of the databases with as few communication and computational overhead as possible. In the sliding-window model, which could for instance be used on a router monitoring distinct source-destination traffic passing through it, it may be that we are only interested in the recent traffic, or we may at least want to give more weight to recent traffic. These variations correspond to the sliding-window and time-decayed models. 

My work closes a long-standing problem in the area of data streams. IBM Research has made a sustained effort to design data stream and sub-linear algorithms for a wide variety of problems, e.g., those in graph theory, machine learning, network traffic analysis, numerical linear algebra, and statistics. This work significantly bolsters that effort. 

I’m very grateful for the amazing amount of freedom and that I’ve been given at IBM, and have been able to use this to highly optimize my time and productivity. Interacting with interesting colleagues is one of the most exciting aspects of my job. It’s really enjoyable and a great learning experience.  

Research interests: data stream algorithms, communication complexity, numerical linear algebra, graph algorithms, coding theory, and cryptography 

Inspirational figures: The super theory group at IBM Research - Almaden 

When I'm not working, I'm: home remodeling, playing basketball, practicing chinese, traveling 

Favorite travel spots: Banff, Canada. Hangzhou, China. Venice, Italy. 

David Woodruff is a Research Staff Member in IBM Research - Almaden's Principles and Methodologies Group. He received a B.S. in computer sceince, B.S. in mathematics, M.Eng in computer science and Ph.D in computer science, each from MIT. David also contributes to IBM's cognitive computing initiative, SyNAPSE. 

IBM Research - Almaden Centennial Colloquia "Planet Scale Analytics"

Over 150 eminent leaders in the area of big data and massive scale analytics came together on Tuesday, September 20th for the IBM Research - Almaden Centennial Colloquia to discuss "Planet Scale Analytics." A truly collaborative event, speakers like Gus Hunt, Chief Technology Officer, CIA, Peter Breunig, GM Technology & Strategy, Chevron Corporation and Arvind Krishna, GM Information Management, IBM, convened to discuss the emerging - and explosive - wave of massive scale structured and unstructured data and the rise of analytics for actionable insight.

Leading executives from Juniper Networks, Coca-Cola, Genetench, Yahoo!, Agilent Technologies and Intuit were among attendees who came together to discuss data challenges from a business perspective, and were intrigued to learn about the findings and potential solutions presented in the technical sessions. The academic and research communities were naturally anxious to hear customer concerns and needs. Todd Myers, Chief Scientist for the National Geospatial Intelligence Agency attended to learn about emerging technologies designed to tackle analytics for massive data sets and commented, "IBM does a very good job of convening great speakers and thought leaders."

The three V's of big data - and the fourth

Arvind Krishna opened the morning, describing big data from the perspectives of the 451 group, IDC and IBM - all very similar by definition - large, complex and dynamic. Krishna shared IBM's spin on the big data definition by 3 v's: volume - data at rest, velocity - data in motion and variety - data in many forms. A fourth 'v', Arvind added, is veracity - data in doubt, used to describe 'contradictory data,' or noisy data - ultimately, unstructured data that experts are not sure how to deal with.

In a much anticipated technical discussion about analytics solutions, Shiv Vaithyanathan, IBM Research senior manager in Intelligent Information Systems, again mentioned these 4 v's in his talk titled "Entity, Relationship and 360-degree View of customers." "Veracity is turning noisy data like jargon and acronyms, even wishful thinking and sarcasm into trustworthy insights," Vaithyanathan said. "We're dealing with social media data from hundreds of sources - 10,000 messages per second from over 100 million active users per source - that needs to be combined and correlated to make near real-time decisions."

Using IBM's System T and SQL language, Shiv and the database experts at Almaden have built a 360 degree customer profile built on more than 2,000 rules, analyzing an average of half a terabyte of data on any given day. Vaithyanathan explained the advantages of the analytics tool throughout his talk and via demonstration. System T uses less than 10% of the cores to keep up with Twitter's daily feed with no drop in quality due to its linear scalability; a sharp decline in core usage from both state-of-the-art statistical systems and state-of-the-art open source rule-based systems that require the thousands of cores that System T does not.

Big data challenges in cities and across industries
Through a series of discussions on industry applications such as "21st Century Water Data: Needs and Availability" presented by Peter Gleick of the Pacific Institute and "Big Data in Finance: Quantitative, Qualitative and Relationship Information," by David Leinweber of Leinweber & Co., several challenges proving the need for deep analytical capabilities were shared. "We're entering a world where anyone can be a data source and upload some publicly interesting piece of data," Gleick remarked. "We can share information to an open source database, where you can add search capabilities." Adding to the notion of the changing face of data analytics, Leinweber commented, "Dealing with this is uncomfortable. We need to expect errors and strange innovations." He closed semi-jokingly with a quote by Ogden Nash: "Progress may have been all right once, but it's gone on far too long."

Chevron's Breunig presenting some noteworthy statistics from the oil and energy area: the alarming rate of data transfer and storage that Chevron is faced with daily. At 20PB day, doubling every 2 years with a declining signal to noise ratio, there's not only a dynamic cluster of data that needs to be wrangled, but a challenge in ways to apply that data to equally dynamic dimensions. "In the oil business, subsurface modeling has many uses, and its users have different needs," Breunig said. "This is a big data integration challenge."

Alexandre Bayen of UC Berkeley suggests that like traffic data, water and earthquake data needs to be online, sharing an interesting thought about traffic analytics: monitoring 2% of traffic in real time is sufficient to predict travel time. Additionally, Bayen proposes that tracking one year of yellow cab data in San Francisco can plot a map of the city with relative accuracy. In putting multiple sources of crowdsourced data, public feeds, texts and videos online, analytics can be applied in very similar ways, as long as they have the same basic properties: 1) a mathematical model, very important for physical phenomena, 2) data, 3) inverse modeling and data simulation resulting in estimates and eventually decisions.

Data privacy and sharing

A close collaborator with IBM, Fran Maier of TRUSTe usually has a hard time describing her job: internet privacy. She explained in a panel on privacy that her company works with companies to ensure that their policies and practices are standard and meet whatever promise they make with the consumer. "It's not easy," Fran said, "because privacy, unlike security doesn't have a 'bad guy'. The important elements are transparency and accountability." The cycle of delivering trust in privacy to internet consumers to allow for more interaction and sharing is important - without it, companies are unable to collect and use data sets.

Harriet Pearson, IBM's chief privacy officer, also shared some views on thought leadership from a large corporation perspective. "Companies like Facebook are experimenting in public and seeing where the norms might be," she explained. "One approach by some of the new entrants in the Silicon Valley is to push the envelope, then retract; push, then retract. Alternately, organizations set norms from the outset, or enlist the help of organizations like trustE. Policy making has not accelerated that much. It puts a premium on those who can function in environments of uncertainty and have the confidence to strive forward."

Next steps

Many of the comments shared throughout the day confirmed positive impressions of IBM's data analytics capabilities and the proximal and technical opportunities to impact the local "hotbed" of massive data streams. Others appreciated the opportunity to hear how other companies make use of technologies and models to analyze complicated situations. Al Leung, vice president and partner for acquisitions & logistics at IBM, expressed the advantages of bringing his clients to this forum to learn new techniques and explore some of IBM's offerings in a more personalized format. The conversation of massive scale analytics at IBM Research will continue locally and from a company-wide perspective, focusing on developing plans to expand collaboration efforts.

Soundbites from the event:

IBM's cognitive computing chips make waves

Last week, IBM unveiled the first ever cognitive computing chips - designed to learn without set programming, the chips emulate the brain's ability for perception, action, reaction and sensation, and have demonstrated the ability to recognize handwritten numbers and provide its best answer for what it "sees" on the screen. The chips have also learned how to play the game Pong, in which it "watches" the ball, predicts the trajectory, and moves the paddle to block the ball with amazing accuracy.

You can read all about the SyNAPSE project at ibm.com/synapse and catch up on the 10 ten things to know about the SyNAPSE project on the IBM Research news blog. Selected press clips follow, as well as a video highlight reel.



New York Times: I.B.M. Announces Brainy Computer Chip
Associated Press: IBM Pursues Chips That Behave Like Brains
The Wall Street Journal: IBM Announces Move Toward ‘Cognitive’ Computing
Bloomberg Businessweek: IBM Chip ‘Senses’ Events to React Like Brain
AFP: IBM unveils computer chips that mimic human brain
NBC: IBM's New Chips Act Like Your Brain
BBC: IBM produces first 'brain chips'
CNET: IBM says new chip mimics the human brain
GigaOm: For our sensor heavy future, IBM cooks up a new silicon brain
MIT Tech Review: IBM's New Chips Compute More Like We Do
Gizmodo: IBM's Processors Will Think Like Brains
Engadget: IBM's cognitive computing chip functions like a human brain, heralds our demise
Venturebeat: IBM produces first working chips modeled on the human brain

"Fascinating? No doubt. It's initiatives like these that demonstrate just how different IBM is from Hewlett-Packard and other peers, especially now that CEO Leo Apotheker is cutting consumer-facing operations to focus on big-ticket sales. Reverse-engineering brainpower would have to seem like a frivolous enterprise."

- Motley Fool

IBM CEO visits IBM Research - Almaden

Researchers at Almaden were thrilled to attend a townhall event with IBM Chairman and CEO Sam Palmisano on Thursday, August 4. In his first speaking engagement (to employees) at Almaden, he gave an overview of IBM's first half, and the audience of over 350 enjoyed his casual and personable presentation style, complete with a few friendly jokes.




Following the townhall, Mr. Palmisano presented a Centennial Lecture at the Computer History Museum in Mountain View, CA. Analyst Rob Enderle, in an article about the event, said:

"I attended a truly brilliant speech last night. It is now one of those that I’ll likely take with me to the grave...[IBM CEO Sam Palmisano] spoke about what it takes to make it through 100 years and I’ll remember both what he said and, more importantly, what he did, which is give IBMers hope."

(Read the rest of Rob's article here: What IBM and Sam Palmisano Could Teach Apple, Microsoft and President Obama on itbusinessedge.com)

Addtional coverage:

Harvard Business Review
IBM at 100: How to Outlast Depression, War, and Competition

EE Times
IBM CEO: How to live to be 100

eWeek
IBM CEO Palmisano Reflects on Company's 100 Years

Forbes
IBM's Palmisano: How You Get To Be a 100 Year-Old Company

Watch Mr. Palmisano's lecture at the Computer History Museum: What Changes and What Endures below:

Intern Spotlight: Summertime at Almaden

The days are longer, the temperature is higher and the Thursday BBQ days in the cafeteria have commenced; IBM Research - Almaden is bustling with a fresh vibrance that can only mean one thing - it's summertime!

With most of the interns being from out of state and even out of the country, a few casual discussions over free afternoon popcorn and tea warranted some interesting perspectives on the lab. When asked what their favorite part about the lab was so far, almost everybody mentioned the location, the weather, the people and the wildlife. University of Denver student DiDi Wei saw cows on her first day and "thought it was really neat." Shripad Nadgowda from Napur, India added that his favorite part is the research that he's involved in during his time here; he chose IBM for his summer internship because "IBM is the biggest technology hub. IBM technologies are very well studied in the academia. I wanted to join IBM to get chance to learn these technologies from people who invented them."

The interns come from a wide variety of scientific and mathematical backgrounds: chemistry, applied mathematics, systems, ubiquitous computing, epidemiological modeling, and robotics and intelligent systems to name a few. So why IBM?

The words legendary, resume-builder, fun and famous came into play. University of Washington MBA candidate, Naureen Manekia replied, "Who wouldn't want to work for such a world famous company in beautiful California? As a Human Resources Intern, I was interested in learning best HR practices from a global, high performance organization and this internship provided me with this valuable opportunity. I will be able to more effectively build my network and expand my experiences with IBM this summer. I'm looking forward to the fun and challenges IBM has to offer!"

Jorge Guerra, a Caracas, Venezuela native, expressed that "as a PhD student, IBM Almaden offers an excellent opportunity to experience research from an industry perspective, which complements the academic training I get from school. Also, I found that IBM has a very diverse group of researchers from different backgrounds and this greatly helps enrich discussions and debate on product ideas."

The internship experience in general is designed to enrich and add value to University studies, future job searches, and life lessons. IBM proves its success - Douglas Willcocks from London lamented that this is "an excellent opportunity to gain experience at a world class research lab." Sureyya Tarkan also agreed on the notion that IBM is a world-class facility and wanted to know what it was like to work in a "large research group."

All in all, their experience at IBM is beneficial in that future employers will recognize that they've contributed to a professional environment and will be prepared to take on challenging tasks. Kim Weaver mentioned that she "always thought that professors should have some experience in industry to better relate to their students, most of whom desire industry and not academic jobs. This was a good opportunity to work with some great, well-respected people."

Finally, there were some discussions on the benefits and perks of being an IBMer, even if it is for a few short months out of the year. Some of the interns are taking advantage of Great America Theme Park tickets at a discount of $20 off regular ticket price; others are enjoying free public transportation all summer long, thanks to IBM's free eco-passes. Discounted movie tickets are popular, as are the on-site dry cleaning service and on-site car wash. Almaden's full service library also got kudos, due to their excellent assistance in providing books ordered specially from other IBM sites.

While Almaden's full-time residents are doing their part in making the lab a comfortable home-away-from-home for the interns, there's definitely mutual benefits - learning, networking, and having fun all come into play during this season, for all parties involved.

Summer Intern Showcase

The wide range of business applications and discoveries developed by the interns this year were largely ideas stemming from existing research projects or those that were yet to be executed by Almaden’s research teams. With the help, enthusiasm and refined expertise of this year’s interns, many of those ideas and projects were realized in just three short months, while others gained significant momentum toward fulfillment.

Many of the interns worked on projects that were vastly different from their current course of study at their universities. Shripad Nadgoda from Stony Brook University took on a GPFS project in the storage area that didn’t align with his coursework, but changed his frame of mind to a new type of research, which he indicated "will be helpful later because I’ve been able to explore a different area and start from scratch." Yang Ye of Colombia University echoed those sentiments stating, "It was nice to learn how to solve real problems in the industry. It was hard at first, but then your mind switches, and you take off in the right direction."

Others, however, were recruited based on their expertise in certain areas. Sunyoung Kim from Carnegie Mellon University authored/co-authored several published papers around air quality research. IBM Human Computer Interaction researcher Tom Zimmerman read her paper and recruited her to join the team of Almaden computer scientists working on an iPhone app that monitors water quality. In this mutually beneficial scenario, Sunyoung was able to build on her current research focus, and IBMers were able to tap into her knowledge about application development for mobile environmental monitoring.

Interns in the Extreme Blue Program at IBM Research - Almaden shared some thoughts on camera:

Services Innovation Lab strengthens Research

This post originally appeared on the IBM Research news blog on July 28.

Today, IBM announced an initiative designed to link state-of-the-art IBM Research projects directly to the largest part of the company - services.

The Services Innovation Lab (SIL), led by researchers with development and client experience in virtually every field of science and technology, will bring together researchers and services professionals to generate ideas, solve problems, capture opportunities and create new innovations that will transform services. You can learn more about the SIL here.

IBM Research - Almaden senior manager for services research, Stefan Nusser, has been named Almaden Lead for the SIL. Below, he provides some additional thoughts on the motivations, goals and ideal outcomes.



Stefan's team, composed of researchers with varying expertise - nanotechnology, computational biology, user interaction, computer science, data mining, security and compliance, mathematics, business optimization and more - includes Sandeep Gopisetty, IBM Research Distinguished Engineer, who already has a portfolio of projects in place for the launch of the SIL:

• Backup and recovery tools to help with the majority of critical situations in the customer environment.
• Provide insights into a customer environment to check the heal and performance, reducing critical situations as well as helping to manage the customer environment more simply.
  
• Addressing the management of firmware and patches for hardware.
• Intelligent performance aware automation for complex storage tasks.
• Unified policy-based ILM for enterprise storage.
  

GPFS breaks file scanning record by 37x

Editor's note: Guest author Richard Freitas is a computer scientist in storage class memory at IBM Research - Almaden.

In our increasingly instrumented, interconnected and intelligent world, individuals and companies are struggling to deal with the explosive growth of data. According to a recent study there will be 1800 exabytes (EB) of digital data in 2011, up from 220 EB in 2007. Such growth places the data user under tremendous pressure to turn data into actionable insights quickly, while straining the world’s IT infrastructure to its limits. This forces the data user to manage the explosive growth of data and storage using tools designed for easier times.

Existing storage management solutions find it difficult to provide timely storage backup, migration to appropriate performance tiers, replication and distribution. In many cases, users go without the kind of daily backup that industry experts would expect of a large data store. As new applications emerge in industries from financial services to healthcare, traditional systems will be unable to process data on this scale, leaving users exposed to critical data loss.

Anticipating these storage challenges decades ago, Almaden researchers created GPFS, a highly scalable, clustered parallel file system. Already deployed in enterprise environments with one billion files to perform essential tasks such as file backup and data archiving, this technology’s unique approach overcomes the key challenge in managing unprecedented large file systems with the combination of multi-system parallelization and fast access to file system metadata stored on a solid-state storage appliance.

GPFS advanced algorithms make possible the full use of all processor cores on all of these machines in all phases of the task (data read, sorting, and rules evaluation). GPFS exploits the excellent random performance and high data transfer rates of the 6.5 TB solid-state metadata storage. The solid-state appliances sustainably perform hundreds of millions of IO operations, while GPFS continuously identifies, selects and sorts the right set of files from the 10 billion-file file system. Performing this selection in 43 minutes was achieved by using GPFS running on a cluster of ten 8-core systems and four Violin Memory solid-state memory arrays. This is 37 times the rate that was achieved in 2007 on a file system containing 1 Billion files.


You can read more about this IBM Research breakthrough here.

Community: Help through horses for autism

When Carolyn Wallace, IBM Research - Almaden Client Relationship Manager learned about IBM's Centennial Day of Service, she was quickly motivated to rally around a particular cause that affects her personally: autism. Carolyn's 10-year old grandson, Max, has autism, and in the spirit of IBM's 100th year, she decided to gather the efforts of her colleagues and friends in the area to donate their time to Gilroy's DreamPower Horsemanship Camp.

The DreamPower Horsemanship Camp offers a number of therapeutic services to individuals affected by physical and mental disabilities, ranging from equine facilitated psychotherapy, groups for cancer patients and confident rider clinics to couples communication workshops and therapeutic horsemanship lessons for children, teens and adults.

"Max struggles with speech and has been in speech therapy for several years," Carolyn said. "I don't know if it's my imagination or wishful thinking, but I believe his speech is much clearer when he is riding."

Founded by Licensed Marriage and Family Therapist Martha McNiel, the program was developed with a specific mission in mind: Building lives filled with dreams, and the power to make dreams come true. Martha is a NARHA Registered Therapeutic Riding Instructor and a Certified Equine Interaction Professional in Mental Health. Along with her staff, she hosts groups at the camp ranging from juvenile delinquents to individuals with cerebral palsy - all with the intent to provide emotional and physical healing for their clients.

With an IBM cash grant of $5,000 used to support the week-long DreamPower Horsemanship Camp for Children on the Autism Spectrum, IBMers and DreamPower staff worked together to create 15 hours worth of horse riding, grooming, arts and crafts and science projects for the children to participate in. Ranging in age from 4 - 13, 16 camp participants from San Jose and Morgan Hill learned about things like horse safety and care, horseriding commands and coordination activities, and also played interactive games like giant Jenga. On the last day of camp, the group got to decorate a mini horse with paint and glitter.

"We have had Max in therapeutic horseback training since November of 2010," Carolyn said. "He is very enthusiastic about his riding classes on Monday evening - he gets ready and stands by the door when it gets close to time to depart for class."

In order to qualify for the grant, which was proposed as part of IBM's Centennial service commitment, the program had to incorporate a science and technology component. By enlisting the help of Schmahl Science - in essence, a science lab on wheels - the kids had an opportunity to explore science, math and engineering through projects tailored specifically to their needs.

Mayor Steve Tate, City of Morgan Hill, coincidentally a 38-year IBMer, supports the philanthropic efforts of IBM, and is proud to have the affiliation attached to his resume: "they're community builders. And that's what we're all about here in Morgan Hill," he said.

In fact, IBMers in the Silicon Valley have donated time directly to the South County in a number of different ways. Community teams have partnered with Solorsano Middle School in Gilroy to put on Lego Robotics Workshops and to invite students to IBM's Annual Technology Camps. Around 25 IBMers volunteer for Rebekah's Children's Services by cleaning up, painting, organizing or planting at various locations. Overall, IBM has donated $182,494 in cash and equipment to the City of Morgan Hill and $10,208 to the City of Gilroy since 2006.

IBM is committed to serving local communities - in fact, IBMers volunteer more than any other company in the world. Since 2003, more than 11 million hours of service has been recorded. In the Silicon Valley alone, IBMers are committed to donating 32,000 hours of service in 2011.



Read about IBM's involvement in the DreamPower Horsemanship Camp in the Morgan Hill Times.

Photo captions, top to bottom:

A camp participant straps on her helmet with the help of a volunteer in anticipation of a therapeutic ride through the arena.

IBM Fellow, CTO and VP, Technical Strategy and Senior Location Executive for the Silicon Valley, Jai Menon, visits the camp to support the nearly 100 IBM volunteers and to express his praise for a program dedicated to an extremely relevant cause.

Middle school girls jump start passion for math and science

Last week, 28 girls gathered at IBM Research - Almaden, some having no idea how interested they were in math and science until they built a PC, had fun with liquid nitrogen, and checked out a homemade robot by local high schoolers that can play soccer. IBM rockstar volunteer, Rick DiPietro, was able to win over a handful more with his "cool chemistry" session where they extracted the DNA from a strawberry. The homemade slime they created was too cool to forget about, and seeing a cotton ball explode was also something many of the girls talked about after the event.

"I learned a whole new side of me that I never knew I had," exclaimed 8th grader Lauren Miller as she tested circuit boards and made her LED light shine. Her fellow camp friend Alice Chu was also surprised at "how fun science can be."

Led by IBM Outreach Program Manager, Bob Martinez, and Corporate Citizenship and Community Affairs Manager Jennifer Hernandez, 42 IBMers volunteered their time throughout the week to help the girls navigate through several science projects. While at Almaden, the group also got to learn about the A/V field with leader Alex DeLuca, how to give an effective business presentation, and a tour of the site which included visits to the computer rooms, telepresence videoconferencing rooms, and the lobby's latest addition, the OmniGlobe. In the middle of the week, the girls were treated to a field trip to San Francisco's California Academy of Science, where the living roof and the 4 story rainforest were favorite exhibits.

Bob Martinez, who has been running this program with the help of his core team for the past 6 years, credits his IBM colleagues for the time they spend volunteering, not only during camp, but throughout the year. Because of the active engagement in youth outreach by IBM employees in the Silicon Valley, Bob and team have developed partnerships with several schools in the area - most with a high concentration of minority students. Additional programs such as Traducelo Ahora and MentorPlace, help give young kids exposure to math and science, which according to experts, is crucial in helping develop the next generation of high-tech professionals.

"The kids get exposure to science and technology during the camp, but they also learn other things that help them prepare for their future," says Martinez. "Creating their project presentations using a manuscript versus conversational method, different teambuilding exercises, and being able to talk intelligently about what they've learned are all things they can take away from this camp and help them excel no matter what career they choose. But when you see the spark in their eye, that's when you know we've done what we set out to do."

A look inside the girl's technology camp from last year:



Picture Captions (Top to Bottom):

The girls participate in cool chemistry with liquid nitrogen.

A MedEvac helicopter, which landed right on-site, gave the girls a chance to learn about air ambulance services, go inside the helicopter, and experience the medical, technical and communication devices used in transport.

A popular project with the girls is soldering wires together to create their own individual LED light box.

The 6-foot OmniGlobe in the building's lobby shows girls the different areas of technology that computer scientists are working on, like 3-D visualization.