World’s Data More than Doubling Every Two Years – Driving Big Data Opportunity, New IT Roles

New IDC Digital Universe Study Sponsored by EMC Estimates Data Growing Faster Than Moore's Law to 1.8 Zettabytes of Data in 2011;
Enterprises to Manage 50X More Data and Files to Grow 75X in Next Decade

News Summary:

• New IDC Digital Universe study, “Extracting Value from Chaos” 1 (sponsored by EMC) finds world’s data is doubling every two years — growing faster than Moore’s Law.
• Study forecasts 1.8 zettabytes (1.8 trillion gigabytes) will be created and replicated in 2011.
• The Digital Universe - and “Big Data” - are driving transformational societal, technological, scientific, and economic changes.
• “Information taming” technologies are driving down the cost of creating, capturing, managing and storing information—one-sixth the cost in 2011 versus 2005.
• Enterprise invest in the digital universe has increased 50%, to $4 trillion (USD), since 2005.

Full Story:

Auckland, New Zealand – June 29, 2011 – EMC Corporation today announced results of the EMC-sponsored IDC Digital Universe study, “Extracting Value from Chaos” – which found the world’s information is more than doubling every two years – with a colossal 1.8 zettabytes to be created and replicated in 2011, which is growing faster than Moore's Law.

The study’s fifth anniversary, measuring and forecasting the amount of digital information created and copied annually—analysing the implications for individuals, enterprises, and IT professionals – has huge economical, social and technology implications for big data and other opportunities.

In terms of sheer volume, 1.8 zettabytes of data is equivalent to:

• If each person in New Zealand was able to assist in downloading all the data that was created in the 2011 DU, then it would take 8,787 days to download.
• If each person in New Zealand tweeted 3 tweets per minute, then they would have to tweet 1,909,490 years solid to equal the amount of information created in the 2011 DU
• Every person in the world having over 215 million high-resolution MRI scans per day
• Over 200 billion HD movies (each 2 hours in length) would take 1 person 47 million years to watch every movie 24x7

The forces behind this relentless growth are driven by technology and money. New “information taming” technologies are driving the cost of creating, capturing, managing and storing information down to one-sixth of what it was in 2005. Additionally, since 2005 annual enterprise investments in the Digital Universe—cloud, hardware, software, services, and staff to create, manage, store and generate revenue from the information—have increased 50% to $4 trillion (USD).

Study Highlights:

• Massive Server, Data Management and File Growth Not Keeping Pace with Staffing: IDC notes that the skills, experience, and resources to manage the deluge of data and resources simply isn’t keeping pace with all areas of growth. Over the next decade (by 2020), IT departments worldwide will experience:
- 10X the number of servers (virtual and physical).
- 50X the amount of information to be managed.
- 75X the number of files or containers that encapsulate the information in the digital universe, which is growing even faster than the information itself as more and more embedded systems, such as sensors in clothing, in bridges, or medical devices.
- 1.5X the number of IT professionals available to manage it all.

• Cloud Computing Cost and Operational Efficiency: While cloud computing accounts for less than 2% of IT spending today, IDC estimates that by 2015 nearly 20% of the information will be "touched" by cloud computing service providers — meaning that somewhere in a byte's journey from originator to disposal it will be stored or processed in a cloud. Perhaps as much as 10% will be maintained in a cloud.

• The Digital Shadow Has a Mind of Its Own: The amount of information individuals create themselves—writing documents, taking pictures, downloading music, etc.—is far less than the amount of information being created about them in the digital universe.

• The Liability and Responsibility is with Enterprises: While 75% of the information in the digital universe is generated by individuals, enterprises have some liability for 80% of information in the digital universe at some point in its digital life.

“The chaotic volume of information that continues growing relentlessly presents an endless amount of opportunity—driving transformational societal, technological, scientific, and economic changes,” said Jeremy Burton, Chief Marketing Officer, EMC Corporation. “Big Data is forcing change in the way businesses manage and extract value from their most important asset – information. EMC is at an ideal crossroad to help our customers—from the world’s largest enterprises to governments to small businesses—exploit the hidden value in the digital universe as they continue on their journey to the cloud.”

Other Key Findings

• New capture, search, discovery, and analysis tools can help organisations gain insights from their unstructured data, which accounts for more than 90% of the digital universe. These tools can create data about data automatically, much like facial recognition routines that help tag Facebook photos. Data about data, or metadata, is growing twice as fast as the digital universe as a whole.
• Business intelligence tools increasingly are dealing with real-time data, whether it’s charging auto insurance premiums based on where people drive, routing power through the intelligent grid, or changing marketing messages on the fly based on social networking responses.
• New storage management tools are available to cut the costs of the part of the digital universe we store, such as de-duplication, auto-tiering and virtualization, as well as to help us decide what exactly to store, as in content management solutions.
• New security practices and tools can help enterprises identify the information that needs to be secured and at what level of security, and then do so from specific threat protection devices and software to fraud management systems and reputation protection services.
• Cloud computing solutions – both public and private and a combination of the two known as hybrid – provide enterprises with new levels of economies of scale, agility, and flexibility, compared to traditional IT environments. Long term this will be a key tool for dealing with the complexity of the digital universe.
• Cloud computing is enabling the consumption of IT-as-a-Service. Couple that with the Big Data phenomenon, and organisations increasingly will be motivated to consume IT as an external service vs. internal infrastructure investments.
• The growth of the digital universe continues to outpace the growth of storage capacity. But keep in mind that a gigabyte of stored content can generate a petabyte or more of transient data that we typically don’t store (e.g., digital TV signals we watch but don't record, voice calls that are made digital in the network backbone for the duration of a call).
• Less than a third of the information in the digital universe can be said to have at least minimal security or protection; only about half the information that should be protected is protected.

About EMC

EMC Corporation is a global leader in enabling businesses and service providers to transform their operations and deliver IT as a service. Fundamental to this transformation is cloud computing. Through innovative products and services, EMC accelerates the journey to cloud computing, helping IT departments to store, manage, protect and analyze their most valuable asset – information – in a more agile, trusted and cost-efficient way. Additional information about EMC can be found at www.EMC.com.

###

1 IDC Digital Universe Study, sponsored by EMC, June 2011

EMC is a registered trademark of EMC Corporation. Other trademarks are the property of their respective owners.

ENDS

BUDGET 2012:
Parliament Debate Live - Video Of Budget 2011
Keith Ng Interactive Graphic: How the Budget Breaks Down
BUDGET 2012 - FULL COVERAGE: Reports / Analysis - Press Kit - Reaction (from everybody) - Previews (from everybody) - Pre-Budget Announcements

Gordon Campbell: On the Budget’s Spreadsheet Victories

It wasn’t as if expectations were sky high, exactly. Chances are, it was always more likely that we’d be seeing Bigfoot rampage through the Beehive lock-up than catch a glimpse of a credible growth agenda from this government. More >>

Sludge Budget Report - Short The Dollar! MEMO: To international bankers FROM: C.D. Sludge Please short the dollar! It'll be good for both you and us. And you know you want to. Greexit, Eurogeddon... watch out... flight to quality and all that. Follow your instincts. The NZ Debt Management Office has been so surprised at the unprecedentedly low interest rates that it can borrow at that it has already entirely pre-funded the 2013 fiscal deficit - all $8 billion of it! More >>

Pattrick Smellie Comment: Doddling along the best we can hope forCriticising Budgets for lacking vision or imagination is like shooting fish in a barrel, but even so, this year's Budget again feels like a missed opportunity. Perhaps it's the intrusion of real world needs that means the government couldn't make better political use of the $558.8 million it expects to gather in its first partial asset sale. More >>