As a clear result, cloud storage will need to be designed on one key principle – it needs to shift to a decentralized model. So what does it mean to decentralize storage?

When thinking of decentralization, the Internet is the first place one should look to since it was designed to be fault tolerant. Storage will need to learn some lessons from the network and behave similarly in order to be successful. TCP/IP decentralizes packet routing enables packets to be routed on different paths through the network, which is beneficial in the long-run.

To borrow from the network and to prevent bottlenecks of scale, the data itself will need to take on a “packetization” architecture. This means that besides using packets to traverse the network, the data itself should be packetized and stored as packets.

Rather than saving data to centralized servers in a single data center that’s at risk for local outages, data will need to be virtualized into packets and stored across multiple servers in multiple data centers. When data is requested back, the system is smart enough to gather enough packets to reconstruct the data. Zero outage in the US Eastern Region data center? No problem.

But wait – you must be screaming about the latency… First off, networks are still increasing in speed so by the time the cloud is truly adopted by the masses, the network isn’t going to be the bottleneck. Second, for the early adopters out there, you could configure such as system to have enough packets at local data centers to address latency concerns while still having some packets spread out. That way, if the local data center is down, you can still access data seamlessly.

Once we can all embrace a decentralized storage architecture, a whole slew of design requirements come into play. Such as, how does the storage system optimize reading and writing of the packets knowing that all may not be available? How can such a decentralized system be used for content distribution to the masses?

This naturally leads to Cleversafe, where we are already packetizing data into something we call slices and already thinking of how a decentralized world of storage will work.

So how could the Amazon outage be avoided and what will the cloud storage model need to evolve to in order to be stable? Move to a decentralized platform.

Cloud services that are succeeding have added value beyond cost per GB. Iron Mountain is still running their File System Archiving (FSA) business which offers policies, indexing and classification.

It’s time for service providers to focus on adding value with data security. It would go a long way to aiding public cloud adoption.

So what does it take to achieve data security?

If service providers couldn’t actually access the data, would that make customers more comfortable with adopting public cloud services?

Suppose customers were in control of virtualizing their data to an unrecognizable format before sending it to the public cloud. And only they had control to put their data back together again.

In this view of the future, service providers would stopped asking customers to trust them with their data, and instead positioned themselves as operators who can deliver on SLAs.

And BTW – such a solution is possible with Cleversafe. Today.

This is my fifth year to attend Storage Networking World (SNW), and each year I continue to expect new and interesting developments in storage; once again I was not disappointed.  The theme of Cloud Storage and its role in the enterprise persisted at this year’s conference, along with Solid State Drives, virtualization and the role of replication/deduplication for data protection. However, of particular interest was the elevated discussion around security in the cloud and how encryption and other security methods are taking front row prominence when discussing Cloud Storage.

I had the pleasure of talking with various industry analysts including Robin Harris of Storage Mojo (one of my favorite storage sites by the way) and David Floyer of Wikibon. Of interesting note is that both commented on the lack of any real “noteworthy” announcements being made at this year’s SNW spring conference and no high-profile vendor announcing any breakthrough technology or product offering. Contrast this with the fall 2009 SNW conference in which Xiotech announced the launch of their Emprise product, complete with magician that handed out dollars like they were candy and about 30 people manning their booth.  And while this seemed overly extravagant at the time, looking back it was highly entertaining and conveyed a sense of excitement and growth within the storage industry.

Additionally, this year I began a stint as co-chair of a new Storage Networking Industry Association (SNIA) group titled “Cloud Storage Initiative Cloud Archive and Long Term Preservation.”  Our focus is on Cloud Storage Archiving and the promotion of archiving and preservation within the cloud to take advantage of the cloud’s cost efficiencies and projected high availability benefits.  As a participant in the SNW Cloud Pavilion, we had the opportunity to participate in the inaugural Cloud Storage Theatre Presentation which allowed participants 5 minutes to describe their company and a topic around cloud storage. I took this opportunity to expand upon the growing challenges and costs associated with ensuring high-availability for petabyte scale storage systems; it’s an area where Cleversafe excels as a result of its information dispersal algorithm technology, which eliminates the need for RAID and replication.  I was actually surprised at how well attended the Theatre presentation was, with about 50 people staying to watch all of the Cloud Pavilion participants.  I am already anxiously awaiting SNW’s fall show with the expectation that Cleversafe will have an even stronger presence.

I personally received a notice from Chase, and a colleagues forwarded breach notices from Target, Verizon, and World Financial Network National Bank, which owns Ann Taylor, Victoria’s Secret and Best Buy store cards.

Epsilon has said around 2% of it’s 2500 customers were affected – it’s highly likely that you are on one of these folks’ lists and received a notification too. Here’s a partial list based on the companies who have reached out to their customers:

Ameriprise Financial, Barclays Bank of Delaware, Bebe, Best Buy, Brookstone, Capital One Bank, Citi, City Market, The College Board, Dillons, Disney Destinations, Eddie Bauer, Ethan Allen, Food 4 Less, Fred Meyer, Fry’s, Hilton Hotels, Home Shopping Network, JPMorgan Chase, King Soopers, Kroger, Lacoste, LL Bean Visa Card, British retailer Marks and Spencer, Marriott Rewards, McKinsey & Co., Moneygram, New York & Company, Ralphs, Red Roof Inns, Ritz-Carlton Rewards, Target, TD Ameritrade, TiVo, U.S. Bank, Verizon and Walgreens.

Epsilon is trying to downplay the breach by saying it is just emails addresses, but there are many non-tech-savvy users out there who will fall victim to email campaigns from people attempting to appear like legitimate businesses to gain further personal information from users.

The Epsilon catastrophe points to a need for better information security for Data at Rest. Storing data as actual data leaves it vulnerable for security breaches. Although Cleversafe hasn’t optimized information dispersal for transactional systems, it is only a matter of time until the fundamentals of dispersal where information isn’t stored as actual data but as slices or fragments that are not centralized, will become a dominate architecture for protecting data. Once information is split and decentralized, it decreases breach exposure since the cost of breaching the data is high – hackers would need to break into multiple decentralized systems versus a single centralized system.

I remember going with my parents to our friend’s house to watch the Superbowl in 1984, and how everyone was absolutely glued to Apple’s “1984” commercial. After it aired, there was much discussion about this truly amazing commercial. And back in 1984, no one had the internet to re-watch it, tweet about it, blog post about it, etc. What this really shows is just how much our society has changed in interacting with media – it’s no longer a one way, passive experience. Today, people interact and comment alongside in real time – just look at Twitter streams from this year’s Super Bowl.

The beauty of MBC today is that allows us to re-watch those historic moments – a mash up of the history of linear broadcast brought to you by today’s non-linear reality. And today’s non-linear reality is changing the fundamental way information is created, shared, and stored.

Hope you enjoy it as much as I do.

During the presentation, I presented a paper titled “AONT-RS: Blending Security and Performance in Dispersed Storage Systems” which I co-authored with Prof. James Plank at the University of Tennessee.  It is on the topic of a particular algorithm developed at Cleversafe, and how it compares to other related algorithms for the secure and reliable storage of data.

The paper is available for download from the Usenix website here along with the presentation slides.  I suggest checking out some of the other papers and  presentations as they were definitely interesting.  I look forward to attending next year’s conference.

My main takeaway from this event is that mobile access to healthcare information–whether it is electronic health records (EHR), electronic medical record (EMR) or picture archiving and communication system (PACS) also known as digital imaging preservation–is not a nice-to-have feature anymore, but rather a requirement that is being reinforced by the Federal Government’s mandate to convert all patient information into digital records by 2014 as part of the American Recovery and Reinvestment Act of 2009.  There were a plethora of vendors that were describing mobile solutions that would allow either a patient or a doctor–or both–to access their health records from a variety of mobile devices such as a smart phone, tablet or laptop computer.  However, when questioned about where the data would be stored in support of this mobile access, they did not have any really good answers.  And when pressed further, they stated that their biggest concern lies around security and the ability to ensure that a patient’s personal information isn’t compromised if they attempt to access it over the network.

This got me really excited because Cleversafe has solved the problem of how to store data securely in the cloud (whether it is a private, public or hybrid cloud storage model) with our Secureslice(R) technology which includes the ability to encrypt the data using the latest 256-bit AES encryption.  But whereas, other solutions exist today that use encryption; with Cleversafe, a customer does not have to concern themselves with key management which is one of the biggest hurdles facing companies who want to encrypt, but don’t want to hassle with the keys.  By using a similar approach to how we “slice up” the data, we also do the same to the encryption/decryption keys and therefore, a customer (patient/doctor) simply needs to have access to a threshold number of slices to allow us to recompile the key and get full (and secure) access to their data.

Cleversafe’s solution is perfect for healthcare’s expanding requirements around secure and immutable storage and I look forward to next year’s HIMSS conference with the expectation that we will evolve from a bystander to active participant.

At the ExecEvent storage conference in Palo Alto last week, I was asked to give a presentation to storage industry analysts.  The Cleversafe team helped create a presentation for the ExecEvent on the following hypothesis:

The unique capabilities of Information Dispersal can be used to solve a fundamental problem in computer science:  the creation of a fully distributed multi-writer file system.

It is an exciting idea.  Cleversafe has already demonstrated that data can be dispersed across many locations.  Dispersal has the unique ability to provide the reliability of multiple copy storage with just a single, dispersed instance of that data.

If that technology could now be applied to disperse metadata in addition to just data, then it could be possible to create a fully distributed multi-writer file system.  One of the key challenges in creating distributed, multi-writer file systems is maintaining concurrency across multiple copies of metadata.  This is particularly difficult with metadata attributes since they change quickly and frequently.  In some cases, it is not possible to guarantee consistency for these attributes across multiple metadata copies in multiple locations.  For example, if you had 10 writers each in a different location across a continent writing to the same directory file that is copied in those 10 locations, how to you guarantee consistency for all of those writers?

Dispersal can solve this fundamental problem by providing the reliability of copies in multiple locations with a single instance of data or metadata dispersed in such a way that can tolerate multiple location failures.  As a result in a dispersed system, there is only a single point in time when a meta-data attribute changes from one state to another (vs. a multiple copy system where each copy of metadata can change states at different times).

Attending this presentation were some key storage analysts, including Howard Marks of Network Computing, George Crump of Storage Switzerland and Marc Staimer of Dragon Slayer Consulting.  It wasn’t surprising that we had a good discussion around this topic with that group.   This is a big idea and I’m looking forward to exploring this idea further.

Chris Gladwin

In comparing some of the storage forecasts, one of particular interest was made by Tom Coughlin of Coughlin and Associates–a storage analyst firm–who believes that within the next decade our storage requirements will grow to:

• 1 TB in your pocket
• 1 PB in your home
• 1 EB in data centers
• 1 ZB in the solar system (we will have begun our habitation outside of earth)

I had the pleasure of presenting as part of a panel on Opportunities and Challenges for Consumer and Enterprise Cloud Storage.  My panel consisted of the following industry leaders:

• Ingo Fuchs, NetApp
• Tracey Doyle, Hitachi Data Systems
• Chris Hamlin, Blackridge Technology
• David Snead, W. David Snead P.C.
• Ben Woo, IDC
• Mike Alvarado, Consultant
• Tom Mulally, Numagic Consulting

The premise of our discussion was to evaluate how the explosive growth of media and other “unstructured” data storage requirements is forcing companies to consider alternative and more cost-effective solutions, such as private and public clouds and erasure codes/information dispersal in place of traditional replication.  While my fellow panelists addressed the security and SLA concerns around cloud storage, my focus was more on the challenges RAID presents with the ever-increasing size of individual hard drives and the propensity for bit failures that surpass RAID’s capabilities.

Some questions that were posed to the audience were around whether Cloud storage was a feasible business solution and also what are the legal implications around the current state of non-regulation.  Interestingly, there was a fairly strong disagreement between two of the panelists regarding Cloud regulation—Ben Woo an analyst at IDC and David Snead an attorney specializing in internet infrastructure .  Ben was arguing in favor of more regulation to help enforce the accountability of the service providers who today get away with no liability (customers are completely exposed to the possibility of data loss); while ironically, David argued that regulation was not required, but rather it should be the responsibility of the customer to negotiate and document in their SLA contracts to ensure that their service provider protects their data and guarantees its availability.  Surprisingly, it was an analyst promoting regulation and a lawyer promoting no regulation…completely altering my appreciation of the legal profession.  Instead, the advice from the lawyer regarding cloud storage was to make sure to address the following in any contact that a customer engages in with a service provider:

• Determine how services will be used
• Evaluate cloud structure(s)
• Understand data collection, processing and transfer policies
• Define security breach notifications
• Identify any high-risk regulatory areas
• Determine data disposition upon termination
• What data will remain in-house vs. outsourced to the cloud
• What access rights must be granted
• What are the current personal use policies
• Determine whether the data will be commingled
• Create an IP review policy

It was evident from the reaction of the audience that they were able to grasp the concept of using dispersal algorithms in place of traditional RAID to protect massive amounts of storage against data corruption and ensure continuous availability.  However, I was a bit surprised when I asked a polling question around how many people were using RAID 6 today—to only see a few hands raise.  This leads me to believe that companies are either not aware of the potential problems that can overcome their SAN if they are using large capacity hard drives, or they are aware, but believe that RAID 6 is sufficient and have yet to experience a major outage.  To this end, I was asked as part of the panel to be one who challenged the status quo in that traditional data protection schemes may not suffice for the latest data proliferation. I posed the following: “IDA = A Better Way”.  This allows storage administrators to focus their efforts on supporting business functions rather than worrying about whether their data is available and protected.  Since there doesn’t appear to be an end in sight for hard drive capacities, it is Cleversafe’s expectation that more and more companies will realize and adopt the use of information dispersal algorithms as the definitive replacement for RAID.