Is primary storage deduplication going entirely mainstream?

Storage vendors are very passionate about the topic of storage deduplication. Vendors who offer storage deduplication, tout it, while vendors who don’t offer it, discredit it.

Before I go too far, let’s agree on what we are talking about so we can disagree later: When I say primary storage, I’m referring to non-backup storage, so these are the logical unit numbers (LUNs) that are providing storage resource for file servers, databases, and virtual machines. When I say deduplication, I’m referring to a technology (blocks, pages, or files) that removes the storage consumption of like areas on disk.

Many solutions offer deduplication on storage other than primary storage; this is frequently on backup tiers or with solutions such as a virtual tape library (VTL). Software solutions can also add deduplication within a file, but are not represented on the storage area network (SAN) as a feature of the storage system.

NetApp is the one mainstream storage vendor that has been providing primary storage deduplication for some time. I am convinced that the company will soon have some competition in the space on the heels of Dell’s acquisition of Ocarina Networks. Ocarina is an interesting solution that provides a combination solution of compression and deduplication. In fact, I had a chance to visit with Ocarina last year, and I was impressed by what I saw.

The question becomes: Is primary deduplication that big of a deal? My answer is yes. NetApp even has a virtualization guarantee program that assures 50% less storage usage for virtualization implementations. Where does Dell’s acquisition of Ocarina come into play? I believe it fits quite nicely when we consider Dell’s acquisition of EqualLogic in 2007.

What does this mean for the consumer? Simply speaking, if more solutions are available in primary storage deduplication from storage vendors we’ll see the competition adjust. Does this ensure that other storage vendors will rush into a primary storage deduplication offering? Not necessarily, but among mainstream vendors there will be competition for the feature.

Do you feel that primary storage deduplication is a necessity today to get the most out of your storage dollar? Share your comments below.