Storagezilla

NetWorker 8.0: To the stars.

After writing more than I expected (and more than I should have) we come to my thoughts on the next decade of NetWorker. Beyond 8.0 and into 8.x you can expect that EMC will play to it’s strengths when it comes to the on going development of NetWorker.

-NetWorker will do more with backup to disk.

-NetWorker will do more with data deduplication.

-NetWorker will do more with virtualization.

And it will do this while maintaining a modern, scalable architecture with richly functional application level integration delivered in a timely manner with the highest possible level of code quality. The proof of the pudding of course is in the eating so if you’re running NetWorker in production today get the 8.0 software as soon as you can, install it somewhere and hammer on it to see what it can do you for.

Some people will upgrade to 8.0 immediately for the new functionality, others will wait until service pack one. Regardless of when you plan on upgrading start looking at 8.0 right now.

While NW 7.6 is one of the best releases of NetWorker ever shipped (I liked it the way I liked 7.2) the architecture of the 7.x code has now reached it’s logical conclusion. While it’s not over for NetWorker production deployments yet as there are various service packs and maintenance fixes still to be delivered, future NetWorker development is now singularly focused on 8.0 and beyond.

There’s more coming.

Be ready.

NetWorker 8.0: Rocket boost.

Since the name of the division is Backup Recovery Systems NetWorker 8.0 does have net new integration points with other products in the BRS portfolio.

In the case of new NetWorker 8.0 and Avamar integration, the nsrconsolidate functionality which handles the generation of synthetic full backups from full, differential and incremental NetWorker backups can also be used with Avamar to create rehydrated backups out of an Avamar system at a remote location which can be written to disk, virtual tape or physical tape for long term retention in a format which can be restored without the need for an Avamar system.

There’s already a lot of NetWorker/Avamar integration today, a bit more now even more later but the keystone integration is deeper Data Domain integration with NetWorker.

As a quick refresher lets cover Data Domain Boost for a moment. DD Boost allows for deduplicated backups to be written to Data Domain logical storage devices at levels of throughput which can far exceed using an Advanced File Type Device on SMB/NFS or emulated tape in a Virtual Tape Library. (VTL)

It does this by moving some, but not all, of the functions in the deduplication process as close to the data to be backed up as possible. Those functions being the creation of variable length segments, the fingerprinting of those segments and the compression operation for the transmission of those segments to the Data Domain system.

The advantages in throughput are cumulative as you eliminate needless data transmission while also ensuring only unique segments are written to Data Domain storage. Since network transmission tends to be the slowest hop in the backup process this can greatly reduce the amount of time spent waiting for backups to complete.   

While DD Boost has been integrated with NetWorker Storage Nodes and NetWorker Application Modules in previous versions of NetWorker, in NetWorker 8.0 it’s now available for use directly with NetWorker File System Clients. The Boost Distributed Segment Processing (DSP) module, which performs the segmenting, fingerprinting and compression operations now being in the core of the NetWorker client.

To DSP we’ve coupled NetWorker Client Direct, together these provide point to point deduplicated backups and restores from Data Domain storage systems, without a NetWorker Storage Node being anywhere in the data transmission process.

I covered this somewhat in a previous entry but it’s worth revisiting. You can perform Client Direct backups to Advanced File Type Devices for non-deduplicated backups, but when writing to Boost enabled logical storage units on a Data Domain system you not only gain deduplication functionality but also the ability to control deduplicated replication of those backups in a catalogue consistent way which allows for the recovery of data from the replica(s) if required.

Deduplicated high speed point to point backups from backup clients directly to Data Domain protection storage.

Possibly the most request feature from NetWorker customers since we first shipped Boost. Available now.

But we’re still not done.

The use of Boost has  also been added to the latest version of the NetWorker Module for Microsoft Applications (NMM) providing deduplicated point to point backups for Microsoft SQL Server 2012 and Microsoft Hyper-V. These join the existing support for Exchange and SharePoint while expanding NMM’s coverage into Hyper-V R2 environments with granular level recovery of files from Hyper-V virtual machines.

Boost early. Boost often.

NetWorker 8.0: Dawn of a new decade.

Fundamentals. A NetWorker Datazone is a single NetWorker server, the storage nodes managed by that server and all the clients that server is protecting. The NetWorker 8.0 architecture allows for about a 3x scalability improvement in a NetWorker Datazone over previous releases of NetWorker.

Lets dig into some of how it does that. 

In 8.0 there have been significant changes, changes that skew large and small. Ranging from things such as modifying the fixed block read process of the NetWorker Client software so it now operates on a variable length block algorithm (Which can increase overall disk read performance by requiring less read operations, any resemblance to Data Domain BOOST is purely intentional), to scaling the number active jobs & devices which can persist inside a NetWorker Datazone at any one time.

This is the dawn of the next decade of NetWorker.

Out of a number of new design points I could choose from I'm going to focus specifically on jobs and devices for a moment as those are where NetWorker administrators are going to see the new architecture in play.

Back in 2005, NetWorker 7.3 introduced an in-memory database (jobsdb) to track all active backup, restore, staging and cloning jobs executing in a NetWorker Datazone. Of the locations where NetWorker stores it's data, behind the backup catalogue itself, this was probably the most important one.

As time has passed, backup & restore windows have shrunk, but the numbers of clients and devices have increased, the resulting impact on the jobs database was that it required greater and greater amounts of server resources to operate effectively.

In NetWorker 8 the jobs database has been redesigned as a high performance transactional database which will not only scale significantly further than it's predecessor but do so using anywhere up to 80% fewer systems resources as it does so.

With the jobs database now redesigned to handle an increased number of concurrent operations we can now look at scaling the number of devices in a NetWorker Datazone. In 8.0 device management operations are now distributed amongst all the Storage Nodes in the Datazone. If you look at the processes running on a NetWorker 8.0 Storage Node you'll find a new nsrsnmd daemon. It's the visible symbol of the new distributed device management functions.

Distributed device management again reduces the amount of system resources consumed on the server by device management operations and when coupled with the new jobs database and the disk as disk changes laid out in Beyond Tape these improvements increase overall NetWorker scalability significantly.

But we're not done with architecture changes yet as all of those changes feed into NetWorker's new multi-tenant design approach, seen most clearly in the concept of NetWorker Restricted Datazones.

A Restricted Datazone is a tenant in a multi-tenant NetWorker Datazone deployment and enforces logical tenant separation of that tenant from all other tenants. That being a separation of access, a separation of the ability to view or start backups & recoveries and a separation of device management.

All of this is coupled to new audit logging functionality built into the 8.0 architecture and surrounded by new user roles to facilitate Role Based Access Control to a greater level of granularity and you have an architecture which allows existing NetWorker administrators to not only scale Datazones but consolidate a number of existing Datazones if they so choose.

There's more. There's a lot more on the new architecture but we can dive into that at a later date.

NetWorker 8.0: A world beyond tape.

For more than 20 years NetWorker was tape right to the bone. But no longer.

Let me explain.

Tape in the 90’s pretty much was the entire secondary storage market. Reading data from disk and packaging that data for tape was NetWorker's very first media handling function and as such it was tape’s performance and access characteristics which defined how NetWorker would write data to storage for the years which followed.

When NetWorker looked beyond tape to what was then an up and coming long term retention format, Optical media, NetWorker treated it exactly as if it were a tape. This was driven by a desire for a consistent if somewhat device independent approach to media management. Around 1996 disk as a backup target emerged blinking into the daylight and NetWorker embraced this new storage target with a File type device…which behaved exactly as if it were a tape.

But quickly you start hitting the specific limitations of tape as a medium. Tape didn’t support concurrent read/write access, tape didn’t groom space if a backup failed or was aborted, tape didn’t, didn’t, didn’t, etc, ect.

Having moved into a new decade and with a view of still keeping management consistency in 2003 with the release of NetWorker 7 came the Advanced File Type Device (AFTD) which did support concurrent read/write operations, did groom space from failed operations and whatnot. But NetWorker did this by pretending each AFTD behaved kind of like tape and a half.

That's a dramatic over simplification of the technology involved but it is accurate that the entire device model paradigm was driven entirely by a view of the world that after it was all said and done, your backup data would end up on a tape cartridge.

That is no longer the paradigm.

In NetWorker 8.0 the storage layer now treats disk as disk while keeping the required operational consistency. For any existing NetWorker users one of the things you’ll first notice in 8.0 is the AFTD Read Only device (AFTD.RO) is gone. It's gone because it's no longer required and the media storage layer has been entirely decoupled from the previous tape-centric design to one where multiple instances of nsrmmd (NetWorker media multiplexor daemon) are now capable of working with devices concurrently.

In 8.0 when using disk for backup, the same disk volume can be now mounted to multiple different AFTDs. This allows NetWorker to load balance backup & restore operations across multiple storage nodes and multiple device paths taking into account how busy devices and storage nodes are at the the time the operation request occurs.

This reduces overall device resource contention across NetWorker deployments, can increase backup and recovery success rates as it can provide multipathing to storage which increases overall system availability and does so while offering a migration path for all of the AFTDs deployed in production today.

Two other major pieces of functionality gained from the new disk as disk approach come in the shape of synthetic fulls, where a NetWorker storage node will aggregate a full backup and a series of incremental backups to generate a restorable full backup and Client Direct.

Before we look at today with Client Direct we should first look back to how things were done.

The NetWorker three tier model was introduced in NetWorker 5.0 and designed to scale the overall throughput of the backup infrastructure and the number of clients and devices a NetWorker Server could manage. At it's heart, NetWorker Clients transmit backups to NetWorker Storage Nodes which in turn perform read & write operations to media storage devices, all while the NetWorker Server manages Client & Storage Node operations, the schedule and the catalogue. Clients, Storage Nodes & Server are the three tiers.

With the ever shortening backup & recovery windows, much greater numbers of clients and expanding numbers of devices required to back up those clients, it was possible the system resources of Storage Node hardware itself could become a gating factor in performance. And to this comes Client Direct.

As the name implies this allows a backup client to directly write to and recover from disk based storage devices without a Storage Node being in the data path at all. Data no longer passes through the storage node, it just goes directly to the storage target. Be that target Data Domain with BOOST, SAN or NAS.

Client Direct can greatly simplify deployments, which has the knock on effect of increasing availability due to fewer things to go wrong, while reducing overall cost as Storage Nodes move from pushing the data to storage to just monitoring device availability and forwarding any metadata updates to the server. Bottom line, with Client Direct you require fewer Storage Nodes.

Disk as disk. Synthetic Fulls. Client Direct.

These are the key elements of the NetWorker 8.0 move to a world without tape. In NetWorker 8.0 by treating disk as disk it has unlocked greater functionality inside the software.

Functionality which was simply not possible with NetWorker's previous tape focused paradigm.