The what, where, why, how and when of data migration

by

Adrian Kingsford, CTO, Viadex

Forward momentum dictates caution. Data is a precious asset; what you do with it is also the subject of scrutiny by regulators. The essential principle is ‘handle with care’.

A moving experience
I’d almost stopped searching for my Swiss Army knife. I’ve never used it, but that’s not the point. Just having it gave me that reassuring feeling that the time would come when I might be in a pickle that only a nifty little tool on the knife would get me out of. Then, the other day, up in the loft, I found it; in a tea chest marked “car”. Don’t ask.

Life’s like that. Whack the wrong label on something and you might never find it again. Then I started thinking, isn’t data like that too? Just like life. Put it in the wrong box and you might end up spending a lot of time looking for it. This is time that modern businesses don’t have. It’s time that the regulators won’t give you. It’s time spent rummaging in the loft, when you could be unleashing new ideas.

Growing businesses never stand still. Operating across countries and borders keeps you even more on your toes. As customer numbers grow, or your services expand, or you deliver greater collaboration capabilities to your workers, you demand more from your IT infrastructure. Change often requires moving data from one system to another; keeping a robust and trustworthy audit trail along the way.

It’s like moving house; making sure you can function just as smoothly in a new environment as you did in the old one, or even more so. Moving house is a time when many of us clear out the accumulated junk that we always thought would one day come in handy but now appreciate never will. So, when we ship out we shape up at the same time.

New beginnings demand new practices
Everyone looks forward to moving into a new home, but not many of us relish the thought of sorting out all of our ‘stuff’, assessing whether we still need it, whether we should discard it, rationalise it, or box it up for longer term storage.

We also need to organise ourselves and create a plan, so that when we do eventually move home, each packing box is labeled, transported and delivered to the correct room in the new home (assuming there aren’t any traffic delays, roadblocks or accidents, and that we are actually allowed to receive the keys and enter the new home in the right timeframe). Migrating data is exactly the same.

Data is growing in line with the 3Vs; velocity, volume, and variety, the three defining properties or dimensions of big data: “Volume refers to the amount of data, variety refers to the number of types of data, and velocity refers to the speed of data processing. According to the 3Vs model, the challenges of big data management result from the expansion of all three properties, rather than just the volume alone — the sheer amount of data to be managed.”

The role of data in the rise of ML and AI
We never delete anything, just in case we might need it in the future (the accumulated ‘junk’ in the attic). This now proves to have been highly opportune behaviour, as it becomes increasingly clear that we do now need it.

The rise of machine learning (and deep learning) and artificial intelligence – essential enablers of automated processes, operational efficiencies, and better customer experiences – is dependant on lakes of data for extracting useful information and making smarter decisions.

The more data, the more accurate or predictable the results of our planned actions and strategies to support and enhance the competitive advantage. “Machine intelligence,” says the MIT Technology Review, “…is starting to transform everything from communications and computing to medicine, manufacturing, and transportation.”

Handle with care
Let’s assume you have used some tooling to get your data into a clean state. The ‘tooling’ would likely comprise software to analyse the current data and see what is redundant, obsolete or trivial.

Doing this confidently is no easy task. It would best be undertaken by an external technology company, to minimise risk, follow best practice, and be based on an assessment of the prevailing best tools for the job at the point at which you identify the need to migrate data, i.e: a solution-agnostic services provider, as opposed to a vendor. You’ll know what data you need to keep, based on regulation and compliance. At the end of the cleansing process, your data has a lovely green sticker on it, saying ‘ready to go’.

Let’s further assume you, therefore, engage a suitable ‘removal company’. They spend time with you to identify the destination you are heading to, they plan the route carefully, know all the pitfalls of the journey, where to stop and take a break. They may even have done a test of the route, to ensure things go smoothly, and they have worked out the correct short-cuts to take to get you there quicker (by the way, there are no short-cuts, there is always a low bridge or fallen tree to navigate around).

I’m making it seem like a fairly daunting task, but that’s the reality if you are going to lift and shift your data as is. Get it clean, plan the route, make sure you can get the keys when you arrive, and know how much it’s going to cost you to live there long-term (make sure you can afford the monthly payments).

Read the road
If there isn’t a motorway to go on, you may need to use A or even B roads. If you have a lot of data, it’s going to take longer to arrive, especially if you get stuck behind a tractor, or horse and cart.

So, what if you sent some of the data (the static, archival stuff) in a separate vehicle ahead of time, so that as soon as you get the house keys it could start to be unloaded, and would be there, ready and waiting for you, when you eventually move in? How about if you could go straight to your Swiss Army knife? How about if whatever the business needed it could experience zero downtime in trying to access it?

As long as you plan the move, manage the journey, take regular breaks and are prepared for unexpected events, it will all go smoothly. All these considerations apply to businesses who have legacy storage arrays and need to migrate data due to capacity, performance or security, or businesses who need a simple way to migrate data without incurring critical business downtime.

Our goal at Viadex is to support your goal; unhindered forward momentum across the business. Our data migration service covers off every step of the journey. It starts in the right place, getting your data ‘clean’ before migration, or at least giving you the insights into your data so that they can migrate with your eyes open. We’ll even have a cup of tea ready and waiting, once we get you to the better place; the new home, the new start, the technological equivalent of the handy multi-tool that I must now bring into regular service. How did I ever do without it?