When moving onto a modern ECM platform, too often the migration of data and content is overly simplified or an afterthought. This oversimplification inserts unnecessary chaos and risk into what is already a complex project.  With appropriate tools, strategy and planning, migrations can improve the end users’ experience with the new platform. This blog post will walk you through Docuvela’s top ten recommendations to ensure a successful ECM migration.

1.  Utilize Resources with Experience

It is natural to try and leverage existing internal resources to perform migration activities.  Too often these resources can struggle with the migration because of:

• Lack of experience with migrations.
• Unfamiliarity with migration software.
• Limited knowledge of the source and/or target content repository.
• Lack of time or desire to focus on what will be a one-time migration activity.

As we have advised many customers in the past, migrations are easily outsourced because the skills are not needed after the project is over.  Customers should look to focus valuable internal resources on becoming experts on the new platform and supplement internal resources with external experts who have deep knowledge and migration experience.

2.  Leverage a Robust, Proven Migration Tool 

There are lots of migration tools on the market. Choose one that has a successful track record. Migration tools do a lot more than move data and content. At a minimum the tool should:

• Be scalable – Has your selected migration tool migrated content at your required volume? Is it multi-threaded?  Does it scale well? Be sure to have deep technical discussions between the business, it and any hired service providers so you understand where to expect the bottlenecks.
• Have robust auditing – Detailed audit logs are paramount to a smooth migration. Migrations will stop for a myriad of reasons (connection lost, database limitations, corrupt data, etc.) being able to quickly understand where a migration stopped and what content and data need to still be migrated is very important to keeping a migration on time and on budget (see more under point 10 below).
• Provide flexibility in how migrations are performed – Companies typically need flexibility to determine how to batch up content. Limiting this flexibility to folder structure is typically not sufficient. It is common to break migrations up by department, commonly accessed content, or by creation date when performing a delta migration.
• Provide the ability to transform data and content – Migrated data and content are often decades old and the result of multiple system consolidations. Cleaning up this old data and content and re-aligning the metadata to current standards and needs should be evaluated and part of any migration. (See more under point 4 below.)

3.  Avoid the “Big Bang” Migration Approach

Sometimes customers will become too focused on retiring the old system as soon as possible.  Successful migrations focus on migrating groups of documents or users to build success in the new system over time rather than a “move everything all at once” approach. Phased migrations that migrate by department, last-in first out (LIFO), document type, etc. have several benefits including:

• Limiting scope, which always increases project success rates.
• The ability to learn from experience and adjust future migration phases. For example, more end user training is required or a larger cutover window is needed.
• Phased rollouts can also provide a longer lead time to move over integrations with other applications. For one customer with over sixty integrations a phased rollout allowed these integrations to be moved and tested over time.
• Finally, performing a phased rollout limits load on the new system initially, allowing for gradual performance tuning and a gradual onboarding of end users into the new environment.

For all these reasons, we commonly recommend delta, gradual, hybrid and rolling migration approaches or a combination thereof.  Watch for a future post that discusses these approaches in more detail.

4.  Migrate with Intent

Include time in your migration plan to “clean house” and perform data cleanup as part of the migration. Often, source systems contain content and data that has been acquired over decades. Common ways of cleaning content include:

•  Removing metadata that is not used.
• Simplifying the security model.
• Rearranging the folder structure.
• Removing duplicate entries.
• Massaging metadata.

Garbage in equals garbage out. Absolutely take time to review data and perform cleansing activities as part of the migration.

Finally, remember that migrations should be done to provide added business benefits to end users.  Many times, migrations are part of a cloud initiative or other IT improvement effort.  With an ever-increasing need for business benefits to justify IT efforts, we would recommend looking for new or updated business benefits rather than only a platform change to justify the migration effort.

5.  Perform Benchmark Migrations

Performing benchmark migrations is a critical step in migration planning. It not only provides an estimate of how quickly the production migration will run, but also gives an opportunity to identify and address bottlenecks that could dramatically slow down a migration at scale. A missing database index, too many files in one folder, and under-powered hardware are all examples of adjustments we have seen customers make as a result of benchmark testing.

It is important to realize that regardless of the amount of testing, issues will arise during migration. Keep in mind that despite volume testing, given large amounts of documents and numerous scenarios, projects should not plan on an idealistic migration timeline.

6.  Test with Production – Not Sample Data

As much as possible, we encourage customers to perform tests or “dry-run” migrations from the production environment, or an exact clone of the production environment, so that migration issues can be identified and addressed before the production migration. Similarly, the more that the test environment can be sized the same as production, the more accurately migration timing estimates will be, leading to a much smoother production migration.

7.  Involve Business SMEs Throughout the Process

We have found it critical to obtain feedback from the business as early as possible in the project so that the end product is primed for success. There are two key points where business users are involved. First, business users should be included in the upfront requirements gathering phase. Key activities requiring their involvement include

• Confirming that IT resources understand the nuances of the data, such as the purpose of each metadata field and any transformation rules being applied, organization strategy, etc.
• Defining and approving the migration mapping/requirements since the business users are the content experts.
• Writing and approving the test strategy, including fully understanding what is being tested and what is not being tested.

The second key point that businesses should consider when obtaining feedback is to include at least one key business user in validating the migration dry runs.  Business users will more readily find incorrect mappings or missing information since they work closely with the content and metadata that is being migrated. This is also an opportunity for the business to adjust the migration prior to the production run. Sometimes seeing the data in context in the new system will cause slight modifications to how content and metadata should be moved.

8.  Be Detail-Oriented

Migrations typically involve moving data and content that have been gathered over decades. Most content repositories contain content from multiple system consolidations as well as company acquisitions.  Understanding and documenting the nuances between the different data sets will significantly reduce costly surprises during testing and production migration runs.

Break repositories down by department, document type, how the content was originally ingested and other factors to determine how many unique scenarios exist. Document and test each scenario thoroughly, including each migration rule and mapping. This process can sometimes lead to hundreds of unique migration cases, but taking these painstakingly detailed steps during the unit testing phase is key to a smooth and successful production migration.

9.  Aim for a One-Step Migration Process

We are often brought into migrations that have already been attempted once before, and typically the migration approach was to use the “export” tool provided by the legacy system, and cobble that together with the vendor provided “import” tool for the new system in a “two step” migration. There are numerous reasons this approach should be avoided, some of which are:

• Issue resolution responsibility between the two different processes and tools.
• Inability to quickly address and retry documents/data that failed to migrate correctly in one or both of the migrations.
• Complexity involved to provide accurate counts of failure and success documents for final decommissioning reports of the legacy system.

When things go wrong with either side of the two-step migration, finger pointing, and ownership questions can erode confidence and the ability to quickly resolve issues. Additionally, two steps are typically close to double the work. It involves two sets of mapping, double the amount of testing (testing export and import process separately) and more places to look when migrations are interrupted, or content and data are not migrated correctly. We highly recommend utilizing a migration approach that can support a one-step migration. 

10.  Have a Verification Plan in Place

Rarely do large migrations run perfectly without any failures, especially when migrating from legacy systems that have exceeded their life expectancy.

It’s critical to identify a sample set of content to verify during the test and production migration runs to ensure that content, metadata, folder structure, security, etc. are all mapped correctly in the target system. Verification is tedious, but it’s important that ALL possible scenarios are verified.

Remember that even with detailed planning and verification, there are almost always instances of corrupt files, bad/missing metadata, and other unexpected scenarios that occur during a migration. Even if the error rate is a very small number, reconciling and resolving errors can be a time-consuming process and should be planned for.

Reconciliation can be greatly simplified if the migration tool being used has a robust logging mechanism for reporting success and failures at an individual document level.  We would recommend this as a key feature of any migration utility that is used for migrating more than a few thousand documents.

SUMMARY

Following these 10 tips will put you on the path to success. If you take anything away from this blog, remember to always treat the migration component of your project with the same level of planning and risk management as you would when releasing a new application. Good luck with your project and reach out if you would like to discuss your migration effort and concerns in more detail.