Unraveling the Spaghetti: Migrating 3 Million Files to SharePoint

Migrating files to SharePoint can be straightforward, or it can feel like untangling a giant bowl of spaghetti. Recently, we helped a client transform a massive file structure into a clean, navigable SharePoint environment.

The Challenge

Our client needed to move 3 TB of data spread across 3 million files from a traditional file share to SharePoint. While migrations are routine for us, this one had a twist: the client wanted to reorganize their file structure during the migration.

The existing file share included Draft and Final folders at the root. Inside each were thousands of client folders, and each client had dozens to thousands of project folders. One of their largest clients alone had 15,000 project folders at a single level, with the Draft/Final structure mirrored across everything.

The client’s goal was to move the Draft and Final folders under each project level, creating a structure that was more logical and easier to manage. At the same time, we needed to ensure SharePoint’s 5,000-item view limit was never exceeded.

Analysis and Design

The first step was a thorough analysis and design phase. We needed to:

• Understand the existing file structure
• Define a workable naming convention
• Design a folder structure that met the client’s needs and adhered to SharePoint’s rules

This planning phase was critical, without it, migrating millions of files into a new structure would have been chaotic.

Crafting the Migration Plan

With the new structure defined and the scope understood, we developed a detailed migration plan.

While ShareGate is our go-to migration tool, this project’s complexity required more advanced techniques. Our approach included:

1. Leveraging PowerShell and ShareGate PowerShell modules to script the migration
2. Creating a database inventory of all source files
3. Developing queries to calculate the correct destination folder based on the new structure
4. Feeding the results back into ShareGate via PowerShell to control the migration tasks

This strategy enabled a two-step migration:

• Initial migration: Move the bulk of the data in advance
• Incremental migration: Perform the final update during the go-live weekend

This ensured the client met their timeline while keeping downtime minimal.

The Result

Through careful planning, custom scripting, and advanced migration tools, we successfully reorganized and migrated 3 million files. The client now has a SharePoint environment that’s logical, scalable, and compliant with SharePoint’s limits, making ongoing management easier and more efficient.

It’s not just about migrating files, it’s about creating a system that’s organized, scalable, and easy to manage long-term.

Facing a similar challenge? Contact us today to see how we can help.