Use network upload to import SharePoint data to Office 365

This article is for administrators. Are you trying to upload your files to SharePoint Online or OneDrive for Business? See Upload files to a library

Use the Office 365 Import service and network upload to import files and documents to SharePoint Online and OneDrive for Business sites in your Office 365 organization. When you use network upload, you copy a SharePoint Online migration package (which consists of data and manifest files) to an Azure storage location in the Microsoft cloud. Then the Office 365 Import service copies the files from the storage area to the target site in Office 365. Using network upload is the most efficient way to import documents to SharePoint Online team sites.

Here are the steps for using network upload to import SharePoint files to sites in your Office 365 organization:

Step 1: Install the SharePoint Online Management Shell and the Azure AzCopy tool

Step 2: Create SAS URLs and container names

Step 3: Create a SharePoint Online migration package

Step 4: Prepare the SharePoint Online migration package

Step 5: Upload the migration package to Office 365

(Optional) Step 6: View a list of the data and manifest files uploaded to Office 365

Step 7: Create a data import mapping file

Step 8: Create a data import job in Office 365

Before you begin

  • You have to be a global administrator in your Office 365 organization to access the Import page (under Data management) in the Office 365 Security & Compliance Center.

  • For frequently asked questions about using network upload to import SharePoint data to Office 365, see FAQ about importing SharePoint data to Office 365.

  • If you need to import data from My Sites in your on-premises SharePoint Server organization to users' OneDrive for Business sites in Office 365, consider the drive shipping method where you copy the migration packages to a physical hard drive and then ship it to Microsoft. Then data center personnel uploads the data to the Azure storage location and the Office 365 Import service imports the data to OneDrive for Business sites. For details, see Use drive shipping to import SharePoint data to Office 365.

Return to top

Step 1: Install the SharePoint Online Management Shell and the Azure AzCopy tool

The first step is to install the tools you'll use to upload your files to Office 365. You'll use the SharePoint Online Management Shell in Steps 3 and 4 to create and prepare a migration package, and then you'll use the Azure AzCopy tool to upload them to a temporary Azure storage location in the Microsoft cloud. You only have to do this step once. After you install these tools, follow Step 2 through Step 8 each time you want to import a batch of SharePoint or OneDrive for Business files.

  1. Go to https://protection.office.com.

  2. Sign in to Office 365 using the credentials for an administrator account in your Office 365 organization.

  3. In the left pane, click Data management and then click Import.

  4. On the Import page, click Go to the Import service.

  5. On the Import data to Office 365 page, click New job Add Icon , and then click Upload SharePoint or OneDrive data.

  6. On the Upload files over the network page, in step 3, click Download the SharePoint Online Management Shell.

  7. Download and install the SharePoint Online Online Management Shell.

  8. On the Upload files over the network page, in step 4, click Download tool to download and install the Azure AzCopy tool .

  9. In the pop-up window, click Run to install AzCopy.

  10. Keep the Upload files over the network page open. You'll use it in the next step to create URLs and containers names.

Return to top

Step 2: Create SAS URLs and container names

The next step is create and save to a file the SAS URLs and container names that you'll need to upload files. The SAS URLs are a combination of the network URL for the Azure storage location your organization and a Shared Access Signature (SAS) key. This key provides you with the necessary permissions to upload files to your Azure storage location. The container names will be used to create the containers that your files will be uploaded to.

Important   You'll have to create SAS URLs and container names for each migration package that you want to upload to the Azure storage . That's why it's important to save them to a text file, and identify the migration package or destination site they'll be used for.

Tip: If the Upload files over the network page is open from Step 1, you can start this procedure on step 6.

  1. Go to https://protection.office.com.

  2. Sign in to Office 365 using the credentials for an administrator account in your Office 365 organization.

  3. In the left pane, click Data management and then click Import.

  4. On the Import page, click Go to the Import service.

  5. On the Import data to Office 365 page, click New job Add Icon , and then click Upload SharePoint or OneDrive data.

  6. On the Upload files over the network page, under step 2, click Create URLs and container names.

  7. After the following URLs and container names are displayed, copy and save them to a file. Be sure to copy the entire line of text.

    • File container name

    • File container SAS URL

    • Manifest container name

    • Manifest container SAS URL

  8. Create additional URLs and containers names (and copy them to the file) for each migration package that you want to upload to the Azure storage location.

  9. Click Cancel to close the Upload files over the network page.

Return to top

Step 3: Create a SharePoint Online migration package

The next step is to create a migration package that's based on the files that you want to upload to the Azure storage location. The migration package also contains metadata files that contains the information used to upload the data files. Use one of the following procedures based on whether the files that you want to import to Office 365 are located on a file share or file server, or located on an on-premises SharePoint Server site.

Create a migration package from source files on a file share or file server

This procedure assumes the files that you want to import to Office 365 are located on a file share or file server in your organization.

  1. On your local computer, open the SharePoint Online Management Shell from the Start page, and then run the following command:

  2. Run the following command:

    $credentials = Get-Credential
  3. In the Windows PowerShell Credential Request dialog box, type the user name and password for your Office 365 administrator account, and then click OK.

  4. Run the following command to connect the shell to your SharePoint Online organization:

    Connect-SPOService -Url https://<Your organization name>-admin.sharepoint.com –credential $credentials

    Tip: To verify that you're connected to your SharePoint Online organization, run the Get-SPOSite cmdlet to get a list of the sites in your organization:

  5. In the SharePoint Online Management Shell, run the following command to create a migration package.

    New-SPOMigrationPackage -SourceFilesPath <File share or server where data files are located> -OutputPackagePath <Location on file share to copy the package to> 

    The following table describes the parameters used in this command, and their required values.

    Parameter

    Description

    Example

    SourceFilesPath

    The path for the file share or file server that contains the data files that will be uploaded to Office 365.

    -SourceFilesPath "c:\Share1\DataFiles\DataImportJob1"

    Or

    -SourceFilesPath "\\FILESERVER01\SPFiles\DataImportJob1"

    OutputPackagePath

    The location where the metadata files for the migration package will be copied to.

    -OutputPackagePath "c:\Share1\DataFiles\ImportJob1\Package"

    Here's an example of the syntax for the New-SPOMigrationPackage cmdlet using actual values for each parameter:

    New-SPOMigrationPackage -SourceFilesPath -SourceFilesPath "\\FILESERVER01\Files\DataImportJob1" -OutputPackagePath "c:\Share1\DataFiles\ImportJob1\Package" 
  6. Leave the SharePoint Online Management Shell open. You'll use it in Step 4 to prepare the migration package so you can upload it to Office 365.

Return to Step 3

Create a migration package from files on a SharePoint Server site

This procedure assumes the files that you want to import to Office 365 are located on an on-premises SharePoint Server site.

  1. In your on-premises SharePoint organization, open the SharePoint Management Shell.

  2. Run the following command to create a migration package:

    Export-SPWeb -Identity <Web site URL> -ItemUrl <Web app URL> -Path <Migration package storage location> -NoFileCompression

    The following table describes the parameters used in this command, and their required values.

    Parameter

    Description

    Example

    Identity

    The URL for the on-premises SharePoint site where the files that you want to import to Office 365 are located.

    -Identity http://contoso/sites/CorpSite

    ItemUrl

    The Web application that will be exported. The identify of this Web application is used during the network upload process to convert the .dat files in the migration package to the appropriate app; for example a Word document or a PowerPoint file.

    Tip   Run the following command to get a list of the possible values that you can use for this parameter.

    Get-SPWeb <Web site URL> | Select-Object -ExpandProperty folders | fl Url

    Common values to use for this parameter are "Documents" (for MySites) and "Shared Documents" (for team sites).

    -ItemUrl "Shared Documents"

    Path

    The location of the folder on a file share or file server where the migration package will be copied to. Because the NoFileCompression switch must be used, this parameter is required.

    -Path c:\Share1\DataFiles\ContosoCorpSite

    NoFileCompression

    This switch disables file compression in the export package. This parameter is required to export a package that can be uploaded to the Azure storage location in Step 5.

    -NoFileCompression

    Here's an example of the syntax for the Export-SPWeb cmdlet using actual values for each parameter:

    Export-SPWeb -Identity http://contoso/sites/CorpSite -ItemUrl "Shared Documents" -Path c:\Share1\DataFiles\ContosoCorpSite -NoFileCompression

Return to Step 3

Return to top

Step 4: Prepare the SharePoint Online migration package

After you create the migration package (from source files on file share, a file server, or an on-premises SharePoint site), the next step is to prepare the migration package by running the ConvertTo-SPOMigrationTargetedPackage cmdlet in your SharePoint Online organization. This cmdlet verifies that the migration package and the source files match, and then validates the SharePoint Online site and document library that you want to import the data files to. The cmdlet creates a target package that will be uploaded to the Azure storage location in Step 5.

  1. In the SharePoint Online Management Shell, run the following command. See the previous step for instructions on how to connect to your SharePoint Online organization.

    ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath <Location of source files> -SourcePackagePath <Location of  metadata files> -OutputPackagePath <location to copy the target package to> -TargetWebUrl <Target SharePoint or OneDrive for Business site> -TargetDocumentLibraryPath <Target document library>

    The following table describes the parameters used in this command, and their required values.

    Parameter

    Description

    Example

    SourceFilesPath

    The path for the file share or file server that contains the data files that will be uploaded to Office 365. Use same value that you used for the SourcesFilesPath parameter when you ran the New-SPOMigrationPackage in Step 3.

    If you created a migration package from files on a SharePoint Server site by using the Export-SPWeb cmdlet, use the same value that you used for the Path parameter.

    -SourceFilesPath "\\FILESERVER01\SPFiles\DataImportJob1"

    Or

    -SourceFilesPath "c:\Share1\DataFiles\ContosoCorpSite"

    SourcePackagePath

    The location where the metadata files for the migration package will be copied to.

    If you created a migration package from files on a SharePoint Server site by using the Export-SPWeb cmdlet, use the same value that you used for the Path parameter. That's because the metadata files were copied to the same location as the data files.

    -OutputPackagePath "c:\Share1\DataFiles\ContosoCorpSite"

    OutputPackagePath

    The location on the file share or file server where the metadata files for the target migration package produced by this cmdlet will be copied to.

    -OutputPackagePath "c:\SPFiles\DataImportJob1\TargetPackage"

    TargetWebUrl

    The URL for the Web site in your SharePoint Online organization that you're importing the data files to. This could be the URL for a SharePoint or OneDrive for Business site.

    -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite

    TargetDocumentLibraryPath

    The name of the document library (located in the site specified by the TargetWebURL) that the data files will be imported to.

    -TargetDocumentLibraryPath "Shared Documents"

    Here are two examples of the syntax for the ConvertTo-SPOMigrationTargetedPackage cmdlet.

    • This example is for a migration package created from source files on a file share or file server.

      ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath "\\FILESERVER01\SPFiles\DataImportJob1" -SourcePackagePath "c:\Share1\DataFiles\ImportJob1\Package" -OutputPackagePath "c:\SPFiles\DataImportJob1\TargetPackage" -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite -TargetDocumentLibraryPath "Shared Documents"
    • This example is for a migration package created from source files on a SharePoint Server site. Note that SourceFilesPath and SourcePackagePath use the same value.

      ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath "c:\Share1\DataFiles\ContosoCorpSite" -SourcePackagePath "c:\Share1\DataFiles\ContosoCorpSite" -OutputPackagePath "c:\SPFiles\DataImportJob1\TargetPackage" -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite -TargetDocumentLibraryPath "Shared Documents"

Return to top

Step 5: Upload the migration package to Office 365

Now you're ready to use the AzCopy.exe tool (installed in Step 1) to upload the migration package (that you created and prepared in Step 4 and Step 5) to Office 365. This tool uploads and stores the data and manifest files to the Azure storage location for your organization. You'll run the AzCopy tool twice: once to upload the data files to the files container and once to upload the manifest files to the manifest container.

  1. Open a Command Prompt (Run as administrator) on your local computer.

  2. Go to the directory where you installed the AzCopy.exe tool in Step 1. If you installed the tool in the default location, go to %ProgramFiles(x86)%\Microsoft SDKs\Azure\AzCopy.

  3. Run the following command to upload the data files to the file container in the Azure storage location.

    AzCopy.exe /Source:<SourceFilesPath> /Dest:<File container SAS URL> /V

    The following table describes the parameters and their required values. Note that the information you got in the previous step is used as the values for these parameters.

    Parameter

    Description

    Example

    /Source:

    Specifies the path for the file share or file server that contains the data files that will be uploaded to Office 365. Use same value that you used for the SourcesFilesPath parameter when you ran the New-SPOMigrationPackage in Step 4.

    If you created a migration package from files on a SharePoint Server site by using the Export-SPWeb cmdlet, use the same value that you used for the Path parameter.

    /Source:"\\FILESERVER01\SPFiles\DataImportJob1"

    Or

    /Source:"c:\Share1\DataFiles\ContosoCorpSite"

    /Dest:

    Specifies the SAS URL for the file container that you got in Step 2. Be sure to surround the value of this parameter with double-quotation marks (" ").

    /Dest:"https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/files-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A10Z&sr=c&sp=wl&sig=XQ6gG7%2B20n%2FRhVEvxwzmKrocks2pGNr8J%2FhDLqL2xbU%3D"

    /V

    Outputs verbose status messages into a log file. By default, the verbose log file is named AzCopyVerbose.log in %LocalAppData%\Microsoft\Azure\AzCopy (for example, C:\Users\admin\AppData\Local\Microsoft\Azure\AzCopy\AzCopyVerbose.log).

    /V

    /S

    This optional switch specifies the recursive mode so that the AzCopy tool will copy files that are located in subfolders in the source directory that is specified by the /Source: parameter.

    /S

    Here's an example of the syntax for the AzCopy.exe tool using actual values for each parameter:

    AzCopy.exe /Source:\\FILESERVER01\SPFiles\DataImportJob1 /Dest:"https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/files-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A10Z&sr=c&sp=wl&sig=XQ6gG7%2B20n%2FRhVEvxwzmKrocks2pGNr8J%2FhDLqL2xbU%3D" /V:C:\Users\Admin\Desktop\AzCopy1.log

    After you run the command, status messages are displayed that show the progress of uploading the data files. A final status message shows the total number of files that were successfully uploaded.

  4. Run the following command to upload the target package to the manifest container in the Azure storage location..

    AzCopy.exe /Source:<OutputPackagePath> /Dest:<Manifest container SAS URL> /V

    The following table describes the parameters and their required values. Note that the information you got in the previous step is used as the values for these parameters.

    Parameter

    Description

    Example

    /Source:

    Specifies the path for the file share or file server that contains the target package that you created in Step 4. Use same value that you used for the OutputPackagePath parameter when you ran the ConvertTo-SPOMigrationTargetedPackage.

    If you created a migration package from files on a SharePoint Server site by using the Export-SPWeb cmdlet, use the same value that you used for the Path parameter.

    /Source:"c:\SPFiles\DataImportJob1\TargetPackage"

    Or

    /Source:"c:\Share1\DataFiles\ContosoCorpSite"

    /Dest:

    Specifies the SAS URL for the manifest container that you got in Step 2. Be sure to surround the value of this parameter with double-quotation marks (" ").

    /Dest:"https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A11Z&sr=c&sp=wl&sig=1%2BGHYQ5nuwTonPxyzxyzR4d9dyxHicEFR2KWHORLzFU%3D"

    /V

    Outputs verbose status messages into a log file. By default, the verbose log file is named AzCopyVerbose.log in %LocalAppData%\Microsoft\Azure\AzCopy. Each time you run AzCopy.exe and specify this parameter, the verbose status message will be appended to the log file.

    /V

    /S

    This optional switch specifies the recursive mode so that the AzCopy tool will copy files that are located in subfolders in the source directory that is specified by the /Source: parameter.

    /S

    Here's an example of the syntax for the AzCopy.exe tool using actual values for each parameter:

    AzCopy.exe /Source:"c:\SPFiles\DataImportJob1\TargetPackage" /Dest:"https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A11Z&sr=c&sp=wl&sig=1%2BGHYQ5nuwTonPxyzxyzR4d9dyxHicEFR2KWHORLzFU%3D" /V:C:\Users\Admin\Desktop\AzCopy1.log

Return to top

(Optional) Step 6: View a list of the data and manifest files uploaded to Office 365

As an optional step, you can install and use the Microsoft Azure Storage Explorer (which is a free, open source tool) to view the list of the data files and the manifest files that you've uploaded to the Azure storage location in Step 5. A good reason to do this is to verify that source files and the manifest files from your organization were successfully uploaded to the Azure storage location.

Note: The Microsoft Azure Storage Explorer is in Preview.

To install the Azure Storage Explorer and connect to your Azure storage area:

  1. Download and install the Microsoft Azure Storage Explorer tool.

  2. Start the Microsoft Azure Storage Explorer.

  3. Right-click Storage Accounts in the left pane, and then click Connect to Azure storage.

    Right-click Storage Accounts and then click Connect to Azure storage
  4. In the box under Connect to Azure storage, do one of the following based on which container you want to view.

    • Paste the file container SAS URL (from Step 2) to view the file container.

    • Paste the manifest container SAS URL (from Step 2) to view the manifest container.

    Paste the SAS URL for the file or manifest container
  5. Click Next.

  6. On the Connection summary page, you can review the connection information, and then click Connect.

    Note: You're assigned the Write and List permissions. This means you can upload files to the Azure storage location, and then view a list of those files. You aren't assigned permissions to delete files or containers.

  7. Under Storage Accounts, expand the (Service SAS) node, and then expand the Blob Containers node.

  8. Right-click the files container or the manifest container (depending on which one you connected to), and then click Open Blob Container Editor.

    Right-click the file or manifest container and then click Open Blob Container Editor

    The Azure storage area, with a list of the source or manifest files that you uploaded in Step 5 is displayed.

    Azure Storage Explorer displays a list of the data or manifest files that you uploaded
  9. Repeat steps 3 - 8 to connect to the other container.

  10. When you're finished using the Microsoft Azure Storage Explorer, right-click the files or manifest container, and then click Detach to disconnect the container from your Azure storage area. Otherwise, you'll receive an error the next time you try to attach to that container.

    Right-click the file or manifest container and then click Detach to disconnect

Return to top

Step 7: Create a data import mapping file

After the data and manifest files have been uploaded to the Azure storage location for your Office 365 organization, the next step is to create a comma separated value (CSV) file that specifies the SharePoint or OneDrive for Business site that the data files will be imported to. This file also specifies of the location of the data and manifest files that you uploaded to the Azure storage location in the previous step. You'll submit the data mapping file the next step when you create the import job.

  1. Download a copy of the CSV Import mapping file.

  2. Open or save the CSV file to your local computer. The following example shows a completed import mapping file (opened in NotePad). It's much easier to use Microsoft Excel to edit the CSV file.

    Workload,FilePath,Name,Mailbox,IsArchive,TargetRootFolder,SPFileContainer,SPManifestContainer,SPSiteUrl
    SharePoint,,,,,,files-2016062818-1ab5ecbab35d436d83d2694db206daa5,manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5,https://contoso.sharepoint.com/sites/SPOCorpSite
    

    The first row, or header row, of the CSV file lists the parameters that will be used by the Import service to import files to SharePoint or OneDrive for Business sites . Each parameter name is separated by a comma. Each row under the header row represents the parameter values for importing files. Be sure to replace (or remove) the placeholder data in the mapping file with your actual data.

    Important: The mapping file can only contain one data row under the header row. This row will contains the information about the migration package that you uploaded to the Azure storage location. Be sure to create a separate mapping file for each migration package you created and uploaded to the Azure storage location. You can submit multiple mapping files when you create an import job in Step 8.

  3. Use the information in the following table to populate the CSV file with the required information.

    Parameter

    Description

    Example

    Workload

    Specifies the Office 365 service that data will be imported to. To import data files to SharePoint Online or OneDrive for Business, use SharePoint.

    SharePoint

    FilePath

    For importing files to SharePoint or OneDrive for Business, leave this parameter blank.

    Not applicable

    Name

    For importing files to SharePoint or OneDrive for Business, leave this parameter blank.

    Not applicable

    Mailbox

    For importing files to SharePoint or OneDrive for Business, leave this parameter blank.

    Not applicable

    IsArchive

    For importing files to SharePoint or OneDrive for Business, leave this parameter blank.

    Not applicable

    TargetRootFolder

    For importing files to SharePoint or OneDrive for Business, leave this parameter blank.

    Not applicable

    SPFileContainer

    Specifies the file container name that you got in Step 2.

    files-2016062818-1ab5ecbab35d436d83d2694db206daa5

    SPManifestContainer

    Specifies the manifest container name that you got in Step 2.

    manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5

    SPSiteUrl

    Specifies the SharePoint or OneDrive for Business site that the files will be imported to. Use same value that you used for the TargetWebUrl parameter when you ran the ConvertTo-SPOMigrationTargetedPackage in Step 4.

    https://contoso.sharepoint.com/sites/SPOCorpSite

    Note: Don't change anything in the header row, including the Exchange parameters; they will be ignored during the SharePoint data import process.

Return to top

Step 8: Create a data import job in Office 365

The last step is to create the data import job in the Import service in Office 365. As previously explained, you can submit multiple data mapping files (that you created in the previous step) for a single import job. After you create the new import job, the Import service will use the information in the mapping files to import data (that you uploaded to Office 365 in Step 5) to the specified SharePoint or OneDrive for Business site.

  1. Go to https://protection.office.com.

  2. Sign in to Office 365 using the credentials for an administrator account in your Office 365 organization.

  3. In the left pane, click Data management and then click Import.

  4. On the Import page, click Go to the Import service.

  5. On the Import data to Office 365 page, click New job Add Icon , and then click Upload SharePoint or OneDrive data.

  6. On the Upload files over the network page, click the I'm done uploading my files and I have access to the mapping file check boxes, and then click Next.

  7. Type a name for the data import job, and then click Next.

  8. Click Add Add Icon to select an import mapping file that you created in Step 7.

  9. After the name of the CSV file appears in the list, select it and then click Validate to check your CSV file for errors.

    The CSV file has to be successfully validated to create a data import job. If the validation fails, click the Invalid link in the Status column. A copy of your mapping file is opened, with a error message for each row in the file that failed.

  10. Add and validate additional mapping files, if applicable.

  11. When the mapping files are added and validated, read the terms and conditions document, and then click the checkbox.

  12. Click Finish to submit the job.

    The job is displayed in the list of import jobs on the Import data to Office 365 page.

  13. Select the job and click Refresh Refresh icon to update the status information that's displayed in the details pane.

  14. In the details pane, click View details to get the latest status for the selected data import job.

Return to top

Share Facebook Facebook Twitter Twitter Email Email

Was this information helpful?

Great! Any other feedback?

How can we improve it?

Thank you for your feedback!

×