O365Datacleaner can be used on almost any OS and Powershell version, and will make the folder structure under the path you supply fully compatible with Onedrive for Business and/or Sharepoint Online standards. No user action is required, even for long paths. Read on to see how such situations are handled automatically.
Script download
Bulk addon
O365BulkDatacleaner (read instructions carefully!)
Usage
- unzip the zip file on a workstation or server with Powershell v3 or higher, the closer the script is to the data, the better the performance
- ensure you have full permissions on the target folder
- Right click the AlphaFS.dll file, go to advanced and click Unblock if you downloaded this file, otherwise Windows will not allow loading this DLL.
- Run the script in read-only mode from an elevated Powershell CMD prompt:
.\O365DataCleaner_v1.12.ps1 -path “C:\temp\user1HD” -username 1 -adminLog c:\temp\test16.txt -commit 0
- Check for any feedback/errors, if none, check %appdata% for the detailed log, if you’re satisfied, run it in read-write mode by doing -commit 1 instead of -commit 0
Conventions
The script will create two folders in the root path you supply to be made compliant.
- O4BMoved – Contains all files that are nested too deeply to comply with the maximum URL length in O4B/Sharepoint Online
- Non-Migrateable – Contains all files and folders that cannot be migrated (due to illegal extensions like .one or other exceptions)
Technet reference
The office technet article details part of the corrections required by Microsoft. Testing in the field has revealed a few additional undocumented restrictions.
Forbidden characters
- When the script encounters ._, #, %, <, >, |, : ,?,”,*,/ in a file or folder name, it will be stripped. If stripping the character causes the file or folder name to no longer be unique, an integer will be appended.
- Trailing or leading spaces will be stripped
Double spaces will be strippedDouble dots will be stripped
Illegal file extensions
- .one
When a file with the above extensions is encountered, it will be moved to the Non-Migrateable folder
Illegal file prefixes
.lock
CON
PRN
AUX
NUL
COM1
COM2
COM3
COM4
COM5
COM6
COM7
COM8
COM9
LPT1
LPT2
LPT3
LPT4
LPT5
LPT6
LPT7
LPT8
LPT9
will be stripped or replaced with numbers.
Illegal strings in file names
- .files
- ~$
If a file contains this string in its name, it will be moved to the Non-Migrateable folder
Illegal strings in folder names
- _files
- -Dateien
- _fichiers
- _bestanden
- _file
- _archivos
- -filer
- _tiedostot
- _pliki
- _soubory
- _elemei
- _ficheiros
- _arquivos
- _dosyalar
- _datoteke
- _fitxers
- _failid
- _fails
- _bylos
- _fajlovi
- _fitxategiak
- _vti_
If a folder name contains these strings, they will be moved to the Non-Migrateable folder.
Path depth
Onedrive and Sharepoint won’t accept file/folder paths over 250-ish characters. The script has several rules to ensure the given path becomes completely compliant with this rule.
- Filenames over 120 characters will be cropped to 90
- Foldernames over 120 characters will be cropped to 90
If the crop results in a name that is no longer unique, an integer will be appended. File extensions will be retained.
If for some reason the full path to a file is still longer than allowed, the file will be placed in the O4BMoved folder directly in the root of the given path.
AlphaFS is used to deal with > 256 path names in Windows, as the normal Powershell/.NET functions don’t work properly, all credits go to the authors of AlphaFS for this module.
Logging
Every action taken will be logged, whether the script runs in Read-Only or Read-Write mode. The logfile can easily be sorted using Excel and using | as the column seperator.
International character sets
The script can handle foreign character inputs (chinese, russian, japanese, etc) in file and folder names.
Disclaimer
I’m not a real programmer, this code is meant as a Proof Of Concept. I do not guarantee this product will work in your setup, and I offer no dedicated support, I try to help everyone on a best-effort basis but also have to work for a living. So make sure you test well and understand the code before you use it.