A python script for automatically moving and deleting files

On our microscopes equipped with high speed (100 fps) sCMOS cameras, we’ve generally set up a fast SSD RAID 0 array for streaming data to and a slower magnetic disk RAID 1 array for longer term data storage. To simplify data management and keep the SSD from filling up, I wrote a script that moves data every night from the SSD array to the magnetic disk array. It also deletes files on the magnetic disk older than 30 days, and benchmarks the write speed of the SSD array, so we can detect any slowdown. In case it’s useful to other people, I’ve posted it on Github.

If you use it, be careful, as it will happily delete whatever directory you tell it to, so you can easily wipe out your OS if you set it up incorrectly.