Python automation - Manipulating files using python

Python is able to do all operations related to files, like moving files between different folders, renaming files, copying files, and so on. The os module in python is used when working with directories to fetch the contents of a directory, create a new directory, switch between directories, etc. We need two more modules for our work, the shutil, and watchdog module. Shutil is an inbuilt module in the python standard library for high-level operations related to files. 

The OS module

The os module in Python provides a way to interact with the operating system. This module is coming under the python standard library itself and no need for installation.  There is a bunch of function in the os module that helps in working with the file system, however, we need two functions for our work.

os.path.join()

import os
  
file = "document.txt"
os.path.join("C://Users//",file)
  
output:
>>> C:\Users\document.txt

os.path.join() helps to create strings for filenames. In the above case, the doument.txt file is joined at the end of the folder name.

os.listdir()

import os
  
path = "C://Users//sidharth//documents"
os.listdir( path )
output:
>>> ['document1.txt', 'document2.txt']

os.listdir() function creates a list of all filenames in a particular directory.

The shutil module

Shutil module in python helps for operations like copying files and folders, moving files, removing files, etc. 

shutil.move()

The shutil.move() function helps to move files from one folder to another respectively

import shutil
  
source = "C://Users//sidharth//documents//document.txt"
new_folder = "E://newdocumentfiles"

shutil.move( source,new_folder )
# moving document.txt from source to 
# new_folder 

Moving files with watchdog module

from watchdog import observers
from watchdog.events import FileSystemEventHandler
import json
import os
import time
import shutil
class FileHandler( FileSystemEventHandler ):
    def on_modified( self,event ):
        for file in os.listdir( path ):
            # string of all filenames in the folder
            doc_file = os.path.join( path,file )
            # Joining the path with file to create
            # an entire string of the directory and
            # assigning to doc_file
            shutil.move( doc_file,target_path )
            # Moving the doc_file to target_path

path = "C://Users//sidharth//Downloads"
target_path = "C://Users//sidharth//Documents"
  
event_handler = FileHandler() # event handler object
observer = observers.Observer()
observer.schedule(event_handler, path, recursive=True)
observer.start()
try:
    while True:
        time.sleep(10)
    except KeyboardInterrupt:
        observer.stop()
        print("Process stopped")
observer.join()

The watchdog module is used not more than monitoring the changes that occurred in a directory. We created an event handler object which continuously checks for the changes in a folder. If there is any change or a new file is added to the downloads folder it automatically detects and moves the file to the documents folder throughout the running of the program. 

Checking the file type

So far we are able to move files automatically, but if your script can identify which type of file is in the directory, it can move those files to their desired folder right!. For example, if you have two files in the downloads folder and one is a document file and another is an image and if the program can detect the document file and image separately, it can move the document file to the document folder and image file to the pictures folder. Checking the file type is not actually a challenging job because it can easily be done by just looking at the extensions, if the extension is 'txt' then it is a text file and if it is 'jpg' or 'png' then it is sure to be an image file. Likewise, we can check every file extension and can distinguish them separately.

class FileHandler( FileSystemEventHandler ):
    def on_modified( self,event ):
        for file in os.listdir( path ):
              # string of all filenames in the folder
             
            if file[-3:] == "txt":
                  # the last 3 characters of filename string 
                doc_file = os.path.join( path, file )
                shutil.move( doc_file, document_folder )
              
            elif file[-3:] == "jpg" or file[-3:] == "png":
                img_file = os.path.join( path, file )
                shutil.move( img_file, image_folder )
  
path = "C://Users//sidharth//Downloads"
document_folder = "C://Users//sidharth//Documents"
image_folder = "C://Users//sidharth//Pictures"

Here we used a conditional statement to check whether the last three letters of the filename are txt,  jpg, or png. if the last 3 characters are txt then it will move to document_folder and if it is png or jpg it moves towards image_folder. You can add more conditions to check more file extensions.

Check this out:Instagram, save posts on the gallery automatically using python and instaloader,

The os module and file handling