DirectoryMonitor

class HSTB.kluster.monitor.DirectoryMonitor(directory_path, is_recursive=True)

Bases: object

Use watchdog to monitor the provided directory and kick off events based on the type of file system event

Watchdog will issue a FileCreatedEvent as soon as the file appears in a directory, NOT when it is finished copying and accessible. We have to wait until the file is finished copying and is readable. We include an event driven system (self.watch_buffer_timer) that will check the files to see if they are readable before pushing them on to the intelligence module. The intelligence module will immediately read the file for data, so if the file hasn’t finished writing, you get a permissions error.

This file will push the new file using the _newfile setter to any class that has a method bound to this class, see self.bind_to

dm = DirectoryMonitor(r’C:data_dirtj_patch_test’, is_recursive=True)

dm.start()

Attributes Summary

newfile

newly found file by the monitor object, is the full file path

Methods Summary

add_to_buffer(filepath, file_event)	self.my_event_handler will use this method to update the file buffer.
bind_to(callback)	Pass in a method as callback, method will be triggered on setting newfile
build_initial_file_state()	Kicking off the watchdog observer will track all file system changes from this point forward, but it will not get you the current state.
push_to_kluster_intelligence()	Method triggered on timer event.
start()	Start the watchdog_observer
stop()	Stop the watchdog_observer

Attributes Documentation

newfile

newly found file by the monitor object, is the full file path

Methods Documentation

add_to_buffer(filepath, file_event)

self.my_event_handler will use this method to update the file buffer. We push to a buffer before pushing to the intelligence module, as we only want the intelligence buffer to see the file once it has finished copying, i.e. is readable.

Parameters

• filepath (str) – absolute file path to the file being added to the buffer

• file_event (str) – one of ‘created’, ‘deleted’

bind_to(callback)

Pass in a method as callback, method will be triggered on setting newfile

Parameters

callback (function) – method that is run on setting newfile

build_initial_file_state()

Kicking off the watchdog observer will track all file system changes from this point forward, but it will not get you the current state. Apply this method in addition to starting the observer to get the initial state.

push_to_kluster_intelligence()

Method triggered on timer event. Every second we check to see if a file is readable (has finished copying). If so, we use newfile to trigger any observers which then get the newly written (or deleted) file

start()

Start the watchdog_observer

stop()

Stop the watchdog_observer