67

I have a need to watch a log file for changes. After looking through stackoverflow questions, I see people recommending watchdog. So I'm trying to test, and am not sure where to add the code for when files change:

import time
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

if __name__ == "__main__":
    event_handler = LoggingEventHandler()
    observer = Observer()
    observer.schedule(event_handler, path='.', recursive=False)
    observer.start()
    try:
        while True:
            time.sleep(1)
        else:
            print "got it"
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

Where do I add the "got it" — in the while loop if the files have been added/changed?

martineau
  • 119,623
  • 25
  • 170
  • 301
Cmag
  • 14,946
  • 25
  • 89
  • 140

5 Answers5

126

Instead of LoggingEventHandler define your handler:

#!/usr/bin/python
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler


class MyHandler(FileSystemEventHandler):
    def on_modified(self, event):
        print(f'event type: {event.event_type}  path : {event.src_path}')


if __name__ == "__main__":
    event_handler = MyHandler()
    observer = Observer()
    observer.schedule(event_handler, path='/data/', recursive=False)
    observer.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

on_modified is called when a file or directory is modified.

David Dehghan
  • 22,159
  • 10
  • 107
  • 95
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • 2
    Excellent, thank you! Now how would i only watch a particular file, and now the whole directory? – Cmag Sep 03 '13 at 18:53
  • 2
    You are welcome. You can watch for the directory that contains the file, then check for `event.src_path` variable in the handler methods. – alecxe Sep 03 '13 at 18:57
  • And, plus, you may watch particular [event](http://pythonhosted.org/watchdog/api.html#event-classes). – alecxe Sep 03 '13 at 18:59
  • 2
    this code locks the file, so it cannot be used by other programs – denfromufa Apr 17 '14 at 20:17
  • I've got problems with this exact code, it sends 4 `Got it!` when I just modify one file. – majidarif Jul 10 '14 at 07:25
  • 1
    @majidarif: define the on_create. on_delete etc methods and see what's going on – Mr_and_Mrs_D Jul 16 '14 at 13:40
  • 1
    Found out that sublime text creates additional files which was causing this. changing the settings fixed this. – majidarif Jul 16 '14 at 14:18
  • 1
    I have the same problem as @majidarif but I'm not using Sublime Text. The directory I'm monitoring is a network share and as I copy a (657MB) file into it, I see several "Got it!" messages. I only want the last one, as the file is not complete until the last one is there, and I want my code to access the file and do something with it. I tried playing with queues and timeouts but had no luck. Any ideas? I'd love a new event that means "i'm done modifying the file!". – Dan Tenenbaum Oct 21 '14 at 01:39
  • Isn't this far more inefficient that `os.stat` polling? since it's still polling and polling a directory? Which you then have to wake-up and see if the event is an actual file-change to a targeted file or not. – user3467349 Jun 12 '15 at 16:30
  • @user3467349 would be glad if u would actually provide that "better" solution. Thanks. – alecxe Jun 12 '15 at 16:56
  • @user3467349 plus, anyway, this is a watchdog-specific question. – alecxe Jun 12 '15 at 16:58
  • 4
    @DanTenenbaum did you ever find out how to fix the repeated 'Got it' messages? I get two consecutive messages as well. – Helk Sep 20 '17 at 00:46
  • @alecxe What does the try block provide? – TheRealFakeNews Feb 01 '19 at 02:25
  • @AlanH It allows the watchdog observer to be ended nicely when the user is ending the program by hitting Ctrl+C – Florentin Le Moal Mar 13 '19 at 16:10
  • 2
    what does ```observer.join()``` do in the code? also does the observer have its own thread or does it run in the main thread (trying to figure out why the ```time.sleep(1)``` call would work since observer seems to use threading)? the watdog docs seem a little sparse so trying to piece together how to use this thing... – bob Dec 06 '19 at 17:19
12

Here's a snippet to prevent it running twice as others have commented in @alecxe answer:

from datetime import datetime, timedelta

class MyHandler(FileSystemEventHandler):
    def __init__(self):
        self.last_modified = datetime.now()

    def on_modified(self, event):
        if datetime.now() - self.last_modified < timedelta(seconds=1):
            return
        else:
            self.last_modified = datetime.now()
        print(f'Event type: {event.event_type}  path : {event.src_path}')
        print(event.is_directory) # This attribute is also available
run_the_race
  • 1,344
  • 2
  • 36
  • 62
  • Thanks for this - exactly what I was looking for, if you're still active could you answer my Q here (actually this Q answers most of it) and i'll green tick : https://stackoverflow.com/questions/57531818/python-watchdog-watch-a-directory-and-rename-file-on-event-modification – Umar.H Aug 17 '19 at 14:15
  • the print statement print(f'Event type: {event.event_type} path : {event.src_path}') gives an error ? i changed it to print(event.event_type) print(event.src_path) print(event.is_directory) maybe someone can clarify why? – Cat Nov 20 '19 at 14:28
  • @Cat, Are you using a version of Python<3.6 ? Since python 3.6 they introduced f-strings: https://realpython.com/python-f-strings/ They are great! – run_the_race Nov 22 '19 at 08:07
  • If there's only single file, this is fine. But if there are multiple files in the directory, this can hide events from the user. – Javon Jan 03 '21 at 18:38
0

Instead of datetime, you may go with the src_path check logic since if the logic after the checking more than 1-second datetime logic will fail.

class EventHandler(FileSystemEventHandler):
    def __init__(self):
        self.src_path = ''

    def on_modified(self, event):
        if self.src_path == event.src_path:
            return
        else:
            self.src_path = event.src_path
        logger.info(f"{event.event_type} occured on file {self.src_path}")
        #your long processing logics goes here.
Basil Jose
  • 1,004
  • 11
  • 13
0

For others that came across this question and want the answer for: "how to watch a specific file", also asked by @Cmag, this is a quick way to watch specific files. You set the specific files as regexes and use

directory_to_watch = './'
event_handler = RegexMatchingEventHandler(regexes=['.*/config.json'])
observer = Observer()
observer.schedule(event_handler, directory_to_watch, recursive=False)
observer.start()
try:
    while True:
        time.sleep(1)
finally:
    observer.stop()
    observer.join()
Ianau Andrei
  • 81
  • 9
0

Here is the code similar to previous answers if you want to watch a specific file and get the content that was changed on it since last checking:

#!/usr/bin/python
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler


class MyHandler(FileSystemEventHandler):
    def __init__(self):
        self.last_content = None

    def on_modified(self, event):
        if not event.is_directory and event.src_path.endswith('filename.txt'):
            with open(event.src_path, "r") as f:
                content = f.read()
                if self.last_content is not None and content != self.last_content:
                   changed = content[len(self.last_content):]
                   print(changed)
                
                self.last_content = content


if __name__ == "__main__":
    event_handler = MyHandler()
    observer = Observer()
    observer.schedule(event_handler, path='.', recursive=False)
    observer.start()

    print('File monitor started')

    try:
        while True:
            time.sleep(10) #sleep seconds
    except KeyboardInterrupt:
        observer.stop()
    observer.join()
Romulo Diniz
  • 173
  • 2
  • 13