I needed a quick locking mechanism to prevent my daemons from stepping over each other. To have a sane daemon startup (and prevent multiple daemon spawns), we need to ensure that we have an exclusive lock before starting the program. Googling around didn't lead to show any context managers that actually use the flock syscalls.
So here goes my attempt that seems to work:
Spinning off some python processes that utilise this context manager shows serialisation taking place:
And here's the output of lsof showing locking for the processes spun off above:
Showing posts with label flock. Show all posts
Showing posts with label flock. Show all posts
Tuesday, July 8, 2014
Sunday, July 22, 2012
xargs and race conditions
I tend to use xargs as a quick paralleling tool. It works great. However, you may notice that due to the multiple processes running, output from one process will overlap with another process creating an unreadable mess. As an example, I want to ping 6 addresses at the same time. I can do it serially at 1 ping/sec for a total of 6 seconds. Linear growth or O(n):
How do you get around it? We can run everything in parallel using xargs
Note that the time taken is markedly reduced but.. Output from each process is interleaved with other commands.
How do you get around that? Well, locking.. Luckily, the shell has a nifty locking utility that is just perfect for this. Enter flock..
Still as fast and neater. Man flock for more awesomeness. The flock utility uses the flock syscall to acquire a lock on a file structure. In the example above, I show that you can use a file that you are reading as a lock file. flock doesn't prevent you from conducting other file ops on your fd.
How do you get around it? We can run everything in parallel using xargs
Note that the time taken is markedly reduced but.. Output from each process is interleaved with other commands.
How do you get around that? Well, locking.. Luckily, the shell has a nifty locking utility that is just perfect for this. Enter flock..
Still as fast and neater. Man flock for more awesomeness. The flock utility uses the flock syscall to acquire a lock on a file structure. In the example above, I show that you can use a file that you are reading as a lock file. flock doesn't prevent you from conducting other file ops on your fd.
Subscribe to:
Posts (Atom)