you may want to do all your I/O operations in one single process (or thread) and communicate the results via sockets, Queue objects (in case of using threads) or whatever, with the client process/thread.
via pytables.org
I'm just writing this here as a bookmark. I have a problem where I have a WSGI app that needs to read and write to an HDF5 file (which is not thread safe). I implemented a writers-preference many-readers/one-writer lock, but now I want to perhaps delete the file from another process, and this is getting too complicated.
I considered using Redis as a decentralized lock system, but for the readers-writer it's far from trivial.