Code Painters The Art of Coding


Gevent monkey patching versus sniffer & nose

Unless you already know, gevent is a great library built on top of greenlet module and libevent event loop, that allows for easy coroutine-based cooperative multitasking in networked applications. By the way, you know the C10K paper, don't you?

One of gevent's important features is monkey patching - ability to patch some standard Python modules to make them cooperative. There's no magic here, thanks to Python's dynamic nature gevent simply replaces the standard APIs with its own cooperative equivalents. Existing blocking code can then be reused without any modifications. Cool, isn't it? Most notably, gevent patches socket (replacing blocking I/O with libevent-based asynchronous one), thread and threading modules (replacing threads with greenlets). Continue reading…


Python care and feeding, the Dropbox way

I've just revealed (by pure accident, actually), that Dropbox client binary (i.e. Dropbox.exe file) is in fact a zip file with compiled Python files inside (.pyc files, that is). Interesting, huh? A closer look at the executable file (as well as googling for some strings found in the binary) points to the tool used to pack the application into a single executable file - it's py2exe!

Needless to say, it's tempting to look closer at the .pyc files, or even try to decompile them. Here comes the surprise - the files seem encrypted. Typically, .pyc contains a lot of human-readable strings (at least the file name and variable/function names), but it's not the case with files from unzipped Dropbox.exe:

czajnik@czajnik:~/work/dropbox/exe$ strings distutils/__init__.pyc  
Oc {

Also, every .pyc file starts with a 4-byte magic number - in case of extracted .pyc files, the magic number equals 0xb7f20d0a - no public Python release ever used this value.

It seems that Dropbox team decided to patch the Python interpreter with some decryption routines, in order to protect the application from reverse engineering. Note, that the Python interpreter (Python25.dll) itself is embedded inside Dropbox.exe as a resource, it can be easily extracted with any resource editor. I guess it's time to start IDA Pro and look deeper šŸ™‚

Edit: For the curious - I've spent some time digging deeper. It turned out there are 2 protection levels - one is a decryption routine plugged into code demarshalling part of static PyObject *r_object(RFILE *p) function (see marshall.c). It's fairly easy to work it around, even without fully disassembling it - nothing stops one from building a small .exe which calls original code in the DLL to decrypt the file. However, there is another protection - the Python virtual machine opcodes are mixed. This required a bit of manual reverse engineering of PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) (see ceval.c), but also turned out to be fairly easy. Now I'm able to uncompyle nearly all the files.


Django deployment, FastCGI and UNIX-domainĀ sockets

For various (mostly security) reasons I prefer to deploy Django applications as separate processes, with web server and actual application running under different UIDs, communicating with each other via FastCGI or SCGI socket. This way application is safer from e.g. vulnerable PHP scripts running within the web server process - it's pretty common for the attackers to use PHP vulnerabilities to read files accessible by the web server itself (for smaller sites it's normal to use a single web server installation for many different applications). Continue reading…