Archive for July, 2006
July 30, 2006 at 9:26 pm · tags: Python SummerOfCode
OpenSSL is a damn beast but I managed to handle it through the ctypes code generator written by Thomas Heller. This code is based upon GCCXML tool. Using the code generator is really straightforward:
rhymes@groove ~ % python2.4 h2xml.py ssl.h -o ssl.xml
It produces an XML representation of the header to feed to xml2py.py command this way:
rhymes@groove ~ % python2.4 xml2py.py ssl.xml -o ssl.py
Now you have a complete wrapping of the main OpenSSL header file to use in your ctypes based application.
Going back to the actual SoC work: I implemented the whole _ssl.c CPython file in rctypes. So you have ssl_socket.issuer(), .server(), .write() and .read().
By now is PyPy only (because the extension compiler doesn’t fully translate it).
July 27, 2006 at 6:43 pm · tags: SummerOfCode
I think I’ve found the most complicated library ever.
That library is really over-engineered. I spent a whole day trying to understand _ssl.c, wrapping ssl.h and the other headers I need with ctypes codegenerator (doing it manually is insane), writing 400+ lines of initialization code, dealing with the OpenSSL man pages and so on and I encountered one problem more:
Some functions like SSL_CTX_set_options are not exported by the shared library so I can’t use them with ctypes. But since the initialization protocol is not complete the SSL_connect fails miserably.
And if I can’t initialize the SSL protocol I can’t write the module. Crystal.
I think I’m cursed 
July 25, 2006 at 9:44 pm · tags: Python SummerOfCode
mmap module is complete. We (I and I, I guess) still have a problem with special
methods. Methods like __xyz__ are not supported by the PyPy’s extension compiler so
I had to implement stuff like len(mmap_object) as mmap_object.get_len().
That’s definitely ugly and breaks the compatibility with CPython so keep attention. It also lacks support of slice() objects.
strptime() function has been moved out the time module. There’s not support for it by now.
The matter is strptime() at application level (Python’s) works correctly and tests passes but it doesn’t compile for a bunch of reasons: first, TimeRE() class is a subclass of dict() objects (managed to fix that); second, it relies on enormous amount of modules being a single apparently harmless function (to name some: calendar, re, datetime, locale which in turn raaelies on the whole encodings package). So I tried to “backport” some of the stuff the ext. compiler didn’t like but I got stuck around the encodings package.
I decided to remove that function by now. I’ll see.
Now I gotta concentrate on bz2 and ssl to complete my tasks.
July 21, 2006 at 6:16 pm · tags: Java
I don’t know what to think about somebody who really likes Java because this really odd:
int[] arr = (int []) arr.clone();
arr is declared as a int []. VERY UGLY. That’s an hack.
Java 5 seems a bit smarter and it doesn’t require the cast.
I used Java 5 while developing but I realized it has to run on top of Java 1.4 so here it is 
Gotta convince people in universities to let go of Java.
July 19, 2006 at 4:38 pm · tags: SummerOfCode
time module:
- asctime() has been implemented working around the *args limitation
- strftime() has been implemented
- strptime() is there but it’s the one which breaks the compilation step.
mmap module:
- mmap creation: ok.
- close(): ok.
- read_byte(), readline(), read(): ok.
- find(), seek(): ok.
- tell(), size(): ok.
I have problems with flush(), and writeXX() because some casting operations
are not supported in rctypes.
bz2 module is still in the void (there’s A LOT of scaffolding in that module)
fcntl module is completed (fcntl.ioctl() still lacks buffer() and array.array() support)
July 16, 2006 at 10:45 pm · tags: SummerOfCode
I just received my O’Reilly Safari-90-day-summerofcode-account!
Yay!
July 14, 2006 at 8:22 am · tags: SummerOfCode
A brief status update:
the time module is almost finished but:
- asctime() is masked because of a problem in the extension compiler to handle *args.
- strftime() and strptime() are not implemented for the same reason.
the fcntl module is basically finished:
- ioctl() does not support array.array() instances and buffer() instances but everything else is implemented.
the mmap module has been started but since the compiler lacks support for **kwargs (and *args too) I had to switch to another task in the meantime.
I’ve started bz2 porting. More on this in a later post when I’ll manage to pass some tests.
All is currently being tested on OSX, Ubuntu Linux and FreeBSD 6.1. Windows support is a tremendous mess because of the nature of Windows, problems with license of software and other stuff. I think I’ll dig into Windows AFTER completing the five modules (_ssl.c is the last one).
July 10, 2006 at 11:44 pm · tags: PyPy Python SummerOfCode
I’m making progresses to feed the rctime module to the extension compiler. He doesn’t like that much tuples so fixing a couple of tupled line to lists decreased the number of translation errors. Now the translation goes fine but I’m stuck in the compiler process. ctypes doesn’t have a mechanism to explicitly tell that one function returns void (nothing) and/or and by default all functions in ctypes return an integer.
The C generated source code from the translation process assigns the “hypothetic” return value of the function I’m calling (in this case tzset()) to a variable and GCC (he’s right ovbiously) does not allow that.
One step forward anyway!
UPDATE: I should think about the importance of None between one debugger session and the other. Thanks to Albert Strasheim. Better go to bed now.
I think I’ve found a bug in the rpython translation layer but more on this later.
July 9, 2006 at 3:52 pm · tags: PyPy Python SummerOfCode Switzerland
My last sprint day was good but I lost a lot of time around some limitations and a bug that’s actually there (anyway unit tests pass) but I’m fighting to win, so I’ll win
I had the chance to pair with Armin Rigo itself and I think that I learned a lot from him (and also from Anders of course). Pair programming with the guys like them it’s important to my knowledge of PyPy and Python.
We have been invited by Richard Jones, a pygame guru, to see a nice and funny presentation about pygame and its capabilities. He was using Bruce as the presentation tool and I’ll definitely try it because it seems awesome!
Other guys have started porting md5, random and zlib also so maybe we’ll managed to port all the Python standard library someday
In the evening Fabrizio and I played to Carcassonne with Armin, Richard and a really nice swedish summer student named Pauline. We italians didn’t know that game but Richard was very kind to teach us how to play. Obviously we lost but Richard, a master board game player, was defeated by Pauline
So that’s all from my first PyPy sprint.
If you want to follow my SoC/PyPy work you’ll find it in pypy.module.rctime (which probably will be renamed as time when completed).
Michael Hudson wrote a PyPy sprint report today to summarize all the work has been done in Geneva: PyPy Post-EuroPython 2006 Sprint Report
July 8, 2006 at 2:03 am · tags: PyPy Python SummerOfCode Switzerland
Today I continued working on the rctime module with the supervision of Anders. I’m improving my knowledge of rpython and how the interpreter level and application level works and how to make them communicate. This is not really documented in deep so you’ve to check out examples also.
Sad, a swiss boy, joined us and started learning how rctypes work. I think he’s porting zlib module by himself. Now he has a codespeak account so he can work on the live project.
We checked in the fix in format_somobject_error because it was needed by other people that encountered the same problem.
I now know the basics of the machinery of making tests pass and the translation process too.
You can’t use space.unwrap() directly in the interpreter level stuff that goes translated because it’s not supported at all (thanks Armin and Anders!) so you have to use XXX_w functions in the object space.
In the morning I also tried to make my stuff work on Windows but I gave up after installing the .NET SDK 1.1, .NET runtime 1.1, mingw compiler and do some hacking to work around the actual problem that python 2.4 is compiled against msvcrt71.dll but it’s part of the MS VC++ 2003 toolkit that is not available anymore. Visual Studio 2005 provides msvcrt80.dll so I didn’t manage to make it work. Windows is very unfriendly to developers not using all MS tools.
I completed time.sleep(), time.clock(), time.ctime(), time.struct_time, time.gmtime(), time.localtime(), time.mktime()
I have a problem using the translated module on CPython because I have a problem in checking the presence of an optional argument. So basically, if you don’t pass anything to, for example, time.localtime() it fails but if you explicitly pass None as the actual argument it works well. On py.py it works very well. Tonight I’ll try to translate all pypy.
After the sprint we had dinner and after Fabrizio, Alexander, Antonio and I joined summer students at a very funny Dutch party. Summer students here at CERN know how to have fun! They’re obviously physicists or computer scientists from all over the world. From Japan to New York City.
Next entries »
