If you’re into Python software development, you might want to take a look at the book webpage, or download a free chapter (PDF) about the standard Python unittest package for now.

Since I’m not a big fan of extensive/exaggerated unit testing and rather believe in functional/component tests, I’m looking forward to read the book and learn some new things. Looking forward to the web application testing chapter, among others.

Disclaimer: I was invited by Packt Publishing to review this book, and they provide a free copy.

MacBook:~ nicolas $ telnet chat.facebook.com 5222
Trying 69.63.176.191...
Connected to chat.facebook.com.
Escape character is '^]'.
<stream />
<?xml version="1.0"?><stream:stream id="F6DE2CB5" from="chat.facebook.com" xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org/streams" xml:lang="en"><stream:error><invalid-namespace xmlns="urn:ietf:params:xml:ns:xmpp-streams"/></stream:error></stream:stream>

Connection closed by foreign host.

Jay!

Last couple of days I wrote some very basic Scala snippets, containing constructs which would be non-trivial or ‘unusual’ to write in Java, compile it to a class file, and then use a Java decompiler to figure out how the Scala compiler maps those constructs to JVM bytecodes.

There’s one thing which took my attention: looks like (basic) tail-recursive functions are optimized into while-loops! This only happens if the last call of a function is a call to itself (the most basic form of tail recursion), but it’s an interesting feature anyway… No more need to put socket accept handling in an infinite while loop

A little demo. First, here’s a Scala object which implements a very basic ‘reduce’ function:

object Reducer {
  def reduce[T, V](fun: (V, T) => V, values: List[T], initial: V): V = {
    if(values isEmpty)
      return initial
    val next = fun(initial, values head)
    return reduce(fun, values tail, next)
  }

  def main(args: Array[String]): Unit = {
    val values = List(1, 2, 3, 4)
    val sum = reduce[Int, Int]((x, y) => x + y, values, 0)
    println("Result: " + sum)
  }
}

We can compile and run this, and it’ll output the expected result ‘10′:

MacBook:reduce nicolas $ scalac Reducer.scala
MacBook:reduce nicolas $ scala Reducer
Result: 10

Now we can open the generated class files in JD. There are a couple of them (it’s interesting to take a look at all of them and figure out what they represent exactly), but in this case we need ‘Reducer$.class’, which contains the implementations of our public functions, including ‘reduce’.

Here’s the Java version of the ‘reduce’ function:

public <T, V> V reduce(Function2<V, T, V> fun, List<T> values, V initial)
{
  while (true)
  {
    if (values.isEmpty())
      return initial;
    Object next = fun.apply(initial, values.head());
    initial = next;
    values = values.tail();
  }
}

‘Function2′ is a built-in Scala type which represents a function taking 2 parameters. As you can see, this code does exactly the same as our Scala version and is most likely the way we’d write the code manually as well (the only thing I don’t get is why ‘next’ is an Object and not a ‘V’, I might figure that out later), but without forcing us to write the imperative code, whilst still producing bytecodes which will most likely show the best performance on the JVM (which currently has no tail recursion optimization support (although that might change one day)).

I like it

[update]
For reference, here’s a slightly more Scala-ish implementation of reduce, showing the same time performance characteristics during some basic profiling. I was not able to get JD nor jad to generate any usable decompiled code though:

def reduce[T, V](fun: (V, T) => V, values: List[T], initial: V): V = {
    values match {
        case List() => initial;
        case head :: tail => reduce(fun, tail, fun(initial, head))
    }
}

It uses Scala’s “List” pattern matching functionality.

Git has a feature to work with SVN repositories, git-svn, but that’s intended to check out existing code from SVN and work on it, not publishing an existing Git tree into a Subversion repository.

A first rather naive approach didn’t work out (as somewhat expected), but then I figured out how to achieve this anyway.

As a test, let’s first create an empty SVN repository and a Git repository with some commits:

$ svnadmin create repo
$ svn co file:///Users/nicolas/Temp/git_to_svn/repo svn_repo
Checked out revision 0.
$ cd svn_repo
$ svn mkdir trunk tags branches
A         trunk
A         tags
A         branches
$ svn commit -m "Create repository structure"
Adding         branches
Adding         tags
Adding         trunk

Committed revision 1.
$ cd ..

$ mkdir project; cd project
$ git init
Initialized empty Git repository in /Users/nicolas/Temp/git_to_svn/project/.git/
$ echo "foo" > test.txt; git add test.txt; git commit -m "Initial version"
master (root-commit) 88464cf] Initial version
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 test.txt
$ echo "bar" > test.txt; git commit test.txt -m "Second version"
master cb62866] Second version
 1 files changed, 1 insertions(+), 1 deletions(-)

We now can set up git-svn:

$ git svn init -s file:///Users/nicolas/Temp/git_to_svn/repo/
$ git svn fetch
r1 = 741ab63aea786882eafd38dc74369e651f554c9c (trunk)

Depending on the layout of your SVN project, you might need to drop the -s parameter and add -t, -T or -b flags, see the git-svn manpage.

A little naive we could try to push everything to the SVN repository now:

$ git svn dcommit
Unable to determine upstream SVN information from HEAD history.
Perhaps the repository is empty. at /opt/local/libexec/git-core/git-svn line 439.

This fails since the git svn command can’t figure out which commits to push: there’s no link between our original Git repository and the Subversion heads.

To fix this, we can use a Git graft to link them. We’ll tell Git the commit which created the SVN folder in which we want to store the project is the parent commit of the first commit in our Git repository:

$ git show-ref trunk
741ab63aea786882eafd38dc74369e651f554c9c refs/remotes/trunk
$ git log --pretty=oneline master | tail -n1
88464cfdf549a82b30ee7c52e53e2b310f0d9ec4 Initial version
$ echo "88464cfdf549a82b30ee7c52e53e2b310f0d9ec4 741ab63aea786882eafd38dc74369e651f554c9c" >> .git/info/grafts

If now you execute git log, you’ll see the “Create repository structure” SVN commit is displayed after our “Initial version” commit.

Pushing to SVN now works fine:

$ git svn dcommit
Committing to file:///Users/nicolas/Temp/git_to_svn/repo/trunk ...
	A	test.txt
Committed r2
	A	test.txt
r2 = 8c72757dd3a7d550ed8ef393bb74c0350d22dbac (trunk)
No changes between current HEAD and refs/remotes/trunk
Resetting to the latest refs/remotes/trunk
test.txt: locally modified
	M	test.txt
Committed r3
	M	test.txt
r3 = ca0fc06d477bcd4dd5c6f6d2ae6d94356b510280 (trunk)
No changes between current HEAD and refs/remotes/trunk
Resetting to the latest refs/remotes/trunk

All set

Enlightened I/O is a specialized virtualization-aware implementation of high level communication protocols (such as SCSI) that utilize the VMBus directly, bypassing any device emulation layer. This makes the communication more efficient but requires an enlightened guest that is hypervisor and VMBus aware.

The drivers seem to be developed by Novell, so I guess the Boycott Novell guys will have some more coverage^Wrants soon (Update: can’t find the reference on this anymore, so this might be a false statement, sorry. Thanks for pointing out RubenV)

Interesting times on the virtualization front… Although I for one do not plan to replace Xen, xVM or VirtualBox anytime soon.

Sources:

• Microsoft Press Release
• Blog of the 451 Group
• LKML

On a side note: Red Hat entered the Standard & Poor’s 500 index, which might show Linux is gaining more interest from enterprises and investors.

Hey,

As discussed before, this is not a fair comparison, since the non-recursive version is much ’smarter’ than the recursive one: it calculates values and will never recalculates them, whilst the recursive version calculates everything over and over again.

Adding some simple memoization helps a lot. First, my testing code:

Here are the benchmarks on my MacBook Pro Intel Core2Duo 2.33GHz with 3GB RAM (running quite a lot of applications). Do note the ‘dumb’ version calculates fib(35), whilst the slightly optimized versions, which still use recursion but much less recursive calls (as they should) or your second version calculate fib(150).

Using MacOS X 10.5.6 stock CPython 2.5.1:

MacBook:Projects nicolas $ python -V
Python 2.5.1

MacBook:Projects nicolas $ python fib.py 35 150
fib(35) = 9227465
Calculation took 12.8542108536 seconds

Calculating the amount of recursive calls to calculate fib(35)
Calculating fib(35) = 9227465 took 29860703 calls

fib2(150) = 9969216677189303386214405760200
Calculation took 0.00020694732666 seconds

memoize_dict(fib)(150) = 9969216677189303386214405760200
Calculation took 0.00141310691833 seconds

memoize_constant_list(151, fib)(150) = 9969216677189303386214405760200
Calculation took 0.000310182571411 seconds

Overall it looks like fib2 and memoize_constant_list perform fairly similar, I guess function call overhead and list.append have a similar influence on performance in this case.

Using Jython 2.5.0 from the binary distribution on the Java HotSpot 64bit Server VM as shipped for OS X 10.5.6:

MacBook:Projects nicolas $ ./Jython/jython2.5.0/jython -V
Jython 2.5.0

MacBook:Projects nicolas $ ./Jython/jython2.5.0/jython fib.py 35 150
fib(35) = 9227465
Calculation took 12.5539999008 seconds

Calculating the amount of recursive calls to calculate fib(35)
Calculating fib(35) = 9227465 took 29860703 calls

fib2(150) = 9969216677189303386214405760200
Calculation took 0.0519998073578 seconds

memoize_dict(fib)(150) = 9969216677189303386214405760200
Calculation took 0.00399994850159 seconds

memoize_constant_list(151, fib)(150) = 9969216677189303386214405760200
Calculation took 0.00300002098083 seconds

The ‘dumb’ fib implementation performs similar in both CPython and Jython. Jython performs significantly less good on the other implementations though, but maybe todays news could help here, not sure how much locking on dict and list access Jython introduces.

Finally, using Unladen Swallow 2009Q2, self-compiled from SVN on the same system, using standard settings:

MacBook:Projects nicolas $ ./unladen-swallow/unladen-2009Q2-inst/bin/python -V
Python 2.6.1

MacBook:Projects nicolas $ ./unladen-swallow/unladen-2009Q2-inst/bin/python fib.py 35 150
fib(35) = 9227465
Calculation took 12.2675719261 seconds

Calculating the amount of recursive calls to calculate fib(35)
Calculating fib(35) = 9227465 took 29860703 calls

fib2(150) = 9969216677189303386214405760200
Calculation took 0.000118970870972 seconds

memoize_dict(fib)(150) = 9969216677189303386214405760200
Calculation took 0.000972986221313 seconds

memoize_constant_list(151, fib)(150) = 9969216677189303386214405760200
Calculation took 0.00036096572876 seconds

which is similar to, slighly better or slightly worse than the CPython run, and when enforcing JIT (which introduces a significant startup time, which is not measured here):

MacBook:Projects nicolas $ ./unladen-swallow/unladen-2009Q2-inst/bin/python -j always fib.py 35 150
fib(35) = 9227465
Calculation took 14.6129109859 seconds

Calculating the amount of recursive calls to calculate fib(35)
Calculating fib(35) = 9227465 took 29860703 calls

fib2(150) = 9969216677189303386214405760200
Calculation took 0.0432291030884 seconds

memoize_dict(fib)(150) = 9969216677189303386214405760200
Calculation took 0.0363459587097 seconds

memoize_constant_list(151, fib)(150) = 9969216677189303386214405760200
Calculation took 0.0335609912872 seconds

which, to my surprise, performs pretty worse than the default settings.

Overall: your first implementation performs tons and tons of function calls, whilst the second one, which resembles memoize_list_fib in my code (which is recursive), performs significantly less function calls and in the end memoize_list_fib performs almost as good as your second version (it performs +- the same number of function calls as the number of times you’re going through your loop).

So whilst I do agree function calls in Python are reasonably slow compared to plain C function calls (which is just a jmp, no frame handling etc. etc. required), your comparison between your recursive and non-recursive implementation is completely unfair, and even if calculating fib(35) takes several seconds, consider you’re doing a pretty impressive 29860703 function calls to perform the calculation.

Time to get some sleep.

Since I wanted to learn some language not resembling any other I already know (even a little), I decided some hours ago to start digging into Clojure, which is a LISP dialect running on the JVM using STM (Software Transactional Memory) and created with concurrency in mind. Check the website for more information.

After some hacking I got a first ‘application’ running. Since recently there’s been some little meme at work regarding echo servers, I decided to write a very basic line-oriented echo server in Clojure.

The result is a server using one thread per connection which just sends back lines to a connected client as-is. Nothing fancy, but might be a useful start for developing basic network applications using Clojure.

Enjoy!

The most common approach is something like (pseudocode):

temp = from
from = to
to = temp

In Python there’s an easier approach though which seems to be unknown to several developers. It uses the multi-assignment/expansion syntax:

from, to = to, from

Basically, the tuple on the right (to, from) is constructed, then expanded to locals ‘from’ and ‘to’.

Just a hint It’s a pretty elegant line of code IMHO.