As mentioned before, some days ago I received a copy of a recent book from Packt Publishing titled “Python Testing – Beginner’s Guide” by Daniel Arbuckle. I read the whole book (it’s not huge, around 220 pages), and wrote a review, as requested by Packt.

The book targets people who know Python (it doesn’t contain a language introduction chapter or something alike, which would be rather pointless anyway), and want to start testing the code they write. Even though the author starts by explaining basic tools like doctests and the unittest framework contained in the Python standard library, it could be a useful read even if you used these tools before, e.g. when the Mock library is explained, or in the chapter on web application testing using Twill.

The text is easy to read, and contains both hands-on code examples, explanations as well as tasks for the reader and quiz questions. I did not audit all code for correctness (although in my opinion some more time should have been invested here before the book was publishing: some code samples contain errors, even invalid syntax (p45: “self.integrated_error +q= err * delta“), which is not what I expect in a book about code testing), nor all quizes. These could’ve used some more care as well, e.g. on p94 one can read

What is the unittest equivalent of this doctest?

>>> try:
...     int('123')
... except ValueError:
...     pass
... else:
...     print 'Expected exception was not raised'

I was puzzled by this, since as far as I could remember, int(’123′) works just fine, and I didn’t have a computer at hand to check. Checked now, and it works as I expected, so maybe I’m missing something here? The solution found in the back of the book is a literal unittest-port of the above doctest, and should fail, if I’m not mistaken:

>>> def test_exceptions(TestCase):
...     def test_ValueError(self):
...         self.assertRaises(ValueError, int, '123')

This example also shows one more negative point of the book, IMHO: the code samples don’t follow PEP-8 (or similar) capitalization, which makes code rather hard to read sometimes.

The solutions for the last quiz questions are missing as well, and accidently I did want to read those.

Don’t be mistaken though: these issues don’t reduce the overall value of the book, it’s certainly worth your time, as long as you keep in mind not to be too confused by the mistakes as shown above.

Topic overview

The book starts with a short overview of types of testing, including unit, integration and system testing, and why testing is worth the effort. This is a very short overview of 3 pages.

Starting from chapter 2, the doctest system is introduced. I think it’s an interesting approach to start with doctest instead of using unittest, which is modeled after the more ‘standard’ xUnit packages. Doctests are useful during specification writing as well, which is in most project the first stage, before any unittestable code is written. The chapter also introduces an overview of the doctest directives, which was useful to read.

In chapter 3 gives an example of the development of a small project, and all stages involved, including how doctests fit in every stage.

Maybe a sample of Sphinx and its doctest integration would have been a nice addition to one of the previous chapters, since the book introduced doctest as part of stand-alone text files, not as part of code docstrings (although it does talk about those as well). When writing documentation in plain text files, Sphinx is certainly the way to go, and its doctest plugin is a useful extra.

Starting in chapter 4, the Python ‘mocking‘ library is introduced. The chapter itself is a rather good introduction to mock-based testing, but I don’t think mocks should be used in doctests, which should be rather small, examplish snippets. Mock definitions don’t belong there, IMO. This chapter also shows some lack of pre-publishing reviews in a copy-paste error, in the block explaining how to install mocker on page 62, telling from now on Nose is ready to be used.

Chapter 5, which you can read here introduces the unittest framework, its assertion methods, fixtures and mocking integration.

In chapter 6 ‘nose‘ is introduced, a tool to find and run tests in a project. I use nose myself in almost every project, and it’s certainly a good choice. The chapter gives a pretty good overview of the useful features nose provides. It does contain a strange example of module-level setup and teardown methods, whilst IMHO subclassing TestCase would be more suited (and more portable).

Chapter 7 implements a complete project from specification to implementation and maintenance. Useful to read, but I think the chapter contains too much code, and it’s repeated too often.

Chapter 8 introduces web application testing using Twill, which I never used before (nor did I ever test a web application before). Useful to read, but Twill might be a strange choice, since there have been no releases since end 2007… Selenium might have been a better choice?

A large part of the chapter is dedicated to list all possible Twill commands as well, which I think is a waste of space, this can be easily found in the Twill language reference.

Chapter 9 introduces integration and system testing. Interesting to read, the diagram-drawing method used is certainly useful, but it also contains too much code listings.

Finally, chapter 10 gives a short overview of some other testing tools. First coverage.py is explained, which is certainly useful. Then integration of test execution with version control systems is explained. I think this is certainly useful, but not at this level of detail. Setting up a Subversion repository is not exactly what I expect here, especially not when non-anonymous, password-based authentication over svn:// is used (which is a method which should be avoided, AFAIK).
Finally, continuous integration using Buildbot is tackled. No comments here, although I tend to use Hudson myself

Final words

Is this book worth your time and money? If you’re into Python and you don’t have lots of experience with testing Python code, it certainly is. Even if you wrote tests using unittest or doctests before, you’ll most likely learn some new things, like using mocks.

I’m glad Packt gave me the opportunity to read and review the book. I’d advise them to put some more effort in pre-publishing reviews for future titles, but the overall quality of the non-code content was certainly OK, and I hope lots of readers will enjoy and learn from this book.