Ikke's blog » couchdb http://eikke.com 'cause this is what I do Sun, 13 Feb 2011 14:58:55 +0000 en-US hourly 1 http://wordpress.org/?v=3.4.1 Filesystem issues and django-couchdb work http://eikke.com/filesystem-issues-and-django-couchdb-work/ http://eikke.com/filesystem-issues-and-django-couchdb-work/#comments Sun, 30 Dec 2007 01:47:22 +0000 Nicolas http://eikke.com/filesystem-issues-and-django-couchdb-work/ Last night, when shutting down my laptop (which had been up for quite a long time because of suspend/resume niceness), it crashed. I don’t know what exactly happened: pressed the GNOME’s logout button, applications were closed, until only my background was visible, then the system locked up, so I suspect my X server (some part of it, GPU driver (fglrx) might be the bad guy). I was able to sysrq s u o, so I thought everything would be relatively fine.

This morning I powered on my system, and while booting, fsck of some partitions was taking a rather long time. It’s pretty normal fsck was taking somewhat longer, but not thát long… I’m using JFS on most logical volumes.

When the consistency check of my /home partition was done, a whole load of invalid files was displayed and later on moved to lost+found: 34068 files. Once booted, I scanned my filesystems again, rebooted, logged in, started X. Everything started fine, until I launched Evolution: it presented my the ‘initial run’ wizard. Other issues (on first sight): all Firefox cookies were gone, and Pidgin’s blist.xml was corrupted. When using my old computer (which had frequent lockups on heavy IO usage) these last 2 issues happened a lot too, which is highly annoying, especially the blist.xml thing as I can’t see any reason to keep this file opened for long periods?

Luckily I was able to get my Evolution up and running again by restoring it’s GConf settings and ~/.evolution using some old backup (15/10/07). I guess I should backup more regularly… Next to this I hope I won’t find any other corrupted files, so the ones in lost+found are just Evolution email files and Firefox caches.

Anyway, here’s a screenshot displaying some of the initial and hackish work I’ve done this evening on integrating Django and CouchDB as I wrote about yesterday:

Django and CouchDB first shot

As you can see, currently I’m able to edit fields of an object. There’s one major condition: an object with the given ID should already exist in the ‘database’ which makes the current code rather useless, but hey ;-) I’ll add object creation functionality later tonight or tomorrow.

Current code is very expensive too, doing way too many queries to CouchDB, mainly in client.py. This most certainly needs work.

Upgraded my WordPress installation to the latest release, 2.3.2, in about 5 seconds. Got to love svn switch (although maybe I should start using git-svn for this installation too and git-pull the release branch in my local copy).

]]>
http://eikke.com/filesystem-issues-and-django-couchdb-work/feed/ 9
CouchDB with Python http://eikke.com/couchdb-with-python/ http://eikke.com/couchdb-with-python/#comments Sat, 29 Dec 2007 01:25:52 +0000 Nicolas http://eikke.com/couchdb-with-python/ Today I’ve been investigating CouchDB a little better (only heard some rumors about it before). It’s actually a pretty nice technology which can, in some places, be pretty useful… I tend to compare it to caching serialized PHP associative arrays or Python dict’s in a Memcached server using some specific prefixes, except it’s not really memory-based (it’s persistent), you get a complete query interface (views), there’s dataset versioning support (!), etc. While writing this I start to wonder what similarities I ever saw between CouchDB and a Python pickled dict in Memcached…

Anyway, one use case I saw was site user profiles: profile data is most of the time not relational at all, so why store it in a relational database, which makes it sometimes rather hard to add extra profile information fields, unless you use some dirty ‘save serialized form’ trick, which renders your data unqueryable? Storing profile information (using eg. a user’s primary email address or login name as key for the user profile document) in CouchDB allows you to extend the profile “schema” easily: just add a field to your profile editting form, make sure it’s processed server-side an stored in the profile document, and add some extra code to your profile rendering template so the extra field get displayed too. No need to alter SQL tables at all!

As in my last site project I also have some sort of user profiles, I was thinking about using CouchDB for storage of these objects. As the site is written using Django, it would be nice to be able to define a standard Django model for the profile, which would be stored in CouchDB, not insome SQL server. This way you can still enjoy newforms goodness, among others.

So I started some new project, called django-couchdb, which should in time provide a model base class (similar to django.db.models.Model), corresponding managers to query the data, and so on. I don’t know (yet) whether all this is possible to achieve, anyway, I started by creating a very basic Python class which allows you to access a CouchDB server in a very Pythonic way: using dicts. A Server is a dict consisting of Databases, a Database is a dict of Documents. All this implemented thanks to the goodness of the DictMixin base class.

The client is not finished yet, at least 3 TODO items are on my list:

  • Error/Exception handling
  • Revision handling
  • View support

Currently there is no support for any of these. Views should be easy to add, error handling a little harder. I think revision handling is the hardest part, escpecially on figuring out how to provide this functionality in a Pythonic manner.

You can find the current code in this Git repository. Patches or external branches are very welcome!

By the way: the website I referred to before has been launched. It’s only of any use (well, maybe) for dutch-speaking users though. You can visit it here. Yes, the template will change.

]]>
http://eikke.com/couchdb-with-python/feed/ 5