Monthly Archives: November 2011

Converting Django models into Graphviz DOT files

This was really easy and the result is really good-lookin.

https://code.djangoproject.com/wiki/DjangoGraphviz

The article is almost accurate. Some really accurate steps:

cd /path/to/your/django/project
git clone https://github.com/django-extensions/django-extensions.git
cp django-extensions/django_extensions/management/modelviz.py . 
ln -s $PWD/django-extensions/django_extensions/templates/django_extensions templates/ 
python modelviz.py name_of_django_app > app.dot
dot app.dot -Tpng -o app.png

(And of course you need http://www.graphviz.org/ first.)

UPDATE: ok the above is really the ugly way of doing it. The nice way is:

# Install django-extensions 
pip install django-extensions

# In settings.py add django_extensions INSTALLED_APPS

# Generate graph
python manage.py graph_models name_of_django_app > app.dot
dot app.dot -Tpng -o app.png

A quite painful MySQL server migration with Japanese characters and utf-8

What was supposed to be a walk in the park, turned out to be quite painful and weirdo experience with mysql and Japanese text entries. The database in question is a mantis (bug tracker) database with a mix of English and Japanese entries. The database and all tables had utf8 character set, both at the source and destination sites. No matter what method I tried to dump and load, the Japanese characters became garbled.

I tried these for dumping:

mysqldump bugtracker > bugtracker.dump
mysqldump bugtracker -r bugtracker.dump
mysqldump bugtracker --default-character-set=utf8 -r bugtracker.dump

And these for loading:

mysql < bugtracker.dump
mysql --default-character-set=utf8 < bugtracker.dump
set names utf8; source bugtracker.dump

Any combination of the above dumping and loading methods resulted in garbled characters after loading.

At this point, I wanted to pinpoint whether the data gets broken during the dump or during the load. To confirm the correctness of the dump, I created an entry with both English and Japanese text in it, so that I can grep with en English pattern and see some Japanese text that should not be garbled. However, modern versions of mysql dump data as so-called extended inserts, which results in extremely long lines (spanning multiple screens), making the grep output unreadable. Luckily this behavior can be changed with a flag, so that entries are dumped as a single insert command per entry:

mysqldump bugtracker --skip-extended-insert --default-character-set=latin1 | grep mantis_bug_text.*English

This way I found the right way to dump, and to my surprise I had to use latin1 character set. But my troubles were not over yet. Although the dump was supposed to be correct, I still could not load it without breaking the Japanese. By chance, I spotted a suspicious looking snippet in the dump file:

SET NAMES latin1

Changing latin1 to utf8 and then loading it with any of the above methods finally did the trick.

In conclusion, not only I had to force latin1 character set when dumping, I also had to hand-edit the dump file to make it loadable. Sounds pretty crazy to me. I don’t understand the logic behind all this, but this is what worked.

My first Android app, a very simple tool to debug Bluetooth sensors, is now available on Android Market

BluetoothViewer is a simple Android application that can connect to any Bluetooth device and display incoming raw data, and can also send data from Android to the Bluetooth device. The purpose of the application is to confirm that a Bluetooth device is online and discoverable, accepting incoming connections, and to observe the raw data messages coming from the device.

We use this as a debugging tool for portable environmental sensors that can transmit data over Bluetooth. Naturally it can be useful to debug any kind of Bluetooth device.

The application is FREE and open-source, contributions are very much welcome. For more details see the project homepage:

http://blog.ecomobilecitizen.com/contact/bluetoothviewer/

Available in Android Market

By the way, publishing on Android Market was really a smooth ride. After publishing, the app’s page on the Market website was ready in a couple of minutes, and after a few hours I was able to find it using the Market application on my Android phone. Totally awesome!

Severed Fifth is totally awesome hard pounding metal, and that for free

Severed Fifth is a metal band (?), with so far two albums that are self-produced, self-released, written, performed and recorded by Jono Bacon, community manager of Ubuntu, author of the awesome (and free) book The Art of Community. I really didn’t expect much from a music made by just one guy, but man, this is totally awesome hard pounding metal! And the guy is releasing the albums completely for free (under Creative Commons Attribution ShareAlike license). That’s right, it’s just out there for everybody to enjoy, so if you like some good metal, listen to it at http://www.severedfifth.com/releases/ (my favourite is The Lake, when the heavy rhythmic pounding kicks in at about halfway)

Yes, this topic is right at home in a hacking journal 😉