Setting up Django environment in CentOS 5

I am trying to setup Django v1.2.3 to run on our server which is CentOS 5.2.

Though Django support python2.4 bundled with this OS, I decide to get it running on Python2.6, which makes me feel not so out. In the beginning I was thinking it could be a task of 10 mins, and not expecting so many unexpected things happened:

The first issue is replacing python2.4 with python2.6 breaks the yum toolkit on CentOS 5.2. Shame for that. Fortunately some other guys has met the problem already and there are lot's of blogs and question/answers addressing this issue. I follow this one: how to install python2.6 on centos5 without breaking yum. It almost works, just a minor problem: yum report that libffi-3.0 not found while installing python2.6. So google again, and there it is:
wget ftp://ftp.pbone.net/mirror/centos.karan.org/el5/extras/testing/i386/RPMS/libffi-3.0.5-1.el5.kb.i386.rpm
After installing this rpm package, I can get python2.6 installed.

Now download Django-1.2.3.tar.gz from official site, unpack, and run python26 setup.py install. Oops, wait, there is problem:

invalid Python installation: unable to open usr/lib/python2.6/config/Makefile

Okay, google again, and someone said it's due to you have not installed python-dev package. Let's try yum install python26-dev and then run python26 setup.py install again. Bingo!

Now that we have the Django installed at /usr/lib/python2.6/site-packages. Let's give it a try:

1. update /etc/bashrc, add "export DJANGO_HOME=/usr/lib/python2.6/site-packages/django"
2. . /etc/bashrc && chmod a+x $DJANGO_HOME/bin/*.py
3. ln -s $DJANGO_HOME/bin/django-admin.py /usr/bin/django-admin.py
4. cd /tmp
5. django-admin startproject mysite

Oops, error again:

Traceback (most recent call last):
  File "/usr/bin/django-admin.py", line 2, in ?
    from django.core import management
ImportError: No module named django.core

Damn it, I forgot django is installed to python26. Okay, let me do something to hack it:

# cd $DJANGO_HOME/bin
# find . -name '*.py' | sed -i 's/env python/env python26/'

now run django-admin startproject mysite works. and let me continue:
6. cd mysite && python26 manage.py runserver

OMG, it breaks again, all the files generated by django-admin use python as default env, not python26. I am getting tired of adding my hacks to Django. I need to figure out some other ways.

Yes, python support virtualenv, that's exactly what I need! Follow the process described in this blog, I can continue again:

# wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.5.1.tar.gz
# tar xzf virtualenv-1.5.1.tar.gz
# cd virtualenv-1.5.1
# su greenl
$ python26 virtualenv.py ~/opt/local
$ cd ~
$ wget http://www.doughellmann.com/downloads/virtualenvwrapper-1.23.tar.gz
$ tar xzf virtualenvwrapper-1.23.tar.gz
$ cd virtualenvwrapper-1.23
$ cp virtualenvwrapper_bashrc ~/bin
$ cd ~
$ mkdir .virtualenvs
$ mkvirtualenv django
$ workon django
$ easy_install django
$ cd /tmp
$ sudo rm -rf mysite
$ django-admin.py startproject mysite
$ cd mysite
$ python manage.py runserver &
$ lynx localhost:8000

Works perfect!

In conclusion:
1. You can install python2.6 on CentOS 5 without breaking yum
2. You can install django latest version (currently v1.2.3) on python2.6 site-packages
3. You need to install virtualenv in order to get django run on python2.6 transparently
4. You may want to install virtualenvwrapper in order to make your life easier with virtualenv.
5. After you have done all these, you have an nice env for your django journey!

Routing problem with GreenScript configurator

In the morning when I play with GreenScript configurator to verify it's compatibility with Play-1.0. I found all these mapping entries do not work anymore:

• http://localhost:9000/gs/
• http://localhost:9000/gs/conf
• http://localhost:9000/gs/configure
• http://localhost:9000/gs/setting

However, this link works: http://localhost:9000/gs (without "/" suffix!)

Later I realized that I have an entry in the routing table of the demo app: GET /gs/ staticDir:public/gs, for shortcut of URL to compressed/minimized files.

Yes, this is the problem. You can't make both static file and controller mapping to the same url path!

Play python interface compatibility

it looks like the new version of GreenScript is quite welcome in the community.

Tom is obviously one of them who love this plugin. He report a bug very promptly. After installed v1.1, as long as he try to invoke any play command like play eclipsify or play greenscript:cp .... it will end up with the following error:

=====
C:\code\phase3>play eclipsify
~        _            _
~  _ __ | | __ _ _  _| |
~ | '_ \| |/ _' | || |_|
~ |  __/|_|\____|\__ (_)
~ |_|            |__/
~
~ play! 1.0.3.2, http://www.playframework.org
~
Traceback (most recent call last):
 File "C:\play-1.0.3.2\play", line 1459, in 
   execfile(commands)
 File "C:\play-1.0.3.2\modules\greenscript-1.1\commands.py", line 4,
in 
   from play.utils import *
ImportError: No module named play.utils
====

This is my fault. I didn't expect the play command interface has been changed since play-1.1. Here is the solution:

try:
    # play_command is defined in Play-1.0 only
    play_command
except NameError:
    play_command = None
if play_command is None:
    # ~~~~~~~~~~~~~~~~~~~~~ running on v1.1
    # go v1.1 logic
else:
    # ~~~~~~~~~~~~~~~~~~~~~ running on v1.0
    # go v1.0 logic

Play-GreenScript v1.1 Released

This is a version with huge improvements:

• Many bug fixes
• Completely new Plugin Configurator
• Add Command to enable user copy module tags/templates to user app directory
• More clear configuration settings
• Even more simplified tag syntax
• Support zero configuration
• Document improvement

Comparison to v1.0

1. Configuration:

   • v1.0 - The dependency configuration file: "greenscript.conf" is mandatory
   • v1.1 - Zero configuration

2. Dynamic configuration

   • v1.0 - A simple window with a <textarea> tag. You are on your own to write the content in the textarea. Error prone: typos, unattended updates to dependencies
   • v1.1 - A sophisticated web interface help you clearly view the status/configuration of GreenScript plugin and prevent you from making mistakes when you try to turn on/off minimizing/compressing/cache.

3. Tags

   • v1.0 - #{greenscript.css import:['css1', 'css2']/}
   • v1.1 - #{greenscript.css 'css1 css2'/}

     For existing greenscript users, the old syntax also works, but you are encourage to move to the new simple version.

4. Commands

   • v1.0 - no command support
   • v1.1 - play greenscript:cp -a --tag=xx | -t --template=xx

For more information please go to GreenScript home.

PS: Forgot the secret weapon: with "play greenscript:cp -a .", you can use tags in this way:

#{js 'js1 js2 ...' /}
#{css 'css1, css2, ...' /}

How can you expect anything simpler than this?

int is not Assignable to Integer!

I've met a strange issue with play-morphia: Model.findById(theId) method failed while Model.filter("_id", theId) method works perfect. Digging into the code stack for while, I found a surprising behavior of java.lang.Class.isAssignableFrom(Class cls) method: If you pass a type of primitive type int/long etc (get from reflection of course) to it's corresponding object type class: Integer.class/Long.class, the method will return false! So while the following line of code compiles and runs perfect (because of autoboxing), the primitive type is NOT assignable to it's peer object type!

Integer i0 = 5; int i1 = i0;

Now the solution:

Object theId = play.data.binding.Binder.directBind(id.toString(), Model.Manager.factoryFor(clazz).keyType());

As you might noticed, the method is fairly poor in terms of performance because of lot of conversion and reflection operations. And here comes my suggestion to play-morphia user who use @Id to annotate user defined ID fields:

always override public static findById(Object id) method, that will do you big favor. The default findById method is provided in case you are too lazy to implement your own, and the result is it could be pretty slow.

For model classes relies on system generated ID, e.g. you have no field annotated with @Id, you don't need to worry about this. The enhancer will provide faster version of findById() method implementation for your model.

Unlimited unbinding loop when using ObjectId

Just got the following error in my play templates:

RuntimeException occured : java.lang.StackOverflowError

play.exceptions.JavaExecutionException: java.lang.StackOverflowError
        at play.mvc.ActionInvoker.invoke(ActionInvoker.java:285)
        at Invocation.HTTP Request(Play!)
Caused by: java.lang.RuntimeException: java.lang.StackOverflowError
        at play.templates.BaseTemplate.throwException(BaseTemplate.java:91)
        at play.templates.GroovyTemplate.render(GroovyTemplate.java:226)
        at play.mvc.results.RenderTemplate.(RenderTemplate.java:23)
        at play.mvc.Controller.renderTemplate(Controller.java:568)
        at play.mvc.Controller.renderTemplate(Controller.java:553)
        at play.mvc.Controller.render(Controller.java:597)
        at controllers.CKBoxes.index(CKBoxes.java:31)
        at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:408)

        at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:403)

        at play.mvc.ActionInvoker.invoke(ActionInvoker.java:176)
        ... 1 more
Caused by: java.lang.StackOverflowError
        at java.lang.Class.privateGetDeclaredFields(Class.java:2278)
        at java.lang.Class.getDeclaredFields(Class.java:1743)
        at play.data.binding.Unbinder.unBind(Unbinder.java:51)
        at play.data.binding.Unbinder.unBind(Unbinder.java:57)
        at play.data.binding.Unbinder.unBind(Unbinder.java:57)
        ...

The problem happens when Unbinder tries to unbind a static field called _nextInc of type org.bson.types.ObjectId:

private static AtomicInteger _nextInc

In AtomicInteger there is a reference to sun.misc.Unsafe which in turn refer to a field with the same type: sun.misc.Unsafe

So what should be the solution? Here are what comes in my mind at the moment:
1. Unbinder do not try to unbind a static field ?
2. Some mechanism should be implemented in Unbinder to prevent endless unbinding loop like this case

As a temporary workaround I add ".toString()" to explicitly get String represent of ObjectId object:

original: #{a @{edit(_.id)}edit{/#}
workaround: #{a @{edit(_.id.toString())}edit{/#}

Play-Menu

Play-Menu published at github. Features include:

* Define menu structure with yaml data file
* Provide easy to use menu tag to insert navigation menu to your web page
* Support current menu
* Support hierarchical structure
* Support customer defined menu model (for those GAE app developers)

Guice your Play Model class?

This blog discuss a DI implementation pattern on Play model class. However it might also be helpful for developer using other MVC frameworks.

As stated in this page, Model should NOT be kept as a set of simple Java Beans and leave the application logic back into a “service” layer which operates on the model objects.

So there are often good reasons to call to some services utilities/executing tasks from within your model class in order to complete an application logic inside the model class. Here is one example in my recent development work. When users login to the web site, they needs to be able to turn on parent mode for some time, say 1 hour. The parent mode setting will automatically turn off as timeout.

The first version of turnOnParentMode() is like

    public void turnOnParentMode(int timeout) {
     if (parentMode) return;
     if (null == ip) throw new IllegalStateException("Cannot set parent mode when IP address is null");
     parentMode = true;
     TimeKeeper.register(username, timeout);
    }

As you can see, TimeKeeper::register is implemented as a static class member function so that we can easily call it from anywhere, like from within a Model class. There is nothing wrong with that approach, unless later I decide to unit test this method with JMock. The immediate question I need to answer is how can I mock TimeKeeper::register as it is a static class member?

Vampires (if you don't know who are vampires, take a look at this blog) from the underworld told me that DJ is the answer to questions like the above one. In the beginning they advocate a beast named Spring, but recently they look like be more addicted to a monster called Guice. Since I decide to be part of them I turned to Guice to look for my answer.

Everything is good as I quickly found Guice is supported by Play with play-guice plugin. Yes so far so good until I read this statement: "... inject into your application’s Controller, Job and Mail classes (based on @javax.inject.Inject)". What? injection can only be operate on Controller, Job and Mail classes? Why my poor model class is not in the list?

Alright, let's do something to get it work! First I will change the implementation of turnOnParentMode() to:

    @javax.inject.Inject private static timer;
    public void turnOnParentMode(int timeout) {
     if (parentMode) return;
     if (null == ip) throw new IllegalStateException("Cannot set parent mode when IP address is null");
     parentMode = true;
     timer.register(username, timeout);
    }

And of couse, TimeKeeper::register method will become a non-static instance member. Now we are ready for injection.

Since GuicePlugin won't inject Model class, specifically Playframework won't inject Model class, we need to hack the code of GuicePlugin by adding a new method:

    private void injectModelClasses(BeanSource source) {
        List classes = Play.classloader.getAssignableClasses(Model.class);
        for(Class clazz : classes) {
            for(Field field : clazz.getDeclaredFields()) {
                if(Modifier.isStatic(field.getModifiers()) && field.isAnnotationPresent(Inject.class)) {
                    Class type = field.getType();
                    field.setAccessible(true);
                    try {
                        field.set(null, source.getBeanOfType(type));
                    } catch(RuntimeException e) {
                        throw e;
                    } catch(Exception e) {
                        throw new RuntimeException(e);
                    }
                }
            }
        }        
    }

And add the following line to the end of onApplicationStart() method:

injectModelClasses(this);

Okay, let's compile GuicePlugin by running ant, and then "play run", everything is okay, TimeKeeper debug info shows out, looks like our code is good, and more importantly we don't rely on static TimeKeeper::register method, we have some flexibility to inject different implementation of TimeKeeper instance to our model class.

But wait, we haven't solve the problem of unit test turnOnParentMode() method yet, how to replace the injection with a mock object of TimeKeeper? I will answer this question in next blog.

BTW, a suggestion to Play team:

Add a new interface say: InjectSupport, and have ControllerSupport, Mail and Job class to extend/implement the InjectSupport tag interface, and change the code of Injector::inject to:

    public static void inject(BeanSource source) {
        //List classes = Play.classloader.getAssignableClasses(ControllerSupport.class);
        //classes.addAll(Play.classloader.getAssignableClasses(Mailer.class));
        //classes.addAll(Play.classloader.getAssignableClasses(Job.class));
        List classes = Play.classloader.getAssignableClasses(InjectSupport.class)
        for(Class clazz : classes) {
            for(Field field : clazz.getDeclaredFields()) {
                if(Modifier.isStatic(field.getModifiers()) && field.isAnnotationPresent(Inject.class)) {
                    Class type = field.getType();
                    field.setAccessible(true);
                    try {
                        field.set(null, source.getBeanOfType(type));
                    } catch(RuntimeException e) {
                        throw e;
                    } catch(Exception e) {
                        throw new RuntimeException(e);
                    }
                }
            }
        }
    }

This way we are not limited DI to only Controller, Mail and Job classes, we add the flexibility to enable developer to inject any class he/she want using play-guice plugin.

Strange java.lang.NoClassDefFoundError

In the beginning I was thinking this is a ClassNotFoundException, and spend many time on looking into my runtime classpath settings, compiling from outside Eclipse, but finally I realized it is not. It's some bug in my code. Here is the story:

The program is trying to decode UDP packets. I try to ping the server by simply echo some string to ncat, and the server failed immediately with a whole bunch of stack trace, and it always report java.lang.NoClassDefFoundError: Could not initialize class radiustool.protocol.Packet$Code, actually almost always report the error upon any packets received, except the first time. Here is the stacktrace of the first time:

java.lang.ExceptionInInitializerError
        at radiustool.protocol.Packet$Codec.decode(Packet.java:429)
        at radiustool.server.PacketDecoder.decode(PacketDecoder.java:21)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
        at org.jboss.netty.channel.socket.nio.NioDatagramWorker.read(NioDatagramWorker.java:419)
        at org.jboss.netty.channel.socket.nio.NioDatagramWorker.processSelectedKeys(NioDatagramWorker.java:3r8)
        at org.jboss.netty.channel.socket.nio.NioDatagramWorker.run(NioDatagramWorker.java:261)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: ""
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
        at java.lang.Integer.parseInt(Integer.java:470)
        at java.lang.Integer.parseInt(Integer.java:499)
        at radiustool.protocol.Attribute$Type.parseInt(Attribute.java:118)
        at radiustool.protocol.Packet$Code$1.setValidators(Packet.java:31)
        at radiustool.protocol.Packet$Code.(Packet.java:201)
        at radiustool.protocol.Packet$Code.(Packet.java:193)
        at radiustool.protocol.Packet$Code$1.(Packet.java:22)
        at radiustool.protocol.Packet$Code.(Packet.java:22)
        ... 11 more

So here we have it, one stupid non-needed line in Attribute.java is the root cause. But wait, there are something wrong some where, isn't it? Why an uncaptured RuntimeException cause a NoClassDefFoundError? Can't we throw out exception in a constructor? Okay, let's do a simple test:

public class T1 {

 private static class T2 {
  public T2(int i) {
   if (i%2 == 0) throw new RuntimeException();
   System.out.println("T2 initialized with: " + i);
  }  
 }
 
 private static enum T3 {
  A(0), B(1);
  
  T3(int i) {
   if (i % 2 == 0) throw new RuntimeException();
   System.out.println("T3 initialized with: " + i);
  }
 }
    
    public static void main(String[] args) {
  try {
   T2 t2 = new T2(0);
  } catch (Exception e) {
   e.printStackTrace();
  }

  try {
   T2 t2 = new T2(1);
  } catch (Exception e) {
   e.printStackTrace();
  }
  
  try {
   T3 t3 = T3.A;
  } catch (Throwable e) {
   e.printStackTrace();
  }
  
  try {
   T3 t3 = T3.B;
  } catch (Throwable e) {
   e.printStackTrace();
  }
    }
    
}

And here is the output:

java.lang.RuntimeException
        at T1$T2.(T1.java:7)
        at T1.main(T1.java:23)
T2 initialized with: 1
java.lang.ExceptionInInitializerError
        at T1.main(T1.java:35)
Caused by: java.lang.RuntimeException
        at T1$T3.(T1.java:16)
        at T1$T3.(T1.java:13)
        ... 1 more
java.lang.NoClassDefFoundError: Could not initialize class T1$T3
        at T1.main(T1.java:41)

So now everything is clear. it's okay for your to throw runtime exception in the constructor of a normal class, like T2 in the above code, there won't be NoClassDefFoundError the next time you refer to that class, but you need to be very careful in the constructor of a enum, if the system captures an exception in a constructor of an enum, you won't get that class anymore.

In conclusion, NEVER THROW EXCEPTION in the constructor of a enum!!!

Play-Morphia plugin

Just finish this new play plugin. It aims to seamlessly provide mongodb access to play application. Here are some features it provides which lack from the current play-mongo plugin:

* Compatible with play.db.Model framework
* Support Fixtures.load and CRUD module
* Smart ID field management. Yes you can choose Long type if you like it. You are also free to mark your own ID field
* Support "OR" relationship in query
* Automatic index management

An updated YABE sample is included in the distribution.

This plugin is published at http://github.com/greenlaw110/play-morphia