OpenWRT + Lighttpd + Fastcgi + Django

Introduction

Here's something I was fighting with for about 3 days. At first I wanted to use virtualenv too, but it all resulted in a headache. I'm a novice in these matters, so forgive me.

Recently I bought a router TP-Link WR-1043ND. Not the best choice now, I'm aware: TP-Link MR3420 has higher quality to price ratio. Anyway, my router got upgraded to OpenWRT, version Backfire 10.3-RC4, thanks to great work of Cezary Jackiewicz aka Obsy.

Installation

Opkg packages

First packages that we can install with OpenWRT package manager:

• lighttpd, lighttpd-mod-alias, lighttpd-mod-rewrite, lighttpd-mod-fcgi - packages needed for http server called lighty
• pysqlite, libsqlite3 - the database
• python wasn't installed at first, so we'll need this too

opkg install lighttpd lighttpd-mod-alias lighttpd-mod-rewrite lighttpd-mod-fcgi
opkg python libsqlite3 pysqlite

Python packages

These will be installed from Python Package Index (PyPI). We will need:

• setuptools - Go to pypi.python.org, search for setuptools, follow the instructions.
• pip - We should be able to call: easy_install pip to obtain it.
• django - With pip in place getting django is as easy as pip install django. Pip, contrary to easy_install, has a nice uninstall option.
• flup - For FastCGI related stuff

There we go. It's all installed.

Configuration

3 ways of doing this

First of all, let me briefly explain how FastCGI works. Actually this might not be a good idea to believe me, because, as I said, I'm a layman in this matters. There are 3 entities in this process: the browser, the server (Lighttpd), the FastCGI process (Django).
First step is forwarding the http request for a webpage to FastCGI process:

Browser => Lighttpd => Django FCGI

Request is processed and Django returns html source code of a webpage. Shove it back to your browser and render it.

Browser <= Lighttpd <= Django FCGI

I came to conclusion that there are 3 ways of setting this up.
1. By bin-path
2. By host and port
3. By socket

My app's directory: /www/apps/homepage. It doesn't have to be anything fancy. It's ok to use brand new project created with django-admin.py startproject homepage.

Bin-path

This method is my favourite so far. We tell lighttpd where to find executable file .fcgi and it starts FCGI processes by itself. All we need to take care of now is that the server is started after reboot. Here are the details:

• Contents of .fcgi file. It sits at the django project's directory.
  

  #!/usr/bin/env python
  import sys, os
  
  # Add a custom Python path.
  sys.path.insert(0, "/www/apps")
  
  # Set the DJANGO_SETTINGS_MODULE environment variable.
  os.environ['DJANGO_SETTINGS_MODULE'] = "homepage.settings"
  
  from django.core.servers.fastcgi import runfastcgi
  # maxspare=2 is minimum from what i saw
  runfastcgi(method="threaded", daemonize="false", maxspare=2)

• Lighttpd config file from /etc/lighttpd/lighttpd.conf. It is edited version of default config file. I just omitted most of commented out lines.

  # lighttpd configuration file
  # 
  ## modules to load
  # all other module should only be loaded if really neccesary
  # - saves some time
  # - saves memory
  server.modules = ( 
   "mod_rewrite", 
  # "mod_redirect", 
   "mod_alias", 
  # "mod_auth", 
  # "mod_status", 
  # "mod_setenv",
   "mod_fastcgi",
  # "mod_proxy",
  # "mod_simple_vhost",
  # "mod_cgi",
  # "mod_ssi",
  # "mod_usertrack",
  # "mod_expire",
  # "mod_webdav"
  )
  
  # force use of the "write" backend (closes: #2401)
  server.network-backend = "write"
  
  ## a static document-root, for virtual-hosting take look at the 
  ## server.virtual-* options
  server.document-root = "/www/public"
  
  ## where to send error-messages to
  server.errorlog = "/var/log/lighttpd/error.log"
  
  ## files to check for if .../ is requested
  index-file.names = ( "index.html", "default.html", "index.htm", "default.htm" )
  
  ## mimetype mapping
  mimetype.assign = (  
   ".pdf"   => "application/pdf",
   ".class" => "application/octet-stream",
   ".pac"   => "application/x-ns-proxy-autoconfig",
   ".swf"   => "application/x-shockwave-flash",
   ".wav"   => "audio/x-wav",
   ".gif"   => "image/gif",
   ".jpg"   => "image/jpeg",
   ".jpeg"  => "image/jpeg",
   ".png"   => "image/png",
   ".css"   => "text/css",
   ".html"  => "text/html",
   ".htm"   => "text/html",
   ".js"    => "text/javascript",
   ".txt"   => "text/plain",
   ".dtd"   => "text/xml",
   ".xml"   => "text/xml"
   )
  
  $HTTP["url"] =~ "\.pdf$" {
   server.range-requests = "disable"
  }
  
  ##
  # which extensions should not be handle via static-file transfer
  #
  # .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi
  static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
  
  ## to help the rc.scripts
  server.pid-file = "/var/run/lighttpd.pid"
  
  #### fastcgi module
  ## read fastcgi.txt for more info
  # this line may help with finding what's wrong, check out errorlog file
  # fastcgi.debug=1
  fastcgi.server = (
   "/homepage.fcgi" => (
    "main" => (
     "host" => "127.0.0.1",
     "port" => 3033,
     "check-local" => "disable",
     "max-procs" => 1,
     "bin-path" => "/www/apps/homepage/homepage.fcgi"
    )
   )
  )
  
  alias.url = (
                  "/media" => "/www/apps/homepage/media",
          )
  
  url.rewrite-once = (
                  "^(/media.*)$" => "$1",
                  "^/favicon\.ico$" => "/media/favicon.ico",
                  "^(/.*)$" => "/homepage.fcgi$1",
          )
  

Host:port

You don't need .fcgi file in this method. Remove line with "bin-path" in fastcgi.server section of lighttpd config. Now you need to make sure that you run this at every boot (maybe you can use start-stop-daemon tool for this):

/www/apps/homepage/manage.py runfcgi method=threaded host="127.0.0.1" port=3033

This will start FastCGI threads. Lighttpd will expect them to be there when he needs a page. Otherwise you'll get 503 or 500. This method is said to be easier because there is no need to set permissions on TCP socket.

Socket

In this method FCGI process and HTTP Server communicate through a socket (aka named pipe). Proper permissions must be in place. FCGI file is also not needed here. You must make sure that you run similar command as in the second method:

/www/apps/homepage/manage.py runfcgi method=threaded socket=/www/apps/homepage/homepage.socket

Lines in fastcgi.server with port and host get removed. Instead add

"socket" => "/www/apps/homepage/homepage.socket"

Conclusion

I use a lot of bad practices here:

• do everything as root
• not using virtualenv
• keeping my app in /www (but I changed my document-root, so maybe I'm ok?)

This is an example of work done by someone who came from "no idea" to "it started working" and all this happens on a router that won't be under a lot of load or work in a cloud of virtual machines. Certainly it's not state of the art, but gets the job done.

There's quite a lot of this kind of tutorials, but to someone without proper knowledge all these terms and blindly followed instructions are no good if something doesn't work as expected. I hope someone will find the wording I used useful. I hope will when I'll need to do everything all over again.