Field Notes Inside an Integrated Communications Agency

Articles

• HOWTO: Set up Django on WebFaction

  Jun 6, 2008 :: Mike Kibbel :: 1 Comments

  This article describes the process we use at Capstrat to set up Django applications on WebFaction shared hosting. The instructions are specific to Mac OSX or Linux, but I've included a few additional notes for Windows users. It is assumed that you are comfortable working with the command line, that you have Python and Subversion client installed on your machine and that you have checked out and worked with Django before.

  1. Create a new WebFaction Shared hosting account (Shared-1 through Shared-4, according to your project's needs).
     1. You can do this at http://www.webfaction.com/signup/
  2. Create a new Django application on your local machine.
     1. I recommend keeping your projects in an easily accessible place, like /src/python/ (C:\src\python\ on Windows).
     2. At the command line, navigate to the directory where you keep your Web projects.
     3. Run:
        python <path-to-django>/bin/django-admin.py startproject <project-name>
        to create a new project.
        1. Note: <path-to-django> refers to the place where the Django source code exists on your machine.
           Additionally, <project-name> will be the name of your new project.
           
        2. Concretely, I would do this on my machine:
           cd /src/python
python /opt/local/lib/python2.5/django/bin/django-admin.py startproject mikesproject

  3. Set up your project's static assets.
     1. I store my project's static assets in a directory named "elements." Create this directory.
        cd /src/python/<project-name>
mkdir elements
mkdir elements/css      # store all CSS here
mkdir elements/img      # store all HTML design images here
mkdir elements/js       # store JavaScript files here
mkdir elements/swf      # store Flash files here
mkdir elements/media    # store user-contributed media here

     2. To simplify local development and deployment, you can use Django's static media server to serve out the elements folder. Note: when the project is deployed to WebFaction, you must be certain that these files are served by Apache and not by the Django media server.
        1. In settings.py, add these new global settings variables (anywhere you want):
           DEVELOPMENT = True
PROJECT_DIRECTORY = '/src/python/<project-name>/'
ELEMENTS = PROJECT_DIRECTORY + 'elements/'

        2. In the global urls.py, add this code at the bottom of the file:
           # Catch elements URLs for working in development
if settings.DEVELOPMENT:
   urlpatterns += patterns('',
      (r'^elements/(?P<path>.*)$', 'django.views.static.serve', { 'document_root': settings.ELEMENTS, 'show_indexes': True }),
   )
        3. You will also need to add this to the top of the urls.py file to avoid a run-time error:
           import settings

     3. If you want to use Django's admin interface, set ADMIN_MEDIA_PREFIX in settings.py like so:
        ADMIN_MEDIA_PREFIX = '/elements/admin/'
  4. Set up your project's database.
     1. To ease development, we typically use sqlite3. The sqlite3 database is stored in a single file and can be easily passed around without the need for a dump/load procedure.
     2. In settings.py, set the database settings like so:
        DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = PROJECT_DIRECTORY + 'db.sqlite3'

     3. Leave the other database settings blank.
     4. If you want to use Django's admin interface, add this line to the end of the INSTALLED_APPS tuple in settings.py:
        'django.contrib.admin',

     5. Finally, create the database file. On the command line (in your project directory), type:
        python manage.py syncdb

        1. You will need to enter your new administrator username, email and password as part of the initial sync.
  5. Create a settings file for WebFaction.
     1. settings.py will be different on WebFaction than on your local machine. Copy your local settings.py to a new file called wf-settings.py. In your project directory, type:
        cp settings.py wf-settings.py

     2. Edit the new wf-settings.py file.
        1. Set the DEVELOPMENT global to False. This will disable the static media server set up in step 3.2.
        2. Modify the PROJECT_DIRECTORY setting to the directory where your project will exist on WebFaction. Assuming that you set up your account as a Django account, it should look like:
           PROJECT_DIRECTORY = '/home/<account-name>/webapps/django/<project-name>/

     3. Maintaining a separate settings file for WebFaction is a reasonably efficient way to manage differences between your local machine and the WebFaction server; however, note that you will need to manually keep the two files in sync. Anything you add to your local settings.py will need to be added to wf-settings.py as well.
  6. Set up Subversion on WebFaction.
     1. Log in to the WebFaction control panel at http://panel.webfaction.com with the username and password provided for your new account by WebFaction.
     2. In the navigation menu, under "Domains/Websites" click on the "Applications" link.
        1. Click on the "Add" button in the bottom right to create a new application.
        2. Type "subversion" into the name field.
        3. Choose "Subversion" from the App type drop-down.
        4. Click the "Create" button.
     3. Now, create a subdomain for Subversion. In the navigation menu, under "Domain/Websites" click on the "Domains" link.
        1. Click on the "Edit" button in the row for "<account-name>.webfactional.com"
        2. Click on the "Add" button in the bottom right of the "Subdomains" section.
        3. Enter "svn" in the new "Prefix" field.
        4. Click the "Update" button.
     4. Create a Website to tie the subdomain to the application. In the navigation menu, under "Domains/Websites" click on the "Websites" link.
        1. Click the "Add" button in the bottom right to create a new Web site.
        2. Enter "subversion" in the name field.
        3. Select "svn.<account-name>.webfactional.com" in the "Subdomains" field.
        4. Click the "Add" button under "Site apps".
        5. Choose "subversion" from the "App" drop-down, and enter "/" in the "URL path" field.
        6. Click the "Create" button.
        7. Now, the Subversion repository is set up and running at http://svn.<account.name>.webfactional.com/
     5. Next, you need to ssh into your WebFaction account to configure the Subversion repository permissions.
        1. At the command line, type:
           ssh <account-name>@<account-name>.webfactional.com

        2. Enter your password to log in.
        3. Navigate to the Subversion directory and set up usernames and passwords to control access to the repository:
           cd ~/webapps/subversion

           1. View the .htpasswd file to see what usernames have been set up:
              less .htpasswd

           2. Delete the "test" username from .htpasswd
              htpasswd -D .htpasswd test

           3. Create a username and password for all the people who will have access to the repository (the example here uses md5 hashes to store the passwords):
              htpasswd -m .htpasswd <username>
(Enter the new password twice at the prompt.)
              
           4. Review the list of usernames to verify that you've added all desired users:
              less .htpasswd

     6. Congratulations! Subversion is ready to use.
  7. Check your Django project into Subversion.
     1. First, create the directories that you'll need in the repository, according to Subversion best practices. At your local command prompt, from your project folder type:
        svn mkdir -m "initial project directory structure for <project-name>" http://svn.<account-name>.webfactional.com/<project-name> http://svn.<account-name>.webfactional.com/<project-name>/trunk http://svn.<account-name>.webfactional.com/<project-name>/tags http://svn.<account-name>.webfactional.com/<project-name>/branches

     2. Now, verify that your local subversion settings are correct before you check in any code. I recommend not storing .pyc files in the repository and also not checking in the settings.py file.
        1. Open this file for editing: ~/.subversion/config ("C:\Documents and Settings\<windows-username>\Application Data\Subversion\config" on Windows)
        2. In the [miscellany] section, uncomment the "global-ignores" line and add any filename patterns that you want Subversion to ignore. On my machine, where I'm primarily performing Django development using TextMate as my editor, my global-ignores settings looks like this:
           global-ignores = *.o *.lo *.la *.pyc *.tmproj settings.py .*

        3. The most imporant entries here are "*.pyc" and "settings.py" - these prevent Subversion from checking in and managing binary compiled Python (.pyc) and Django settings.py files.
     3. Now, you can import your new project into the Subversion trunk you set up in step 7.1. In your project's directory, type:
        svn import . http://svn.<account-name>.webfactional.com/<project-name>/trunk/<project-name>

        1. Note: This will import all files and directories from your project with the exception of settings.py (because of your "global-ignores" setting).
     4. At this point, your code is stored in the repository, but your local files are not yet managed by your Subversion client. You need to check out the project you just imported to get a managed local copy. Starting in your project directory, perform these commands.
        cd ..                                        # go up one directory
mv <project-name> <project-name>_tmp         # move the project to a temp directory
svn co http://svn.<account-name>.webfactional.com/<project-name>/trunk/<project-name> <project-name> # check out the project
cp <project-name>_tmp/settings.py <project>  # copy the original setting.py file into the checked out project.
rm -rf <project_name>_tmp                    # remove the temporary project folder

     5. Now you have a local working copy of the code stored in the Subversion repository.
  8. Check out the project on the WebFaction server.
     1. You must have a Django application set up on the server before you can perform this step. If you chose Django as the application type when you created your WebFaction account, this is already done. Otherwise, perform these steps:
        1. In the WebFaction control panel's navigation menu, under "Domains/Websites" click on the "Applications" link.
        2. Click the "Add" button in the bottom right.
        3. Enter "django" in the "Name" field.
        4. Choose a version of Django from the "App type" drop-down. At the time of this writing, I'm using "Django 0.96.2/mod_python 3.3.1/Python 2.5"
        5. Click the "Create" button.
     2. SSH into WebFaction. Navigate into your Django project directory and check out the project:
        cd ~/webapps/django
svn co http://svn.<account-name>.webfactional.com/<project-name>/trunk/<project-name> <project-name>

     3. Create the setings.py file for the server by copying wf-settings.py
        cd ~/webapps/django/<project-name>
cp wf-settings.py settings.py

  9. Create a new application to serve static media in your project's elements folder.
     1. In the WebFaction control panel's navigation menu, under "Domains/Websites" click on the "Applicatins" link.
     2. Click the "Add" button in the bottom right.
     3. Enter "django_elements" in the "Name" field.
     4. Choose "Symbolic Link" from the "App type" drop-down.
     5. Enter the absolute path to the elements folder in the "Extra info" field. It should look like this:
        /home/<account-name>/webapps/django/<project-name>/elements

     6. Click the "Create" button.
  10. Set up Apache to serve your Django application.
      1. SSH into the your WebFaction account. Navigate to the Apache configuration folder:
         cd ~/webapps/django/apache2/conf

      2. Edit the httpd.conf file. Modify this line:
         SetEnv DJANGO_SETTINGS_MODULE myproject.settings
to read:
         SetEnv DJANGO_SETTINGS_MODULE <project-name>.settings

  11. Finish the remaining tasks in the WebFaction control panel.
      1. In the WebFaction control panel's navigation menu, under "Domains/Websites" click on the "Websites" link.
      2. If you chose Django as the application type when you created your WebFaction account, there should already be a Web site called <account-name>. If not, create one by following these steps:
         1. Click the "Add" button in the bottom right.
         2. Enter "<account-name>" in the "Name" field.
         3. Choose the desired subdomain(s) (something other than svn.<account-name>.webfactional.com).
         4. Under "Site apps," click the "Add" button.
         5. Choose "django" from the "App" drop-down. Enter "/" in the "URL Path" field.
         6. Click the "Create" button.
      3. Now, edit the Web site by clicking the edit button in the "<account-name>" row.
         1. Under "Site apps," click the "Add" button.
         2. Choose "django_elements" from the "App" drop-down. Enter "/elements" in the "URL Path" field.
         3. Now, your site is set up to serve out the elements files with Apache and to handle all other URLs with your Django application.
  12. Tie it all together.
      1. SSH into your WebFaction account.
      2. Check out the version of Django you require for your application:
         cd ~/lib/python2.5
You may need to remove an existing copy of django: rm -rf django
svn co http://code.djangoproject.com/svn/django/trunk/django django
If you need a different revision than the latest trunk, do this: svn update -R <desired-revision> django
         
      3. Set up a symbolic link to Django for you project:
         cd ~/webapps/django/lib/python2.5
rm -rf django                       # remove the WebFaction-provided copy of Django
ln -s ~/lib/python2.5/django django # create a symbolic link to your revision of Django
Note: Here, we have modified the project-specific Python lib folder that Apache uses when it runs Django. If you need to run different revisions of Django for two different applications on the server, you could alternatively check out the required revision in the project-specific lib folder, rather than using a symbolic link to the global copy of Django.
         
      4. Create a symbolic link for serving Django's admin media files out of your elements folder:
         cd ~/webapps/django/<project-name>/elements
ln -s ~/lib/python2.5/django/contrib/admin/media admin
Note: the symbolic link name "admin" corresponds to the ADMIN_MEDIA_PREFIX in settings.py from step 3.3.
         
      5. Make Subversion ignore the new symbolic link. From the same directory, type:
         svn propset svn:ignore admin .

      6. Configure the Subversion client on WebFaction. Just like in step 7.2, you'll need to edit the Subversion config file:
         1. Open this file for editing: ~/.subversion/config
         2. In the [miscellany] section, uncomment the "global-ignores" line and change it to look like this:
            global-ignores = *.o *.lo *.la *.pyc *.tmproj settings.py .*

         3. Again, be sure to add "settings.py" to the list so that you don't clobber your local machine's version of this file.
      7. Check in the working copy from the server:
         cd ~/webapps/django/<project-name>
svn ci -m "working copy is now set up on WebFaction"

      8. Restart Apache:
         cd ~/webapps/django/<project-name>
../apache2/bin/stop
../apache2/bin/start

      9. You should now be able to view your project running in a Web browser at the subdomain you specified in step 11.2.
      10. Finally, you'll probably need to set up an alias to Python 2.5 if you intend to work in the Python interactive shell on WebFaction.
          1. SSH into your WebFaction account and edit the file ~/.bashrc
          2. At the bottom of the file, add this line:
             alias python=/usr/local/bin/python2.5

          3. Save the file and run the command source .bashrc to make the changes take effect. Now, running python will invoke Python 2.5, so Django will be available for import.
  13. Develop.
      1. Write your Django application incrementally in your local working copy.
      2. When you have created and tested a new feature, check it in to subversion.
      3. SSH into WebFaction and update the working copy to apply the checked-in changes.
         cd ~/webapps/django/<project-name>
svn update

      4. If you have modified any Python files (*.py files), restart Apache so that mod_python reloads your files:
         ../apache2/bin/stop
../apache2/bin/start

  Tags: django howto htpasswd subversion webfaction

• Stronger Validators for the Django Admin

  Mar 12, 2008 :: Mike Kibbel :: 3 Comments

  In the current development version of Django, model field validation in the admin has one difficult shortcoming: validator functions don't know if the fields they are validating are for an object that is being created or edited. In the common case, this isn't a problem, but certain problems cannot be solved without this information. In particular, these problems can't be solved with Django validators:

  • Automatically creating unique URL slugs from other fields in the model (e.g. the "title" CharField)
  • Preventing self-referencing foreign key infinite loops

  Consider this partial implementation of a Blog Article model that has URLs like this "/2008-3-11/my-article/" which allows articles to have the same title (actually, the same slug) so long as they are published on different dates:

  #...
class BlogArticle(models.Model):
   title = models.CharField(
      'Name',
      max_length = 100,
   )
   slug = models.CharField(
      'Slug',
      max_length = 100,
      editable = False,
   )
   published = models.DateField(
       'Date',
   )
   # more fields here...

   class Admin:
      # options...
      pass

   class Meta:
      unique_together = (('slug', 'published',),)

   def save(self):
      from django.template.defaultfilters import slugify
      self.slug = slugify(self.title)
      super(BlogArticle, self).save()

   def get_asbolute_url(self):
      p = self.published
      return "/%d-%d-%d/%s/" % (p.year, p.month, p.day, p.slug,)
#...


  This model has a "slug" field that gets populated from the "title" field in the "save()" method. This seems okay at first glance, but consider what would happen if a user entered these two articles in the admin:

  title = "My Article"
published = "2008-3-11"
title = "my article"
published = "2008-3-11"


  Despite the change in letter case, these articles will have the same slug and the same date, so the produced URLs won't be unique. Unfortunately, the "unique_together" constraint will not help us here in the admin, because "slug" is not calculated until "save" is called, which happens after all validators have run. This situation will produce a database exception, rather than a validation error, resulting in an ugly and confusing user experience. This little experiment also shows that adding an additional "unique_together" constraint for ('title', 'published',) would also be ineffective, because we could still get non-unique URLs from two similar titles with letter case changes or other subtle changes.

  To solve this properly, we'll need to write a validator for the "title" field that checks whether the slugified version of the title is actually unique for the given date. It should look something like this:

  def validate_title(field_data, all_data):
   # do nothing if 'title' or 'published' didn't pass initial validation
   if 'title' not in all_data or 'published' not in all_data:
      return

   # calculate the slug and published date and check if another article with the
   # same slug and published date already exists
   from django.template.defaultfilters import slugify
   from datetime import date
   slug = slugify(field_data)
   published = date(*[int(d) for d in all_data['published'].split('-')])

   if BlogArticle.objects.filter(slug = slug, published = published).count():
      # error ???
      pass


  This looks almost right. It will work perfectly when we're creating a new BlogArticle, but this will prove to be a miserable bug when editing an existing BlogArticle, because we'll be forced to change the blog title or publish date to get the validation to work. To correct this logic, the validator needs access to the BlogArticle object being edited. Suppose the validator has access to this object, call it "original_object," which will be set to None when we are creating an object. Then the validator would look like this:

  def validate_title(original_object, field_data, all_data)
   # do nothing if 'title' or 'published' didn't pass initial validation
   if 'title' not in all_data or 'published' not in all_data:
      return

   # calculate the slug and published date
   from django.template.defaultfilters import slugify
   from datetime import date
   slug = slugify(field_data)
   published = date(*[int(d) for d in all_data['published'].split('-')])

   # we need to check for unqiueness if we are creating a new object or
   # the existing object has changed
   if not original_object or original_object.slug != slug or original_object.published != published:
      if BlogArticle.objects.filter(slug = slug, published = published).count():
         raise ValidationError("A blog article with a similar title was already published on this date. Please change the title.")


  This will accomplish the validation we need, but there's a big problem here: how do we figure out the "original_object?" We could try to query for an existing object based on some data in "all_data," but that would be a fallacy, because "all_data" is being edited by the user and may no longer match an existing object in the database. We know that the admin view code must be aware of whether an object is being added or edited. If you take a quick look in the admin view functions (django.contrib.admin.views.main), you'll see an "add_stage" and a "change_stage" function. These functions instantiate an object of your model class' AddManipulator or ChangeManipulator, respectively, to accomplish the adding or editing process. The ChangeManipulator object stores the object being edited in its "original_object" attribute, and the AddManipulator has no such attribute. We need to find a point to hook into the admin / field / manipulator code to gain access to this "original_object" so we can pass it to our validator.

  The validation occurs after POST when the manipulator object's "get_validation_errors" function is called. This function calls "get_validation_errors" for each oldforms.FormField object in the manipulator (the manipulator's "fields" are all objects that subclass django.oldforms.FormField). The oldforms.FormField's "get_validation_errors" function calls each validator function in its "validator_list" attribute, passing each function "field_data" and "all_data," the value of the current field being processed and all the POSTed values, respectively. This is a clean mechanism for handling validation, but the oldforms.FormFields' "get_validation_errors" functions are still hopelessly unaware of the current object being edited.

  The trick that I've discovered is to hook into the code at the point where the manipulator's oldforms.FormField objects are created. At this point in the code, the "validator_list" can be modified with more powerful validators. When the admin views construct an AddManipulator or ChangeManipulator object, the manipulator constructor creates an oldform.FormField object for each field in your model class (for your reference, the model class fields inherit from django.db.models.fields.Field). The manipulator constructor accomplishes this by calling each model fields' "get_manipulator_fields" function, passing itself to each function as a parameter. The "get_manipulator_fields" function creates the oldforms.FormField object and, of course, sets its "validator_list" from the corresponding attribute stored in the model field object. At this point, we can hook in and modify the "validator_list" before the framework creates the oldforms.FormFields. I suggest doing this by subclassing a model field class. For django.db.models.fields.CharField, it looks like this:

  class ContextValidatedCharField(models.CharField):
   def __init__(self, context_validators = [], *args, **kwargs):
      # call the superclass constructor
      super(ContextValidatedCharField, self).__init__(*args, **kwargs)

      # keep track of the original validator list
      import copy
      self._orig_validator_list = copy.deepcopy(self.validator_list)

      # context_validators can be a single function or a list of functions
      if type(context_validators) is not list:
         context_validators = [context_validators,]
      self.context_validators = context_validators

   def get_manipulator_fields(self, opts, manipulator, change, name_prefix = '', rel = False, follow = True):
      # pass on original_object information to the custom validator(s)
      from django.utils.functional import curry
      new_validators = []
      for validator in self.context_validators:
         # convert the three-parameter validator into a two-parameter function by
         # currying in the original_object as the first parameter
         new_validators.append(curry(validator, getattr(manipulator, 'original_object', None)))
      self.validator_list = self._orig_validator_list + new_validators

      # just use the framework, which will incorporate our modified validator_list
      return super(ContextValidatedCharField, self).get_manipulator_fields(opts, manipulator, change, name_prefix, rel, follow)

   # we need this to make the field behave correctly
   def get_internal_type(self):
      return "CharField"


  Now, in our BlogArticle model, we'll change the "title" field to this:

     title = ContextValidatedCharField(
      verbose_name = 'Name',
      context_validators = [validate_title,],
      max_length = 100,
   )


  Note the new parameter "context_validators" that accepts our modified validate_title validator function. Now, slugs will be safely created from the title field and all blog articles will have unique URLs.

  Now, what about preventing infinite loops in self-referencing foreign keys? As a concrete example of this problem, consider this: We want to build a model representing a simple website's navigation system. We'll create a NavigationNode model that stores each node's parent NavigationNode (with top-level nodes having no parent) and information about the URL and page to display. A stripped-down version of the model might look like this:

  class NavigationNode(models.Model):
   parent = ContextValidatedForeignKey(
      'self',
      verbose_name = 'Parent Node',
      context_validators = [validate_parent,],
      blank = True,
      null = True,
   )
   slug = models.SlugField(
      'Slug',
   )
   full_path = models.CharField(
      max_length = 255,
      editable = False,
   )
   page = models.ForeignKey(
      FlatPage, # assume that we have a model called FlatPage
      verbose_name = "Flat Page",
   )
   # more fields here...

   class Admin:
      pass

   def save(self):
      # store the full path for easy URL lookups and other performance optimizations
      if self.parent:
         self.full_path = self.parent.full_path + self.slug + '/'
      else:
         self.full_path = '/' + self.slug + '/'
      super(NavigationNode, self).save()

   def get_absolute_url(self):
      return self.full_path


  Here, we've used a new field subclass called "ContextValidatedForeignKey" that works similarly to the "ContextValidatedCharField" described above. Its definition looks like this:

  class ContextValidatedForeignKey(models.ForeignKey):
   def __init__(self, to, context_validators = [], *args, **kwargs):
      # call the superclass constructor
      super(ContextValidatedForeignKey, self).__init__(to, *args, **kwargs)

      # keep track of the original validator list
      import copy
      self._orig_validator_list = copy.deepcopy(self.validator_list)

      if type(context_validators) is not list:
         context_validators = [context_validators,]
      self.context_validators = context_validators

   def get_manipulator_fields(self, opts, manipulator, change, name_prefix = '', rel = False, follow = True):
      # pass on context information to the custom validator(s)
      from django.utils.functional import curry
      new_validators = []
      for validator in self.context_validators:
         new_validators.append(curry(validator, getattr(manipulator, 'original_object', None)))
      self.validator_list = self._orig_validator_list + new_validators

      return super(ContextValidatedForeignKey, self).get_manipulator_fields(opts, manipulator, change, name_prefix, rel, follow)

   def get_internal_type(self):
      return "ForeignKey"


  Finally, we need to implement the validator:

  def validate_parent(original_object, field_data, all_data):
   # if previous validation of parent or slug failed, skip
   if 'parent' not in all_data or 'slug' not in all_data:
      return

   # get the parent and slug values
   slug = all_data['slug']
   if field_data:
      parent_id = int(field_data)
   else:
      parent_id = None

   # if we're creating a new object or the existing object has changed, we need to verify
   # that the URL will be unique
   if not original_object or original_object.parent_id != parent_id or original_object.slug != slug:
      # first check if the slug is okay here
      if parent_id:
         if NavigationNode.objects.filter(parent__id = parent_id, slug = slug).count():
            raise ValidationError("Another node already uses this URL. Please change the slug.")
      else:
         if NavigationNode.objects.filter(parent__isnull = True, slug = slug).count():
            raise ValidationError("Another root node already uses this URL. Please change the slug.")

   # next, we need to check for an infinite loop
   # if we're editing an object, we need to verify that the object doesn't exist anywhere
   # in its ancestor path
   if original_object and parent_id:
      p_id = parent_id
      while p_id:
         # try - just in case
         try:
            parent = NavigationNode.objects.get(pk = p_id)
         except:
            parent = None

         if parent and original_object.id == parent.id:
            raise ValidationError("Recursive path detected! This node cannot be in its own parent path.")

         # move up the path
         p_id = parent.parent and parent.parent.id or None


  And that's it. An attempt to solve a similar problem was written up on djangosnippets. The author wrote a standard validator, but the code falls short of solving the problem, because it assumes that the slug field does not change. The code is still prone to producing an infinite loop.

  Clearly, this "ContextValidatedField" method is a bit of a hack. A better solution might be to modify the Django framework to pass the original object data to all validator functions, preferably in a backwards-compatible way. One obvious problem with that approach is that validator functions could do perverse things like delete the original object. Maybe a better solution would be to have the framework pass another dictionary called "old_data" or "original_data" as a third parameter to each validator function that would contain all of the original object's data but would provide no mechanism to alter the original object.

  I wasn't able to find any TRAC tickets or other information with a quick search of djangoproject.com, and I think this problem may merit opening a ticket. I welcome any comments or ideas on how to solve this problem more cleanly or easily in Django.

  Tags: automatic django foreign infinite key loop slugs validators

• Making Django Environmentally Friendly

  Nov 6, 2007 :: Mike Kibbel :: 7 Comments

  If you've ever needed to use the Django database API or templating system from a standalone Python script, you've probably written a quick workaround or ugly hack to get the Django environment working properly. Specifically, this problem has to be solved in order to run Django scripts as cron jobs. Some possible solutions to this problem were written up by James Bennet and Scott Newman. I prefer the clean approach in this method documented by Jared Kuolt.

  Jared uses an undocumented function in django.core.management called setup_environ to prepare a script so that Django can be used normally, just like you would use it in any of your website's view modules. It goes like this:

  from django.core.management import setup_environ
import settings
setup_environ(settings)


  The setup_environ function simply takes your project's settings module as a parameter and sets up Django accordingly. Search for this function in django/core/management.py if you want to see the details of how Django sets itself up. Though this method is simple, elegant, and decently abstracted, it has two main limitations. It may only be a three-line incantation, but it's a pain to memorize these sort of things, and it would really be ideal if this could be accomplished in one line. More importantly, in order for this method to work, your script's file must live in the root of your project, so that "settings.py" is its neighbor. I prefer to keep my cron job scripts or data importing scripts in an appropriately named subdirectory, such as "scripts." From a subdirectory, import settings will not work.

  I've written a script to solve this problem so that you only need to write one line of code that's easy to memorize. You'll need to copy and unzip this script, setup_django.py, into the directory where your script lives. Write this as the first line of your script:

  import setup_django

  And voila! That's all you need to do. You can now use the Django database API just as you'd expect. The code borrows some ideas directly from the setup_environ function in django.core.management. It goes like this:

  import os, sys, re

# stop after reaching '/' on nix or 'C:\' on Windows
top_level_rx = re.compile(r'^(/|[a-zA-Z]:\\)$')
def is_top_level(path):
   return top_level_rx.match(path) is not None

def prepare_environment():
   # we'll need this script's directory for searching purposess
   curdir, curfile = os.path.split(os.path.abspath(__file__))

   # move up one directory at a time from this script's
   # path, searching for settings.py
   settings_module = None
   while not settings_module:
      try:
         sys.path.append(curdir)
         settings_module = __import__('settings', {}, {}, [''])
         sys.path.pop()
         break
      except ImportError:
         settings_module = None

      # have we reached the top-level directory?
      if is_top_level(curdir):
         raise Exception("settings.py was not found in the script's directory or any of its parent directories.")

      # move up a directory
      curdir = os.path.normpath(os.path.join(curdir, '..'))

   # set up the environment using the settings module
   from django.core.management import setup_environ
   setup_environ(settings_module)

prepare_environment()


  When you import this module, it runs prepare_environment. First, the function calculates its own path. It then takes this path and moves upward one directory at a time, attempting to import a module named "settings." At each step, it checks if it has reached the root directory using the is_top_level function, which employs a simple regular expression for detecting the root directory on Unix or Windows. If the root has been reached, an exception is raised; otherwise, when the settings module is found, it is passed to Django's setup_environ function.

  Some may not find this method to be as nice as others, because you'll need to copy the "setup_django.py" script into all of your script directories, but the one-line import setup_django solution is very nice. The main caveat to consider is that if you have a file named "settings.py" other than the project's main "settings.py" in the directories above your script's path, this method will not work. This was only tested in Python 2.5.x, but there shouldn't be any problems running it in slightly older versions of Python.

  Enjoy!

  Tags: command-line cron crontab django environment python script standalone

• Web Standards: Accessibility - What You May Not Have Considered

  May 31, 2007 :: Mike Kibbel :: 0 Comments

  Accessibility has become an increasingly popular buzzword in the world of web development. Its principles should drive all modern website design, but the reasons why they should may be more significant than you might first think. The definition of web accessibility is often oversimplified as the practice of creating sites that can be used by people with disabilities. This definition glosses over the business rationale of investing in accessible design. Any business or organization with a web presence needs to make accessibility a priority, not only because it will increase your site's visitor-base, but most importantly because it is the right thing to do. An inaccessible site is exclusionary, turning away would-be visitors with a lack of empathy that only a cold, hard machine like a web server can manage. Visitors who can't access the content on your site will directly associate this lack of empathy with your organization. Clearly, excluding users of your site for lack of accessibility is not just bad for your image - it's simply unethical.

  I know I'm being a little preachy here, but with good reason. Given the technologies available for web development today, there is little excuse for making sites that are not accessible. With the ability to separate site content from format (see my article on this here), web developers, designers, and authors have all the power they need to create sites that can be used by all of their visitors despite any disabilities those visitors may have.

  It is obvious that accessibility is important, but what is it really about? Accessibility is about more than addressing the obvious concerns of how individuals with vision or hearing disabilities use your site. It also addresses how someone with a repetitive stress injury or someone who cannot accurately control a mouse might use your site. It even considers how to allow access to your site by a visitor who does not have a disability but has a slow Internet connection and wants to view your site using a text-only browser. Generally, accessibility is all about being as inclusive as possible. The method of designing for accessibility makes sites that are usable by individuals with disabilities. Its also adds the benefit that the sites it produces are cleaner and more intuitive for everyone to use. I encourage web developers and CEOs alike to consider how the concepts of accessibility are applied in practice. The World Wide Web Consortium has published an excellent document here illustrating several accessibility scenarios. Check it out - it'll be well worth the ten-minute read.

  So maybe now you've considered a bit more about web accessibility than you had before. Designing sites for accessibility is here to stay. Just like building architects must ensure that all people will be able to access every floor and room in their designs, web architects must guarantee that everyone will be able to access all of the pages and use all of the features in their sites. Rather than considering design for accessibility an obstacle, embrace it - it will produce a more intuitive website, and its use will better represent your organization in your market and community.

  At Capstrat, we're committed to keeping up with the web. Our design experts will address and resolve your site's accessibility issues. Whether you have an aging site that could use a facelift or you need a new site built from the ground up, our designers, developers, and communications experts will help your business or organization achieve the future-ready web presence that you need to succeed in this digital age.

  Tags: accessibility standards web

• Web Standards: Better Browsers - Brighter Future

  May 31, 2007 :: Mike Kibbel :: 0 Comments

  A promising development for the future of the web has been in the works for the past five years. An increasing number of users have switched to web browsers like Firefox, Opera, and Safari [wikipedia], helping to usher in the era of modern, standards-compliant websites. These browsers are heroes for the web development community, as they provide excellent support for W3C web standards (why does this matter). However, Microsoft's Internet Explorer is still the most commonly used web browser throughout the world, accounting for slightly more than eighty percent of the browser market right now [wikipedia]. For those of us working in web development, this fact presents a daily hurdle, as Internet Explorer's support for modern web standards has traditionally been lacking. Developers write web page code that works with standards-compliant browsers and then they painstakingly write fixes or "hacks" for the code (in a bizarre and contrived fashion) to get the pages working in IE. This process can discourage a developer from being creative with his or her usage of existing web standards, but - there is hope on the horizon. With Microsoft's release of Internet Explorer 7 in October 2006, the challenge of creating standards-compliant websites has begun to diminish. IE7 fixes many of the Internet Explorer quirks that have plagued developers since the widespread adoption of HTML and CSS, and it now supports some features that have long been available in competing browsers.

  Throughout 2007, more users will be upgrading their web browsers to Internet Explorer 7 or switching to alternative browsers. As a direct consequence, web designers, developers, and authors will further shift toward the uniform usage of web standards. So what does this imply? How about a rich user experience regardless of the way the Internet is accessed. It won't matter whether you're visiting a site on your office desktop, your mobile phone, or your next-generation refrigerator - you'll have full access to a site's content and features. How about accessibility for all. Users with disabilities will not be hindered from using any part of a site. And how about lower development, maintenance, and hosting costs. Websites designed with modern standards are much easier to maintain, modify, and extend than those created with the design methodologies popularized in the late nineties. Such sites also use less bandwidth, because modern pages are composed of smaller files than their ancestors.

  At Capstrat, we're committed to keeping up with the web. We're ready for the next generation of web browsers, and we are prepared to utilize these changing technologies to their maximum potential. Whether you have an aging site that could use a facelift or you need a new site built from the ground up, our designers, developers, and communications experts will help your business or organization achieve the future-ready web presence that you need to succeed in this digital age.

  Tags: accessibility browsers standards web

• Web Standards: Why Won't This Nerd Stop Saying "CSS?"

  May 31, 2007 :: Mike Kibbel :: 0 Comments

  Have you ever had the distinct pleasure of listening to a web development nerd ramble on about why your website needs to use CSS and XHTML? Did you find your mind wandering or eyes glazing over when he went on and on about W3C standards? Don't worry - this happens all the time. As a web developer, I apologize for our kind's overzealous and sometimes ridiculous excitement over these obscure acronyms. But apologies aside, I would like to say a few things about why we all get so worked up about a few simple letters.

  The technology for producing and publishing on the web has grown out of its infancy. The changes adopted in the past five years are now the cornerstone of modern web design and development: two languages called XHTML and CSS. The magic of these two languages (the part that gets web developers so excited) is their power and flexibility. In a nutshell, XHTML is a web language focused on content. It is used to spell out the textual content, media, and links on a web page. CSS is a language focused on formatting. It gets coupled with XHTML to specify things like where text should be placed on the page or what color the background of the page should be.

  Bored yet? Well, don't be - here comes the good part... The underlying point here is not to simply explain these acronyms (by the way XHTML = eXtensible Hypertext Markup Language and CSS = Cascading Style Sheets), but rather to examine the design philosophy that these languages enable us to use. Modern web standards dictate that page content should be separated from page format. Concretely, this is actually done by storing the page content in an XHTML file and the page format in a separate CSS file. It may not be immediately clear why anyone would want to do things this way, so consider the following scenarios:

  • Your web site has over 500 distinct pages. Recently your organization decided to change the background color of all the pages from pure white to a striking blue that matches your logo. Unfortunately, the pages weren't designed with XHTML and CSS, so a poor intern at your organization gets assigned the tedious task of changing the background color in hundreds of web page files. With modern design methods, all that would need to be changed is a single line of code in a CSS file.
  • Market analysis shows that a significant and increasing number of your company's customers use their mobile phones to browse the Internet, and you'd like to attract them to your website. Your site was designed with its formatting embedded in the content (rather than in a separate CSS file). The site's width is 750 screen pixels, but a mobile-ready version of the site must be only 300 pixels wide. Since the site's format was specified in the same file as its content, every single file on the site will need to be rewritten to accommodate the change in width. Had the site been designed using XHTML and CSS, it would only have been necessary to modify a handful of CSS files to produce layout changes throughout the entire site.
  • A customer with a vision disability visits your website using an assistive technology called a screen reader that reads the content of the site's pages aloud. Your site was designed in the year 2000, and its layout was created by embedding text and images inside of tables nested within tables (sound messy?). This sort of design was very common just a few years ago, because it allowed web designers to achieve precise control over how the website displayed in different web browsers. Unfortunately, the customer's screen reader cannot make sense of the table layout, and it reads the textual contents of the site in a confusing and illogical way. Frustrated by the experience, the customer returns to Google to search for a competitor's site that is more accessible. Had this site's content been separated from its format using XHTML and CSS, the customer's screen reader would have read the text on the site in the correct, logical order that was intended by the designer.

  These scenarios demonstrate the key features of separating format from content in web design: maintainability, modifiability, and accessibility (and that's not even mentioning code readability, content manageability, or how-could-it-possibly-be-more-awesomeability). In a world where we all use document editors that let us format our text line by line and see the results immediately, it can be difficult to imagine how separating the text content from its format could be useful. What I mean is, imagine having to create and edit an extra file to change the font face from Times to Arial in the letter you just wrote in Word. That doesn't seem convenient or intuitive, but in a website, it's essential. Modern sites are typically hand-coded in XHTML and spread out across many files. There is no method to "select all text" and "change the font face to Arial" for every page in the site - that is, unless you're using CSS! So the next time your web development nerd rambles on about CSS, rest assured, he's speaking in your best interest.

  At Capstrat, we're committed to keeping up with the web. We are experts in the usage of XHTML, CSS, and other key web technologies. Whether you have an aging site that could use a facelift or you need a new site built from the ground up, our designers, developers, and communications experts will help your business or organization achieve the future-ready web presence that you need to succeed in this digital age.

  Tags: accessibility css web xhtml