Making Django Environmentally Friendly
If you've ever needed to use the Django database API or templating system from a standalone Python script, you've probably written a quick workaround or ugly hack to get the Django environment working properly. Specifically, this problem has to be solved in order to run Django scripts as cron jobs. Some possible solutions to this problem were written up by James Bennet and Scott Newman. I prefer the clean approach in this method documented by Jared Kuolt.
Jared uses an undocumented function in django.core.management called setup_environ to prepare a script so that Django can be used normally, just like you would use it in any of your website's view modules. It goes like this:
from django.core.management import setup_environ
import settings
setup_environ(settings)
The setup_environ function simply takes your project's settings module as a parameter and sets up Django accordingly. Search for this function in django/core/management.py if you want to see the details of how Django sets itself up. Though this method is simple, elegant, and decently abstracted, it has two main limitations. It may only be a three-line incantation, but it's a pain to memorize these sort of things, and it would really be ideal if this could be accomplished in one line. More importantly, in order for this method to work, your script's file must live in the root of your project, so that "settings.py" is its neighbor. I prefer to keep my cron job scripts or data importing scripts in an appropriately named subdirectory, such as "scripts." From a subdirectory, import settings will not work.
I've written a script to solve this problem so that you only need to write one line of code that's easy to memorize. You'll need to copy and unzip this script, setup_django.py, into the directory where your script lives. Write this as the first line of your script:
import setup_django
And voila! That's all you need to do. You can now use the Django database API just as you'd expect. The code borrows some ideas directly from the setup_environ function in django.core.management. It goes like this:
import os, sys, re
# stop after reaching '/' on nix or 'C:\' on Windows
top_level_rx = re.compile(r'^(/|[a-zA-Z]:\\)$')
def is_top_level(path):
return top_level_rx.match(path) is not None
def prepare_environment():
# we'll need this script's directory for searching purposess
curdir, curfile = os.path.split(os.path.abspath(__file__))
# move up one directory at a time from this script's
# path, searching for settings.py
settings_module = None
while not settings_module:
try:
sys.path.append(curdir)
settings_module = __import__('settings', {}, {}, [''])
sys.path.pop()
break
except ImportError:
settings_module = None
# have we reached the top-level directory?
if is_top_level(curdir):
raise Exception("settings.py was not found in the script's directory or any of its parent directories.")
# move up a directory
curdir = os.path.normpath(os.path.join(curdir, '..'))
# set up the environment using the settings module
from django.core.management import setup_environ
setup_environ(settings_module)
prepare_environment()
When you import this module, it runs prepare_environment. First, the function calculates its own path. It then takes this path and moves upward one directory at a time, attempting to import a module named "settings." At each step, it checks if it has reached the root directory using the is_top_level function, which employs a simple regular expression for detecting the root directory on Unix or Windows. If the root has been reached, an exception is raised; otherwise, when the settings module is found, it is passed to Django's setup_environ function.
Some may not find this method to be as nice as others, because you'll need to copy the "setup_django.py" script into all of your script directories, but the one-line import setup_django solution is very nice. The main caveat to consider is that if you have a file named "settings.py" other than the project's main "settings.py" in the directories above your script's path, this method will not work. This was only tested in Python 2.5.x, but there shouldn't be any problems running it in slightly older versions of Python.
Enjoy!
All Articles
I have an idea: take a simple, pre-built solution that works perfectly, and overcomplicate it for no reason.
Hello there!
I'm still not sure how to use this - suppose I have project located in directory 'testproject'. I then add some applications etc, well, it doesn't matter now. But I have my models.py located in one of this application. Normally, when I start shell via 'python manage.py shell' from root directory of my project i have to import models like 'from app1.models import Model1' and it works just fine.
And here is my problem: I have created directory 'scripts' in root dir of my project, then copied your script into this 'scripts' dir, then tried to run suppose_to_be_standalone_script.py which looks like (for example):
==========================
#!/usr/bin/python
# encoding utf-8
import setup_django
from app1.models import Model1
print Model1.objects.all()
==========================
... and it doesn't work (rises exception that module app1.models couldn't be found. It's probably some basic stuff (at least I guess it should be) but still I cannot figure out what is wrong, so if you can help me, pls do.
When using sqlite3, if you want to avoid using an absolute path to the db file, you can drop this into settings.py (assuming the db file is in the same dir as the settings):
import os
settings_path = os.path.dirname(__file__)
DATABASE_NAME = os.path.join(settings_path,'site.db')
Just a suggestion about your code: Move "sys.path.pop()" to the except portion.
try:
sys.path.append(curdir)
settings_module = __import__('settings', {}, {}, [''])
#sys.path.pop() # Unreachable when an exception rises
break
except ImportError:
sys.path.pop() # Unset path only if not settings found in dir
settings_module = None
This way,
1) Sys.path does not get cluttered with unuseful paths (every directory that *does not contain* settings.py is added to the path with the previous code, as sys.path.pop() is unreachable when an exception rises).
2) When settings.py is found the directory that contains it is added to the path, allowing import of classes and methods exactly as if you were working inside your django project.
(Note: Remember to indent the code, the comment system of this blog doesn't allow whitespaces at beginning of line =P)
Just a note to anyone using sqlite as a db backend- be sure to use an absolute path when defining your db location in settings.py.
I ran into a peculiar bug using this script because I used a relative path off of /project/. When using this to run a python script in /project/script/ it couldn't find the db so it recreated an empty one in the current directory (/project/script/).
Thanks so much - it works like a charm.
Django is mentioned quite a bit at work - you know, because we build Django sites. However, this pop culture reference crept up twice this weekend.(Django is named after Django Reinhardt: http://en.wikipedia.org/wiki/Django_Reinhardt)
Django Haskins played a show in Durham: http://www.djangohaskins.com/
David Crosby was featured on CBS Sunday morning and his son is named Django: http://www.ceder.net/pictures/view.php4?SetId=46
Two times, I thought about my developer friends at work when I was trying to relax. Rats.
Post a comment
We look forward to hearing what you have to say. Before joining the conversation, please take a moment to review our comment policy.