A simple FormAlchemy tutorial
December 4th, 2009
Here’s a quick tutorial on the usage of FormAlchemy with Elixir and Pylons.
The model
To illustrate how things work, we will take a very simple example.
Let’s say you have a website where people have to sign in.
Here’s a very simple User class
from elixir import Entity
# Field represents bounds an attribute to a column in your table
from elixir import Field
# Unicode, Integer, DateTime etc. are column types. Pay attention to encoding when you’re using Unicode.
from elixir import Unicode
# This will allow us to represent relations between classes. Elixir will figure out how to represent them
# in the database, between the tables
from elixir import ManyToOne, OneToMany
import datetime # we will use this to put default values in DateTime columns
# Elixir will create a table for each class inherting form Entity
class User(Entity):
# By default, your table names will look like "ModuleName_ClassName".
# To change this, you can either pass in a shortnames=True argument to
# the using_options function, which will tell elixir to create your tables
# based on ClassName only, or a tablename=’My_Table_Name’ argument.
# I’m used to have plural names for tables, and single names for classes,
# so I’ll pass in a tablename arguemnt
# Next thing is the inhertince argument. This will tell Elixir
# that User can be subclassed.
using_options (tablename=“Users”,inheritance=“multi”)
# I like powers of 2
username = Field(Unicode(32))
email = Field(Unicode(64))
password = Field(Unicode(32))
phone = Field(Unicode(16))
address = Field(Unicode(64))
# Many users may live in One City, so in this case it’s Many(users)ToOne(City)
city = ManyToOne(“City”)
creation_date = Field(DateTime,default=datetime.datetime.now)
# Let’s make a distinction between Persons and Companies. Both are users, wich means they share a certain amount of
# properties, but they can also have different additional attribtues of their own.
class Person(User):
using_options (tablename=“Persons”,inheritance=“multi”)
first_name = Field(Unicode(32))
last_name = Field(Unicode(32))
def __str__(self):
“”” This method is called when you try to print a user.”“”
return “%s %s (%s)” % (self.first.encode(“utf-8″), self.last.encode(“utf-8″), self.username.encode(“utf-8″))
def __repr__(self):
“”” This method is called when you try to print a representation of a user.”“”
return “” % str(self)
class Company(User):
using_options (tablename=“Users”,inheritance=“multi”)
logo = Field(String(32)) # we store only the filename here.
name = Field(Unicode(64))
website = Field(Unicode(64))
Things are looking pretty good so far ! Now let’s write a simple registration form.
The mako template
Let’s begin with the mako template :
<form action="${h.url_for(action=’processRegistrationForm’,controller=’registration’)}">
${c.form}
<input type="submit" value="go" />
</form>
Very simple code. c.form will contain the html code of the form, generated by formalchemy.
On to the controller action.
The controller
We’ll create a controller named registration for this purpose. Don’t forget to activate your virutal environement.
Now let’s edit controller/registration.py
from pylons import request, response, session, tmpl_context as c
from pylons.controllers.util import abort, redirect_to
from myapp.lib.base import BaseController, render
# These imports up there are generated for you by paster
# Now you’ll need two other imports to generate the form : FieldSet and your model’s Person
# This is a customized FieldSet that will render itself following the template that you’ll find
# in templates/forms/fieldset.mako.
# I have edited that template to use semantic html elements like dl, dt and dd instead of bare div elements.
from myapp.forms import FieldSet
# This represents a form field, we will use this to create a password check form field.
from formalchemy import Field
from myapp.model import Person
log = logging.getLogger(__name__)
class RegistrationController(BaseController):
def personRegistrationForm(self):
“”“
Call this action in your mako templates to render the registration form
““”
fs = self.getPersonFieldSet()
c.form = fs.render()
return render(“/pages/registration.mako”)
def processRegistrationForm(self):
“”“
Put this method in your registration’s form’s action attribute.
<form action=”/registration/processRegistrationForm“>
or use
<form action=”${h.url_for(controller=‘registration’,action=‘processRegistrationForm’)}“>
““”
fs = self.getPersonFieldSet()
# This will populate the FieldSet with values found in request.params
fs.data = request.params
# This will populate the Person instance that FieldSet stores as an attribute named "model"
fs.sync()
# Here’s your object :) !
user = fs.model
# now do whatever you want with it. For this example, we’ll just return a simple message
# saying that everything was ok.
return “User (%s) successfully created” % str(user) # This will call user.__str__
def getPersonFieldSet(self):
“”“
Return the fieldset corresponding to a Person
““”
# Just pass in the class, not a specific instance
fs = FieldSet(Person)
# Now let’s add a password check field.
# name is the attribute name that will be generated
#
# By default, formalchemy will append the class name as a prefix to every field name.
# This is to prevent confusion when two classes may have the same name for one of their attribute.
pwdcheck= Field(name=“pwdcheck”,type=Unicode)
# We’ll add it to the FieldSet
fs.append(pwdcheck)
# Now we’ll customize the generated HTML with the conigure method.
# the include argument tells FieldSet objects what fields you actually want to render.
# A FieldSet consists of a group of Fields. Each field is accessed as an attribute of the FieldSet
# it’s contained within. The attribute name of that field is the same as the Field name you used
# in your class declaration.
fs.configure(include=[fs.first,fs.last,fs.username,fs.city,fs.adresse,fs.email,
# Now here we’ll tell fs.password to render itself as a password field
# instead of a usual text field. This will return the modified field.
fs.password.password(),
# Now we’ll configure the pwdcheck field to have a meaningful and helpful label
fs.pwdcheck.password().label(“Confirm your password”)])
return fs
That’s it :) you’re ready to go ! This is actually not too much code, let’s strip the comments to see how much code we needed.
Let’s recap’
the model code, model/__init__.py
import datetime
class User(Entity):
using_options (tablename=“Users”,inheritance=“multi”)
# I like powers of 2
username = Field(Unicode(32))
email = Field(Unicode(64))
password = Field(Unicode(32))
phone = Field(Unicode(16))
address = Field(Unicode(64))
city = ManyToOne(“City”)
creation_date = Field(DateTime,default=datetime.datetime.now)
class Person(User):
using_options (tablename=“Persons”,inheritance=“multi”)
first_name = Field(Unicode(32))
last_name = Field(Unicode(32))
def __str__(self):
“”” This method is called when you try to print a user.”“”
return “%s %s (%s)” % (self.first.encode(“utf-8″), self.last.encode(“utf-8″), self.username.encode(“utf-8″))
def __repr__(self):
“”” This method is called when you try to print a representation of a user.”“”
return “” % str(self)
class Company(User):
using_options (tablename=“Users”,inheritance=“multi”)
logo = Field(String(32)) # we store only the filename here.
name = Field(Unicode(64))
website = Field(Unicode(64))
the mako template
<form action="${h.url_for(action=’processRegistrationForm’,controller=’registration’)}">
${c.form}
<input type="submit" value="go" />
</form>
The Fieldset.mako template
The interresting part of the mako template for the form generation, templates/forms/fieldset.mako.
Here, I consider the form as a definition list, every label is a definition term, and the inputs are data definitions.
% if field.requires_label:
<dl>
<dt>
<label class="${field.is_required() and ‘field_req’ or ‘field_opt’}" for="${field.renderer.name}">${[field.label_text, fieldset.prettify(field.key)][int(field.label_text is None)]|h}</label>
</dt>
<dd>
${field.render()|n}
</dd>
% if ‘instructions’ in field.metadata:
<span class="instructions">${field.metadata[’instructions’]}</span>
% endif
% for error in field.errors:
<span class="field_error">${_(error)}</span>
% endfor
</dl>
The controller
The controller’s code, controller/registration.py. This is where formalchemy comes into play.
from pylons import request, response, session, tmpl_context as c
from pylons.controllers.util import abort, redirect_to
from formalchemy import Field
from myapp.model import Person
from myapp.forms import FieldSet
log = logging.getLogger(__name__)
class RegistrationController(BaseController):
def personRegistrationForm(self):
“”“
Call this action in your mako templates to render the registration form
““”
fs = self.getPersonFieldSet()
c.form = fs.render()
return render(“/pages/registration.mako”)
def processRegistrationForm(self):
“”“
Put this method in your registration’s form’s action attribute.
““”
fs = self.getPersonFieldSet()
fs.data = request.params
fs.sync()
user = fs.model
return “User (%s) successfully created” % str(user) # This will call user.__str__
def getPersonFieldSet(self):
“”“
Return the fieldset corresponding to a Person
““”
fs = FieldSet(Person)
pwdcheck= Field(name=“pwdcheck”,type=Unicode)
fs.append(pwdcheck)
fs.configure(include=[fs.first,fs.last,fs.username,fs.city,fs.adresse,fs.email,
fs.password.password(),
fs.pwdcheck.password().label(“Confirm your password”)])
return fs
Column ‘column’ already exists in table ‘table’
November 26th, 2009
This is an idiot mistake.
I wanted to create 3 Entities, all sharing a certain amount of behaviour and properties (that is methods and attributes). So I decided to a create a BaseClass where the common code is shared, then create 3 Classes that subclass the later.
So how’s the code gonna look like ?
At first guess, I thought “why would I subclass all my classes from Entity AND BaseClass ? wouldn’t it be better to just subclass BaseClass and then subclass BaseClass from Entity ? thus allowing all the subclasses of BaseClass be Entities ?”
#Some useful methods and attributes here
#…
class SubClassOne(BaseClass):
# Your actual class
# …
class SubClassTwo(BaseClass):
# Your class here
# …
#… more subclasses
Bad idea ! If you’re doing this, you are defining BaseClass as an entity, that will be created in the database. And so you’ll get an Exception like
when elixir will try to create the second subclass.
So, no, the only way to do it right is to subclass from Entity AND BaseClass.
#Some useful methods and attributes here
#…
class SubClassOne(BaseClass,Entity):
# Your class here
# …
class SubClassTwo(BaseClass,Entity):
# Your class here
# …
#… more subclasses
Using beaker as a second level cache for sqlalchemy in pylons (better approach)
November 23rd, 2009
With the help of Ben bangert, I rewrote my code this way :
def __call__(cls,key):
“”“
If it’s in the cache, return the cached version
If not in the cache :
If it’s in the database, retrieve it, cache it and return it
If it’s not there, create it, cache it and return it
““”
thecache = cache.get_cache(cls.__name__,type=“memory”)
def makeTag():
theTag = cls.query.filter_by(**{cls.filterby:key}).first()
if not theTag:
#not in the database either
theTag = type.__call__(cls,key)
session.add(theTag)
return theTag
return thecache.get_value(key=key,createfunc=makeTag)
class Ville(Entity):
using_options (tablename=“Villes”)
__metaclass__ = UniqueMeta
nom = Field(Unicode(64))
filterby = “nom”
def __init__(self,nom,*args,**kw):
Entity.__init__(self,*args,**kw)
self.nom = nom
def __repr__(self):
return “” % self.nom
class TypeBien(Entity):
using_options (tablename=“TypesBiens”)
__metaclass__ = UniqueMeta
type = Field(Unicode(64))
filterby = “type”
def __init__(self,type,*args,**kw):
Entity.__init__(self,*args,**kw)
self.type = type
def __repr__(self):
return “” % self.type
There’s an improvement here over the UniqueObject recipe : more classes can use the UniqueMeta metaclass if they define a filterby class attribute.
Using beaker as a second level query cache for SQLAlchemy in Pylons
November 21st, 2009
The code below is outdated, please see a recent version here
SQLAlchemy doesn’t provide a cahing mechanisme, because the designers thought that this is not an ORM’s business after all, and I think they are right.
Pylons offers you a way to cache arbitrary objects. This is handy to store server sessions objects, or, as described here, your SQLAlchemy mapped objects.
Now, I am a fan of the O3 principle, so I’m not arguing about speed performances/overheads, i’m arguing about doing things Once, and Only Once (O3), because I was taught to do so in my computing education.
It is frequent in applications to get the same rows from the database that have not been changed over and over again. Fortunatly, almost all databases are smart enough to figure that and implement a query cache to handle this situation. This means that if you repeat exactly the same query during a certain laps of time, it will return the results from the in-memory cache, not from the disk, which is times faster.
But some people do not like to talk to the database if the exact same queries have been processed in the near past. They like to implement an applicaiton-level cache, also referred to as a “second level cache”.
This is what I have tried to do with SQLAlchemy in a Pylons application. And here’s the result :
from pylons import cache
class MetaCity(EntityMeta):
cache = cache.get_cache(“cities”,type=“memory”)
def __call__(cls,name):
“”“
If it’s in the cache, return the cached version
If not in the cache :
If it’s in the database, retrieve it, cache it and return it
If it’s not there, create it, cache it and return it
““”
theCity = MetaCity.cache.get_value(key=name,createfunc=lambda:None)
if not theCity :
#not in the cache
theCity = cls.query.filter_by(name=name).first()
if not theCity:
#not in the database either
print “not in the database”
theCity = type.__call__(cls,name)
session.add(theCity)
#Adding it to the cache, after creating it in the database if it wasn’t there
MetaCity.cache.set_value(key=name,value=theCity)
return theCity
class City(Entity):
__metaclass__ = MetaCity
using_options (tablename=“Cities”)
name = Field(Unicode(64))
def __init__(self,name,*args,**kw):
Entity.__init__(self,*args,**kw)
self.name = name
def __repr__(self):
return “” % self.name
metadata.bind = “mysql://username:password@localhost:3306/db”
metadata.bind.echo = True
And here’s the model I played with on my ipython interpreter (using paster shell).
NOTA : Ville is french for City and Villes (plural) for Cities. I also added some prints here and there to show where the objects come from.
VIRTUAL_ENV -> /home/chaouche/PYTHONENV/lib/python2.6/site-packages
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
Type “copyright”, “credits” or “license” for more information.
IPython 0.9.1 — An enhanced Interactive Python.
? -> Introduction and overview of IPython’s features.
%quickref -> Quick reference.
help -> Python’s own help system.
object? -> Details about ‘object’. ?object also works, ?? prints more.
All objects from interimmo.lib.base are available
Additional Objects:
mapper - Routes mapper object
wsgiapp - This project’s WSGI App instance
app - paste.fixture wrapped around wsgiapp
In [1]: from someproject.model.somemodel import *
In [2]: Ville(”Rouiba”)
not in the cache…
23:00:31,473 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] BEGIN
/home/chaouche/PYTHONENV/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/engine/default.py:230: SAWarning: Unicode type received non-unicode bind param value ‘Rouiba’
param.append(processors[key](compiled_params[key]))
23:00:31,480 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] SELECT `Villes`.id AS `Villes_id`, `Villes`.nom AS `Villes_nom`
FROM `Villes`
WHERE `Villes`.nom = %s
LIMIT 0, 1
23:00:31,480 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Rouiba’]
not in the database
adding it to the cache
Out[2]:
In [3]: Ville(”Rouiba”)
Out[3]:
In [4]: Ville(”Rouiba”)
Out[4]:
In [5]: Ville(”Kolea”)
not in the cache…
23:00:54,416 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] INSERT INTO `Villes` (nom) VALUES (%s)
23:00:54,416 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Rouiba’]
/home/chaouche/PYTHONENV/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/engine/default.py:230: SAWarning: Unicode type received non-unicode bind param value ‘Kolea’
param.append(processors[key](compiled_params[key]))
23:00:54,656 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] SELECT `Villes`.id AS `Villes_id`, `Villes`.nom AS `Villes_nom`
FROM `Villes`
WHERE `Villes`.nom = %s
LIMIT 0, 1
23:00:54,656 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Kolea’]
not in the database
adding it to the cache
Out[5]:
In [6]: Ville(”Babezzouar”)
not in the cache…
23:01:15,955 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] INSERT INTO `Villes` (nom) VALUES (%s)
23:01:15,955 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Kolea’]
/home/chaouche/PYTHONENV/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/engine/default.py:230: SAWarning: Unicode type received non-unicode bind param value ‘Babezzouar’
param.append(processors[key](compiled_params[key]))
23:01:15,962 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] SELECT `Villes`.id AS `Villes_id`, `Villes`.nom AS `Villes_nom`
FROM `Villes`
WHERE `Villes`.nom = %s
LIMIT 0, 1
23:01:15,962 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Babezzouar’]
not in the database
adding it to the cache
Out[6]:
In [7]: Ville(”Babezzouar”)
Out[7]:
In [8]: Ville(”Babezzouar”)
Out[8]:
In [9]: Ville(”Alger”)
not in the cache…
23:01:32,024 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] INSERT INTO `Villes` (nom) VALUES (%s)
23:01:32,024 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Babezzouar’]
/home/chaouche/PYTHONENV/lib/python2.6/site-packages/SQLAlchemy-0.5.5-py2.6.egg/sqlalchemy/engine/default.py:230: SAWarning: Unicode type received non-unicode bind param value ‘Alger’
param.append(processors[key](compiled_params[key]))
23:01:32,029 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] SELECT `Villes`.id AS `Villes_id`, `Villes`.nom AS `Villes_nom`
FROM `Villes`
WHERE `Villes`.nom = %s
LIMIT 0, 1
23:01:32,030 INFO [sqlalchemy.engine.base.Engine.0x…5d6c] [’Alger’]
adding it to the cache
Out[9]:
In [10]: Ville(”Alger”)
Out[10]:
In [11]: Ville(”Kolea”)
Out[11]:
In [12]: Ville(”Rouiba”)
Out[12]:
A small non-techi introduction to SQLite
November 25th, 2008
db_row vs dtuple
August 29th, 2008
MySQLdb result sets do not offer a nice way to acess information : tuples. This means that whenever you change your SQL query, you have to modify the code that extract information from the data set because the position of elements may have changed.
result[0] was for example Name but now it is Password.
If you google a bit, you will pass by dtuple and db_row. Which one should you use ? I googled for a comparison of the two but found nothing in the first results. So here’s a little description of what I have found so far after testing the two :
dtuple is flexible, but column names are case sensitive. That means that row.Name or row[’Name’] is ok but not row.name nor row[’name’]. In the other hand, the results are mutable, this means that you can add some other field in the result set if another function or method needs it. On the contrary, db_row does not allow you to change the result set because they are immutable (they use python’s __slot__ thing that prevent you from adding new attributes to your object). The good thing is that this is faster because attribute lookup is boosted. Besides, db_row is memory friendly because it does not create an object per row. This means that if your SELECT returns 100.000 lines, it wont create 100.000 objects. db_row allows also case-insensitive key/attribute fetching, as a little “syntactic sugar”.
There’s a third alternative, sqlWrap, which offers very nice reporting solutions, like export to xhtml, pretty printing, and export to restructuredText. I did not try this one but looks fun.
Replace a string in multiple files
August 28th, 2008
Just a little memo…
One handy alias :
Emacs is great for prototyping
August 14th, 2008
Split the window as you wish, this allows you to see different files at the same time to rapidly look for the information you need. I found this really useful yesterday when I was stuck on my application model. I needed to have a snapshot of the different classes and objects, when and where to store things in the session, and when to retrieve them etc. This window splitting system comes in handy when you need to do a global refactoring of your code and review the interactions and relations between your various objects.
Convert non-ascii characters to html entites with Tidy
July 17th, 2008
–input-encoding : your encoding (actually supports utf-8, latin1 and latin15)
–ouptput-enconding : specifying ascii here forces tidy to convert non ascii chars to the corresponding html entities.
notice that
will output non-ascii characters as they are (no conversion) while
will break you characters because he does not take the input file with the appropriate encoding.
That’s why you have to specify –input-encoding and –output-encoding to get the job done.
Paste and execute code on the web
June 8th, 2008
Why anybody thought about it before ? http://codepad.org/






























