In a traceback you may see the error message: UnicodeDecodeError: 'ascii' codec can't decode byte...
Mako uses unicode internally. This error usually occurs when you've given Mako a non-ascii raw string. Upon converting the raw string to unicode (via unicode(your_str)), the UnicodeDecodeError is raised.
Make Mako input (and output) UTF-8
You can tell Mako the encoding of strings in the template by specifying a "magic encoding comment" at the top of the file (like you would with a .py file):
1 | # -*- coding: utf-8 -*-
|
Alternatively, the encoding can be specified for all mako templates in your Pylons project by adding the following to the end of your project's load_environment function in the YOURPROJECT/config/environment.py file:
1 | tmpl_options['mako.input_encoding'] = 'utf-8' |
By default, Pylons configures Mako to output to a UTF-8 encoded raw string (i.e. when you call render() on a mako template). This would be equivalent to the following code in the same file:
1 | tmpl_options['mako.output_encoding'] = 'utf-8' |
Make sure you use proper unicode strings
Specifying the encoding of the template tells Mako about the strings defined within the template. This would solve UnicodeEncodeErrors for a template such as:
1 2 3 4 5 6 | <html> <head><title>voix m'a réveillé</title></head> <body> ${u"voix m'a réveillé."} </body> </html> |
However Mako cannot and will not assume the encoding of raw strings passed to it via expressions, such as:
1 | ${h.submit(value='Änderungen speichern')} |
Mako will internally call unicode() on the output of h.submit in this case (which would be a raw string containing non-ascii characters), causing a UnicodeDecodeError. Non-ascii strings used in an expression must be unicode:
1 | ${h.submit(value=u'Änderungen speichern')} |
If the request parameters (request.params) are being used in Mako expressions, they can be converted to Unicode automatically for you. Pylons 0.9.6 does this by default; to enable automatic unicode conversion of request parameters in versions prior to 0.9.6, modify the pylons.config.Config constructor line in load_environment to the following:
1 2 | return pylons.config.Config(tmpl_options, map, paths, request_settings=dict(charset='utf-8', errors='replace')) |
Alternatively...
Mako's calling of unicode(data) occurs because the unicode function acts as Mako's default filter. You can change the default filter to decode raw strings to 'utf-8' via the decode.utf8 filter. However, this is a lazy way of solving this issue, has performance repercussions, and is NOT recommended:
1 | tmpl_options['mako.default_filters'] = ['decode.utf8'] |
 | From the mako docs: Note that the built-in decode object is slower than the unicode function, since unlike unicode its not a Python builtin, and it also checks the type of the incoming data to determine if string conversion is needed first. |
See also:
The Mako Unicode Chapter
Pylons Unicode Document
Pylons Unicode Document - Request Parameters
Christoph Haas, Ksenia Marasanova, Philip Jenvey
Comments (2)
Mar 16, 2008
Maciej Litwiniuk says:
Alternative way with adding .syntax { background: #f8f8f8; } .syntax .c { col...Alternative way with adding
at the end of your environment.py is required, if you wan't to use things like
{c.news.title}inside your template, where "news" if object fetched from DB, and title is unicode string.
Mar 04, 2009
Marius Gedminas says:
I think you mean "and title is a UTF-8 string". You can tell SQLAlchemy whether...I think you mean "and title is a UTF-8 string".
You can tell SQLAlchemy whether you want Python unicode objects or Python str objects with 8-bit characters in your model definition, by choosing Unicode() versus String() columns. With Unicode() there's no need to change mako's default_filters.