Django admin converts HTML entities into their characters

Question

I have a strange problem happening with the Django Admin site, where I store fragments of html inside a text field.

enter image description here

Once I hit save, the field becomes this:

enter image description here

Which is fine -- the HTML renders correctly anyway. The real problem is when I hit save again:

enter image description here

All HTML entities are affected by this bug, including  .

Why is this happening?

OS: Amazon Linux x64
Python: 2.6
Django: 1.4
Database: MySQL 5.5

As a string, I want my HTML entities in it to remain unmodified. — Brian, Aug 02 '13 at 19:08
What you want to do is that if you put this `→` stays in `→` not in the right arrow? — Victor Castillo Torres, Aug 02 '13 at 19:42
See this answer then http://stackoverflow.com/questions/2080559/disable-html-escaping-in-djangos-textfield — Victor Castillo Torres, Aug 03 '13 at 02:00
Your proposal is well-known, and I understand. However, this is a problem specific to [THE Django Admin site](https://docs.djangoproject.com/en/dev/ref/contrib/admin/), whose source code is beyond my control post-deployment. I was asking Stack Overflow if my any part of my set-up introduces this behaviour, because this bug is not known to exist across all platforms. — Brian, Aug 03 '13 at 02:52

dan-klasson · Accepted Answer · 2013-08-09T00:00:55.647

1

Your problem is that Django inserts your data as unicode but your mysql stores it as latin_swedish.

If you're not using South just drop your database with DROP DATABASE foo;. Then re-create it using CREATE DATABASE foo CHARACTER SET UTF8; and run syncdb again.

Alternatively you can convert your existing database by doing:

ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE tablename CHARACTER SET utf8 COLLATE utf8_general_ci;

But then you would have to apply that manually for each table. Although I think it's possible to only apply it for the table you're experiencing problems with.

Make sure you back your data up before doing it.

edited Aug 09 '13 at 00:00

answered Aug 07 '13 at 06:17

dan-klasson

13,734
14
63
101

Tried it, I think we're heading the right direction, but this does exactly the opposite as intended: now every kind of entity is converted to its character. The correct solution consists of NO HTML entity being shown as their represented characters. – Brian Aug 08 '13 at 15:41
Actually I think I might be dead wrong. Can you please do `SHOW TABLE STATUS` after selecting your db. What does the `Collation` row say for your tables? – dan-klasson Aug 08 '13 at 16:15
[latin1_swedish_ci](http://stackoverflow.com/questions/6769901/why-is-mysqls-default-collation-latin1-swedish-ci), possibly by default... – Brian Aug 08 '13 at 16:44
The db is mission critical, so we'll have to wait a bit before we try this out. Will definitely report back if it works! – Brian Aug 08 '13 at 23:46
changed my answer again – dan-klasson Aug 09 '13 at 00:01
Nope. Altering table collation, table engine, or both, showed no sign of removing the problem. :/ – Brian Sep 08 '13 at 18:18
What is the collation for your tables now? – dan-klasson Sep 09 '13 at 14:05
It might be an AWS bug. Amazon is investigating - more to come. – Brian Sep 26 '13 at 00:04

Django admin converts HTML entities into their characters

1 Answers1