0

In my ruby on rails application,

When i dump data in to my data base from file like (CSV, txt, XLS) it display special character like D�e�m�o in my production database. But in my file the data is "Demo". So why it happens. It looks like my real time data is corrupted.

I am using,

Ruby 1.9.2p180 
Rails 3.0.5 
Phusion Passenger version 3.0.13
Mysql

It still displays in my browser:

enter image description here

Why it happens?

Please help

Thanks in advance

Sumit Munot
  • 3,748
  • 1
  • 32
  • 51

1 Answers1

2

You're probably using an incorrect charset for your table (see here), connection (see here) or you aren't specifying one at all. Also, the file (CSV, TXT or XLS for example) could be in a specific encoding and be read (e.g. "interpreted") with another (incorrect) encoding.

Before you start changing stuff around, guessing the correct charset to use, you should read The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) to save yourself a lot of headaches and frustration.

RobIII
  • 8,488
  • 2
  • 43
  • 93
  • Thanks, i am using UTF-8 encoding in my application while dumping data in to database. I googled and found it happens due to UTF-16 encoding. But it's not worked for me still am having the same problem. – Sumit Munot Mar 26 '13 at 11:26
  • 1
    Did you read the article I linked to? You're saying you're using UTF-8 in your *application*. So what about the database? Is it in UTF-8 too? And the actual table? And did you set the correct charset on your connection? And how about the file(s)? Are they all UTF-8 as well? I've provided plenty of links in my answer and the one with the longest title ending in "No excuses" is in there with a reason (as well as the 'No excuses!' part in the title). Also, when you say *"But it's not worked for me"* you should at least specify *what* you changed. – RobIII Mar 26 '13 at 11:28
  • Yes, i have checked links and it explained nicely +1 for your efforts. But in my application i am using same encoding in data base as well. My files are also in UTF-8 charset. – Sumit Munot Mar 26 '13 at 11:31
  • 1
    So your files are in UTF-8, your database (and table!) is in UTF-8, your connection is set to UTF-8 and Ruby uses UTF-16 as default (I'm assuming this from your comment(s), I'm not familiar with Ruby other than reading some blogposts about it sometimes). See where I'm [hinting at](http://stackoverflow.com/q/5908774/215042)? Either way, *somewhere* in the chain there's a different encoding so you'll need to start checking the entire chain, step-by-step. You'll find your problem for sure. Also, to be honest, I think you *"read"* that article pretty fast ;-) Are you sure you **understand** it? – RobIII Mar 26 '13 at 11:33
  • I am checking it on my local system. By setting default encoding to UTF-8 as mentioned in link. As my production server has important data. So it will takes some time to check on live. – Sumit Munot Mar 26 '13 at 11:39
  • Also, did you try googling for an answer? I found one in a blink of the eye: [ruby mysql utf8](https://www.google.com/search?q=ruby+mysql+utf8): [here](https://my.a2hosting.com/knowledgebase.php?_m=knowledgebase&_a=viewarticle&kbarticleid=794). Seems you need to change `database.yml` as well (which makes sense I guess). – RobIII Mar 26 '13 at 11:40
  • I have found my problem in the above mentioned link. Now i am implementing it. Lets see what happens. database.yml represents encoding UTF-8 in mycase. – Sumit Munot Mar 26 '13 at 11:40
  • Now again it happens, please see my update question screenshot of my browser – Sumit Munot Mar 26 '13 at 11:56
  • 1
    Sceenshots show nothing but the problem; what we need is *relevant* code / settings. This includes the charset used in your HTML output (e.g. ``), code where you set connection charsets (e.g. `db.options(Mysql::SET_CHARSET_NAME, 'utf8')`), alter/create-table scripts that define DB/table charsets, the aforementioned `database.yml` etc. As I've regurgitated about 15 times by now: **any one of those** (and more) can be the source of your problem. Your problem is (and was) perfectly clear, even without the screenshot. – RobIII Mar 26 '13 at 11:58