0

I have a file that fetches descriptions from the DB. The values have special characters in them. So while writing the same into the file after converting to DOS the characters change to something else.

So as a correction i have used sed command to replace the converted characters to the original special character and it works. I could do for all the special characters that were present in DB.

Examples :

Original : CANDELE 50°

After conversion to dos its visible as CANDELE 50Áø .So i used a sed command sed -e 's/Áø/°/g'

What i want to do now is a permanent fix to automatically change any special character that comes in. is there any command that automatically converts a special character to its original after conversion to dos so that I can avoid a manual addition for every character.

Kindly help me doing the same:

 changeFileFormat () {
    cd $1

    echo "Changing file format Unix -> DOS." >> $LOG_FILE
    for file in `ls *.csv`
    do  
        mv ${file} ${file}_unix
        unix2dos ${file}_unix >> ${file}_dos
        sed -e 's/ÁøU/°/g' -e 's/Âû/Ó/g' -e 's/‹¨«//g'  -e 's/ª/ì/g' -e 's/¸/ù/g' -e 's/Áœ/£/g' -e 's/Á¨/¿/g' -e 's/ƒâª/€/g' ${file}_dos >> ${file}
        if [ $? -ne 0 ]; then
            echo "Conversion failed for file: [ $file ]." >> $LOG_FILE
            mv ${file}_unix ${file}
        else
            rm -f ${file}_dos
            rm -f ${file}_unix
        fi;
    done
    echo "Conversion finished." >> $LOG_FILE
}
  • DB description : CANDELE 50°
  • CSV file that gets created in unix : ART|M|02A_1057M5706 |CANDELE 50°
  • After DOS conversion : ART|M|02A_1057M5706 |CANDELE 50Áø
  • After SED command : ART|M|02A_1057M5706 |CANDELE 50°
Shil N
  • 35
  • 1
  • 1
  • 7
  • Can you provide an example here ? I mean 2-3 lines of your csv file. – smilyface Dec 03 '15 at 06:39
  • DB description : CANDELE 50° CSV file that gets created in unix : ART|M|02A_1057M5706 CANDELE 50° After DOS conversion : ART|M|02A_1057M5706 |CANDELE 50Áø After SED command : ART|M|02A_1057M5706 |CANDELE 50° – Shil N Dec 03 '15 at 07:18
  • 1
    The file apparently is in UTF-8 encoding, simply convert it to UTF16-LE or Western-European 1252. – wOxxOm Dec 03 '15 at 07:33
  • @wOxxOm : The final file has to be in DOS format and therefore i cannot use NLS LNG and convert it to Western-European 1252 – Shil N Dec 03 '15 at 08:11
  • There's no such thing as `DOS format` (also DOS? not Windows?). The file is a plain text which can be saved in any encoding. Where/how do you plan to use it? – wOxxOm Dec 03 '15 at 08:16
  • Just a quick hint... there are LOTS of errors in your script - try pasting all except the first and last line into http://shellcheck.net so you can get them all identified and fix them. – Mark Setchell Dec 03 '15 at 08:18
  • @wOxxOm : Ok. What i'm trying to say is, i can convert ° to ° using NLS lang. However since there is a unix2ods command in the script present already it changes it back to a special character again. My final file has to be in windows format. Kindly help – Shil N Dec 03 '15 at 08:44
  • [Best way to convert text files between character sets?](http://stackoverflow.com/q/64860) – wOxxOm Dec 03 '15 at 08:47

0 Answers0