Up until now my INI text files have been Unicode encoded and I have been doing things like this to read it:
::GetPrivateProfileStringW(
                strUpdateSection,
                strTalkKey,
                _T(""),
                strTalk.GetBuffer( _MAX_PATH ),
                _MAX_PATH,
                m_strPathINI );
strTalk.ReleaseBuffer();
The file gets downloaded from the internet first and then accessed. But I was finding that for Arabic the text file was getting corrupted. Unless I change the encoding to UTF-8. When I do that, it downloads right. But then it doesn't read the data right from the INI.
So does a INI file have to be Unicode encoded or should it also be OK to use UTF-8 with the function calls?
I think it is about time I convert this part of my program to UTF-8 encoded XML files instead! Way to go.
But wanted to ask the question first.
I should clarify that the text file was initial saved as UTF-8 using NotePad.
Update
This is what it looks like when I read the file:
This is how I download the file:
BOOL CUpdatePublicTalksDlg::DownloadINI()
{
    CInternetSession    iSession;
    CHttpFile           *pWebFile = NULL;
    CWaitCursor         wait;
    CFile               fileLocal;
    TCHAR               szError[_MAX_PATH];
    int                 iRead = 0, iBytesRead = 0;
    char                szBuffer[4096];
    BOOL                bOK = FALSE;
    DWORD               dwStatusCode;
    CString             strError;
    // ask user to go online
    if( InternetGoOnline( (LPTSTR)(LPCTSTR)m_strURLPathINI, GetSafeHwnd(), 0 ) )
    {
        TRY
        {
            // our session should already be open
            // try to open up internet session to my URL
            // AJT V10.4.0 Use flag INTERNET_FLAG_RELOAD
            pWebFile = (CHttpFile*)iSession.OpenURL( m_strURLPathINI, 1,
                INTERNET_FLAG_TRANSFER_BINARY |
                INTERNET_FLAG_DONT_CACHE | INTERNET_FLAG_RELOAD);
            if(pWebFile != NULL)
            {
                if( pWebFile->QueryInfoStatusCode( dwStatusCode ) )
                {
                    // 20x codes mean success
                    if( (dwStatusCode / 100) == 2 )
                    {
                        // Our downloaded file is only temporary
                        m_strPathINI = theApp.GetTemporaryFilename();
                        if( fileLocal.Open( m_strPathINI,
                            CFile::modeCreate|CFile::modeWrite|CFile::typeBinary ) )
                        {
                            iRead = pWebFile->Read( szBuffer, 4096 );
                            while( iRead > 0 )
                            {   
                                iBytesRead += iRead;
                                fileLocal.Write( szBuffer, iRead );
                                iRead = pWebFile->Read( szBuffer, 4096 );
                            }
                            fileLocal.Close();
                            bOK = TRUE;
                        }
                    }
                    else
                    {
                        // There was a problem!
                        strError.Format( IDS_TPL_INVALID_URL, dwStatusCode );
                        AfxMessageBox( strError,
                                       MB_OK|MB_ICONERROR );
                    }
                }
            }
            else
            {
                AfxMessageBox( ID_STR_UPDATE_CHECK_ERR, MB_OK|MB_ICONERROR );
            }
        }
        CATCH( CException, e )
        {
            e->GetErrorMessage( szError, _MAX_PATH );
            AfxMessageBox( szError, MB_OK|MB_ICONERROR );
        }
        END_CATCH
        // Tidy up
        if( pWebFile != NULL )
        {
            pWebFile->Close();
            delete pWebFile;
        }
        iSession.Close();
    }
    return bOK;
}
This is how I read the file:
int CUpdatePublicTalksDlg::ReadTalkUpdateINI()
{
    int         iLastUpdate = 0, iUpdate;
    int         iNumTalks, iTalk;
    NEW_TALK_S  sTalk;
    CString     strUpdateSection, strTalkKey, strTalk;
    // How many possible updates are there?
    m_iNumUpdates = ::GetPrivateProfileIntW(
                        _T("TalkUpdates"),
                        _T("NumberUpdates"),
                        0,
                        m_strPathINI );
    // What what the last talk update count?
    iLastUpdate = theApp.GetLastTalkUpdateCount();
    // Loop available updates
    for( iUpdate = iLastUpdate + 1; iUpdate <= m_iNumUpdates; iUpdate++ )
    {
        // Build section key
        strUpdateSection.Format( _T("Update%d"), iUpdate );
        // How many talks are there?
        iNumTalks = ::GetPrivateProfileIntW(
                            strUpdateSection,
                            _T("NumberTalks"),
                            0,
                            m_strPathINI );
        // Loop talks
        for( iTalk = 1; iTalk <= iNumTalks; iTalk++ )
        {
            // Build key
            strTalkKey.Format( _T("Talk%d"), iTalk );
            // Get talk information
            ::GetPrivateProfileStringW(
                            strUpdateSection,
                            strTalkKey,
                            _T(""),
                            strTalk.GetBuffer( _MAX_PATH ),
                            _MAX_PATH,
                            m_strPathINI );
            strTalk.ReleaseBuffer();
            // Decode talk information
            DecodeNewTalk( strTalk, sTalk );
            // Does it already exists in the database?
            // AJT v11.2.0 Bug fix - we want *all* new talks to show
            //if( !IsExistingTalk( sTalk.iTalkNumber ) )
            //{
            //if(!LocateExistingTheme(sTalk, false))
            AddNewTalkToListBox( sTalk );
            //}
        }
    }
    // Return the actual amount of updates possible
    return m_iNumUpdates - iLastUpdate;
}
This is the file being downloaded:
http://publictalksoftware.co.uk/TalkUpdate_Arabic2.ini
Update
It seems that the file is corrupt at the point of being downloaded:
Please see updated Watch:
Making progress, I now confirm the data OK at this point:



