5

What part of the C++ IO streams does the \r to \r\n conversion? Is it the stream_buf itself or is it part of the internal to external encoding conversion by codecvt facet?

UPDATE 1

You all say that it is done in streambuf/filebuf. Ok. But how does this arrangement deal with, e.g., external encodings like UTF-16? Then it seems that the file has to be opened with ios::binary flag which disables the translation.

wilx
  • 17,697
  • 6
  • 59
  • 114
  • 1
    Shouldn't that be `\n` to `\r\n` conversion? Usually what you'll get from portable code is a `\n` and you need to prepend the line feed for "windows like" representations. – πάντα ῥεῖ Jun 06 '12 at 22:15
  • To prepend the extra `\r` has nothing to do with the used character encoding, it's still an extra character. The `\r` controls a terminal write cursor to start at column 0 for some terminals. `codecvt` controls how many bytes are used to encode a single character. – πάντα ῥεῖ Jun 07 '12 at 14:10

3 Answers3

2

This conversion is not (usually) performed by stream, streambuf, or facet. It is the responsibility the C library code (e.g. fputc()) that is called by streambuf's overflow() and underflow().

If you need it for some reason (e.g. when implementing a dos2unix routine), there's a handy example in boost.iostreams.

EDIT: std::filebuf only supports multibyte encodings for text files, e.g UTF-8 or GB18030 or whatever the locale uses. A UTF-16 file would have to be opened in binary mode, as a plain byte stream (which can be interpreted as UTF-16 with C++11's codecvt facilities), and yes the line endings would not get converted.

Cubbi
  • 46,567
  • 13
  • 103
  • 169
  • streambuf implementations don't necessarily need to use fputc(), this depends on the context. We have implementations for streambuf that directly output to a UART for debugging purposes, the (RT)OS used doesn't even provide file handles, so how to use fputc() then?? – πάντα ῥεῖ Jun 07 '12 at 14:16
  • @g-makulik The streambuf used by `std::cout` in particular is required to use the same underlying buffer as what the C I/O uses when writing to `stdout`, so this would be non-standard. As for filebuf and other streambufs, well, that's what my "usually" was for. – Cubbi Jun 07 '12 at 14:40
1

IFAIR it's done in streambuf implementation, codecvt just deals with the locale representation specifics.

πάντα ῥεῖ
  • 1
  • 13
  • 116
  • 190
  • Same comment as to Éric Malenfant's answer: Why is it done in filebuf? Does that not mean that filebuf has to have some knowledge of external encoding provided by codecvt facet? – wilx Jun 07 '12 at 09:29
  • It has nothing to do with character encoding IMHO, but how the OS handles line endings in terminal displays. – πάντα ῥεῖ Jun 07 '12 at 10:01
1

It is performed by std::filebuf, if it was open without the ios::binary flag.

Éric Malenfant
  • 13,938
  • 1
  • 40
  • 42
  • Why is it done in `filebuf`? Does that not mean that `filebuf` has to have some knowledge of external encoding provided by `codecvt` facet? – wilx Jun 07 '12 at 09:29