Pass a pipe FILE content to unknown size char * (dynamic allocated)

Question

I have a FILE * from a pipe (popen), and I wanna pass it to char *artist. The size of information that will be in the FILE is unknown, so it should use malloc().

FILE *tmp1;
char *artist;
tmp1 = popen("cmus-remote -Q | grep 'tag artist' | sed s/'tag artist'/''/g | sed '1s/^.//'", "r");

How can I do this?

See also [`char *` being different when coming from a FILE](http://stackoverflow.com/questions/25026107/char-being-different-when-coming-from-a-file). — Jonathan Leffler, Jul 30 '14 at 03:26

Ahmed Masud · Accepted Answer · 2014-07-30T13:14:23.590

1

The way to do this is to use a temporary buffer to read chunks in and attach them to the artist as following:

char buf[8192];
char *artist = NULL;
size_t len = 0, total_len = 0;
FILE *fp;

fp = popen("cmus-remote -Q | grep 'tag artist' | sed s/'tag artist'/''/g | sed '1s/^.//'", "r");

while (!feof(fp)) {
      if ( fgets(buf, sizeof(buf), fp) == NULL ) 
             break;
      len = strlen(buf);
      artist = realloc(artist, len+1); /* +1 for \0 */
      if (artist == NULL) {
             fprintf(stderr, "Ran out of memory\n");
             exit(-1);
      }
      strncpy(artist+total_len, buf, len+1); /* concatenate the string at the end of artist */ 
      total_len += len;
}

edited Jul 30 '14 at 13:14

answered Jul 30 '14 at 02:42

Ahmed Masud

21,655
3
33
58

1

Note that [`while (!feof(file))` is always wrong](http://stackoverflow.com/questions/5431941/while-feof-file-is-always-wrong). – Jonathan Leffler Jul 30 '14 at 03:00
@HenriqueLengler no, this way keeps going until the system cannot expand a logically contiguous allocation. On a modern system with virtual memory, contiguity should not be an issue, since any free physical page can be mapped to the next logical address. I'll admit I might be tempted to process in an implementation page size, but that's often 4096 and 8192 is a nice multiple. It would get risky on a system without virtual memory though! – Chris Stratton Jul 30 '14 at 03:06
So I wrote this code and compiled it. There is a problem, everything works fine but when it reach total_len += len; the char become duplicated. e.g Smog\n turns in Smog\nSmog\n. – Jul 30 '14 at 03:30
@Ahmed Masud Any tip? – Jul 30 '14 at 04:46
Fixed using fgets to control the loop. @JonathanLeffler are right while (!feof(file)) is wrong. – Jul 30 '14 at 05:18
Replace the `strncpy()` with either `strncpy(artist + total_len, buf, len + 1)` or `memmove(artist + total_len, buf, len + 1)` (or you can use `memcpy()` instead of `memmove()` in this context, but don't change to `memcpy()` unless you know why it is safe now and when it is unsafe. – Jonathan Leffler Jul 30 '14 at 05:21

score 1 · Answer 2 · answered Jul 30 '14 at 03:05

The easiest way to do this on a machine with POSIX getline() is:

char *buffer = 0;
size_t bufsiz = 0;
ssize_t nbytes;

FILE *tmp1 = popen("cmus-remote -Q | sed -n '/tag artist./ { s/tag artist//g; s/^.//p; }'", "r");
if (tmp1 == 0)
    …report problem and do not use tmp1…

while ((nbytes = getline(tmp1, &buffer, &size)) > 0)
{
    …process line of data…
}
free(buffer);

This limits your lines to the size of the allocatable memory; this is seldom a real limitation. Note that you only need a single sed command to process the output of cmus-remote.

Pass a pipe FILE content to unknown size char * (dynamic allocated)

2 Answers2

Linked