Reading large amount of data from socket in c

Question

I have a python tcp server that accepts connections and generates a random string of length between (0,1M) characters, on the other side I have a c client that needs to listen on that socket and read the string and convert it into a single char of the same length as the string returned by the server

int receiver(int soc_desc, char * buffer)
{
    char *arr = (char *)malloc(sizeof(char));
    unsigned int received , total_received;
    while (1)
    {
        memset(arr, 0, MAX); // clear the buffer
        if ( received = recv(soc_desc, arr , MAX, 0) < 0)
        {
            break;
        }
        else
        {
            total_received += received;
        }
    }
    printf("%s\n",arr);
    return received; 
}
// soc_desc is the socket descriptor 
// buffer is the buffer that will hold the final output 

The only way that I can think of is using malloc to read chunks of the data returned from the server but I am having bad time trying to figure it out and I need to convert the array of char pointers into a single char when the client is done receiving data from the server

Answer 1

Reassembling network data, particularly from TCP, can get tricky. The following code is untested and surely doesn't account for all contingencies, but hopefully is down the right path of what you need to do.

ssize_t receiver(int soc_desc, char * buffer)
{
    // Whats the buffer argument used for?
    // what you had before only allocated space for 1 char. That's not what you want
    // This allocates for MAX+1 chars (I'm doing +1 for a NUL terminator)
    char *arr = malloc(MAX+1);
    // if MAX is small enough, you could do
    // char arr[MAX+1];

    // 0 buffer. You could use calloc instead of malloc + memset
    memset(arr, 0, MAX+1);
    // initialize total_received to 0
    ssize_t received , total_received = 0;
    size_t spaceLeftInBuf = MAX;
    while (1)
    {
        // don't memset here, you'll erase the data you received last iteration

        // write data to arr+total_receieved. This way you won't overwrite what
        // you received the last iteration
        received = recv(soc_desc, arr+total_received, spaceLeftInBuf, 0);
        if (received < 0)
        {
            // there was an error
            perror("recv failed: ");
            // do something with the data already received? Ok, break and
            // print what we've got
            break;
        }
        else if (received == 0)
        {
            // socket closed gracefully, suppose we can break again and print
            // what we've got
            break;
        else
        {
            // update counters
            total_received += received;
            spaceLeftInBuf -= received;
            // is our buffer full? This may not be the right check, you need to
            // decide when to process the data
            // total_received better not ever be > MAX...
            if (total_received >= MAX)
            {
                // "process" the data by printing it
                printf("%s\n", arr);
                
                // reset
                total_received = 0;
                spaceLeftInBuf = MAX;
                // not particularly necessary to reset this to all 0s, but should
                // make sure printing goes smoothly if we break out of this loop
                memset(arr, 0, MAX);  // arr[MAX] should already be '\0' from above
            }
            
        }
    }
    printf("%s\n",arr);
    return received; 
}

See Do I cast the result of malloc?

Answer 2

I found a way to do it but this is not tested enough and for sure will cause memory issues

    char *arr = malloc(sizeof(char));
    char tmp_buff[MAX];
    memset(arr,0,MAX);
    while (recv(soc_desc, tmp_buff , MAX, 0) > 0 )
    {
        strcat(arr , tmp_buff);
        printf("Size : %ld  arr : %s\n",strlen(tmp_buff),tmp_buff);
    }