Read a txt file line by line without using fopen()

Question

I need to read a text file line by line.

The problem is that in my code I cannot use fopen() and then fgets() to read the content of each line, because fopen() fails when I use a lot of threads (it seems that it gets overload because it is being opend so many times).

But how can I change to the next line using read? read function reads all the characters along, but without changing line. — bardulia, Feb 14 '14 at 10:07
`read()` `char` by `char` until a new-line (`"\n"` or `"\r\n"` or ... whatever your file uses as line terminator). — alk, Feb 14 '14 at 10:08
Please add updates to your question to the question itself, but squezzing them into a comment. — alk, Feb 14 '14 at 10:11
unwind, sorry, I did not explained well. fopen fails when I use a lot of threads (it seems that it gets overload because it is being opend so many times) — bardulia, Feb 14 '14 at 10:11
@cuartango Why do you think switching from `fopen` to `open` is going to fix the errors in your thread based implementation. — Brandin, Feb 14 '14 at 10:14
Brandin, muy program uses multi-threading. When I run the program with 4 threads, all fopen work fine. But when I increase this number of threads, suddenly fopen does not work. — bardulia, Feb 14 '14 at 10:15
Try to find the cause, why `fopen()` does work for more then four thread, as it should be able to handle this perfectly well. — alk, Feb 14 '14 at 10:16
Two of his previous questions (1) http://stackoverflow.com/questions/18700785/open-versus-fopen-in-c-program-with-multi-threading (2) http://stackoverflow.com/questions/18609512/fopen-does-not-deal-with-more-than-60-files-at-the-same-time — nodakai, Feb 14 '14 at 10:35
@alk, I will try what you suggested about the terminator. Thanks! — bardulia, Feb 14 '14 at 10:55

fer y · Answer 1 · 2014-02-14T11:07:29.087

0

you can use getline() You would need to specify the delimiter and a while or for loop to stop when reach EOF.

EDIT: you said you dont want to use fopen() and fgets() and so :

#include <iostream>
#include <fstream>
#include <string>
using namespace std;

int main () {
  string line;
  ifstream myfile ("example.txt");
  if (myfile.is_open())
  {
    while (! myfile.eof() )
    {
      getline (myfile,line);
     if (line = something)
      {
      // read next line and print it... but how?
      }
    myfile.close();
  }

  else cout << "Unable to open file"; 

  return 0;
}

edited Feb 14 '14 at 11:07

answered Feb 14 '14 at 10:12

fer y

505
4
19

I doubt a call to `getline()` without having called `fopen()` on the `FILE*` passed in would work. – alk Feb 14 '14 at 10:13
I cannot use fopen (thread problems). I can use ofstream maybe? – bardulia Feb 14 '14 at 10:14
1

@cuartango: `ofstream` is C++, it's not available in C. – alk Feb 14 '14 at 10:15

unwind · Answer 2 · 2014-02-14T10:24:26.497

0

Most operating systems have limits on how many files a particular process can open at the same time. Since fopen() is "just" a wrapper on top of open(), it won't help to use a lower-level interface if this is your problem.

You can verify that this is indeed your problem by checking errno when fopen() fails, i.e. if it returns NULL. You sound as if you've already done this, but you weren't very specific. I would expect EMFILE if you're running into the per-process limit (see the open() manual page).

You need to investigate what your particular limits are and see if you can change them, or re-design your program so that you don't open as many files in parallel.

edited Feb 14 '14 at 10:24

answered Feb 14 '14 at 10:16

unwind

391,730
64
469
606

unwind, after a lot of investigations, the only thing we have found is that fopen fails when it is used so many times (due to multi-threading). I already changed fopen to open in binary files reading, the problem now is to read no binary files with open. – bardulia Feb 14 '14 at 10:18
one time per thread, and it fails when I am using more than 4 threads. The thing is that fopen seems to overload when using it a lot of times when opening also different files. – bardulia Feb 14 '14 at 10:26
@cuartango: What platform are you experiencing this on? For a standard PC even a really mature one 5 files opened in parallel shouldn't provoke any issues. – alk Feb 14 '14 at 10:39
It's a modern PC, with Windows 7. – bardulia Feb 14 '14 at 10:54

score 0 · Answer 3 · edited Feb 14 '14 at 10:25

0

You can use open (man open (2)) and read (man read(2)).

What is the real problem with fopen()?

edited Feb 14 '14 at 10:25

alk

69,737
10
105
255

answered Feb 14 '14 at 10:19

Zodoh

100
7

@cuartango which OS do you use, Windows, Linux or OS X? (I guess it isn't BSD) – nodakai Feb 14 '14 at 10:26
1

@cuartango I found the answer by myself! You're on Windows with Borland, and have been annoyed by this problem for half a year! http://stackoverflow.com/questions/18700785/open-versus-fopen-in-c-program-with-multi-threading – nodakai Feb 14 '14 at 10:32
It is not the same problem, is a consequence ;) – bardulia Feb 14 '14 at 10:51

score 0 · Answer 4 · answered Feb 14 '14 at 10:31

0

If open/read works and fopen/fread doesn't work, this may be because you're running out of lock structures. Using unlocked_getc in a loop (reading until you get a newline) might be the easiest option.

answered Feb 14 '14 at 10:31

user3303729

284
1
6

score 0 · Answer 5 · answered Feb 14 '14 at 11:56

Some folks recommended you to use open(2) and read(2) as replacements of fopen(3) and fgets(3), but since they are Unix (Linux and OS X) API, you cannot use them with Borland on Windows. What corresponds to open(2) and read(2) on Windows are Win32 APIs CreateFile and ReadFile

But I really doubt this is the way to go for you... You're trying to solve a problem which should never happen on a normal program. You need to find the root cause of your problem.

Read a txt file line by line without using fopen()

5 Answers5