Stack Overflow
Stack Overflow

Questions tagged [mmap]

mmap is a POSIX-compliant Unix system call that maps files or devices into memory.

mmap() creates a new mapping in the virtual address space of the calling process.

https://en.wikipedia.org/wiki/Mmap

1936 questions
347
votes
6 answers

When should I use mmap for file access?

POSIX environments provide at least two ways of accessing files. There's the standard system calls open(), read(), write(), and friends, but there's also the option of using mmap() to map the file into virtual memory. When is it preferable to use…
Peter Burns
  • 44,401
  • 7
  • 38
  • 56
238
votes
13 answers

mmap() vs. reading blocks

I'm working on a program that will be processing files that could potentially be 100GB or more in size. The files contain sets of variable length records. I've got a first implementation up and running and am now looking towards improving…
jbl
  • 2,710
  • 3
  • 18
  • 13
102
votes
2 answers

Unexpected exec permission from mmap when assembly files included in the project

I am banging my head into the wall with this. In my project, when I'm allocating memory with mmap the mapping (/proc/self/maps) shows that it is an readable and executable region despite I requested only readable memory. After looking into strace…
Ben Hirschberg
  • 1,410
  • 1
  • 12
  • 17
79
votes
1 answer

Do I need to keep a file open after calling mmap on it?

I have a program that maps quite a few (100's) of sizable files 10-100MB each. I need them all mapped at the same time. At the moment I am calling open followed by mmap at the beginning of the program, followed by munmap and close at the end. Often…
camelccc
  • 2,847
  • 8
  • 26
  • 52
75
votes
2 answers

Linux shared memory: shmget() vs mmap()?

In this thread the OP is suggested to use mmap() instead of shmget() to get shared memory in Linux. I visited this page and this page to get some documentation, but the second one gives an obscure example regarding mmap(). Being almost a newbie, and…
BowPark
  • 1,340
  • 2
  • 21
  • 31
73
votes
3 answers

Mmap() an entire large file

I am trying to "mmap" a binary file (~ 8Gb) using the following code (test.c). #include #include #include #include #include #include #include #define…
Emer
  • 3,734
  • 2
  • 33
  • 47
70
votes
8 answers

Will malloc implementations return free-ed memory back to the system?

I have a long-living application with frequent memory allocation-deallocation. Will any malloc implementation return freed memory back to the system? What is, in this respect, the behavior of: ptmalloc 1, 2 (glibc default) or 3 dlmalloc tcmalloc…
osgx
  • 90,338
  • 53
  • 357
  • 513
57
votes
7 answers

malloc vs mmap in C

I built two programs, one using malloc and other one using mmap. The execution time using mmap is much less than using malloc. I know for example that when you're using mmap you avoid read/writes calls to the system. And the memory access are…
Peter
  • 985
  • 3
  • 12
  • 14
49
votes
8 answers

Setting up Laravel on a Mac php artisan migrate error: No such file or directory

Pulled a perfectly working laravel project from a git into a mac running MAMP. Project ran perfectly on a linux machine. composer install php artisan migrate, got the following error: [PDOException] …
Timothy
  • 4,198
  • 6
  • 49
  • 59
49
votes
8 answers

Why doesn't Python's mmap work with large files?

[Edit: This problem applies only to 32-bit systems. If your computer, your OS and your python implementation are 64-bit, then mmap-ing huge files works reliably and is extremely efficient.] I am writing a module that amongst other things allows…
Scott Griffiths
  • 21,438
  • 8
  • 55
  • 85
48
votes
3 answers

Why mmap() is faster than sequential IO?

Possible Duplicate: mmap() vs. reading blocks I heard (read it on the internet somewhere) that mmap() is faster than sequential IO. Is this correct? If yes then why it is faster? mmap() is not reading sequentially. mmap() has to fetch from the…
Lunar Mushrooms
  • 8,358
  • 18
  • 66
  • 88
47
votes
1 answer

Is there a memory mapping api on windows platform, just like mmap() on linux?

Is there an api to do memory mapping, just like mmap() on linux?
Fernandez
  • 471
  • 1
  • 4
  • 3
46
votes
3 answers

Speeding up file I/O: mmap() vs. read()

I have a Linux application that reads 150-200 files (4-10GB) in parallel. Each file is read in turn in small, variably sized blocks, typically less than 2K each. I currently need to maintain over 200 MB/s read rate combined from the set of files.…
Bill N.
  • 835
  • 1
  • 7
  • 13
41
votes
1 answer

What is the purpose of MAP_ANONYMOUS flag in mmap system call?

From the man page, MAP_ANONYMOUS The mapping is not backed by any file; its contents are initialized to zero. The fd and offset arguments are ignored; however, some implementations require fd to be -1 if …
Jagdish
  • 1,848
  • 3
  • 22
  • 33
34
votes
3 answers

Python - Download File Using Requests, Directly to Memory

The goal is to download a file from the internet, and create from it a file object, or a file like object without ever having it touch the hard drive. This is just for my knowledge, wanting to know if its possible or practical, particularly because…
Anon
  • 2,267
  • 3
  • 34
  • 51
1
2 3
…
99 100