3

I get this error:

FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory

but I'm using --max-old-space-size=12288 and I have 16GB of memory

Using process.memoryUsage() I can see the heap only goes up to ~1GB; Also, the error always happens at the same line of code, namely types.js:~94, which is part of a library I'm using.

This makes me thing that there's a bug somewhere, maybe in the argument I pass to the function, but I can't figure out how to spot it.

What should I look for?

The full error is:

    <--- Last few GCs --->

[28425:0x2739620]   406975 ms: Scavenge 3689.5 (3833.4) -> 3673.7 (3833.9) MB, 4.2 / 0.0 ms  (average mu = 0.991, current mu = 0.990) allocation failure 
[28425:0x2739620]   407874 ms: Scavenge 3728.8 (3873.4) -> 3713.1 (3873.9) MB, 4.5 / 0.0 ms  (average mu = 0.991, current mu = 0.990) allocation failure 
[28425:0x2739620]   408785 ms: Scavenge 3768.1 (3913.4) -> 3752.4 (3913.9) MB, 4.5 / 0.0 ms  (average mu = 0.991, current mu = 0.990) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2a42914841bd]
Security context: 0x281401b9e6c9 <JSObject>
    1: decode [0x1c8f15ad00a9] [/media/Data/dev/btc-crawler/node_modules/bitcoin-protocol/src/types.js:~94] [pc=0x2a429178f88b](this=0x1c8f15acb299 <Object map = 0x736c200e801>,buffer=0xeeb6cf17c29 <Uint8Array map = 0x3d1fa97d04e9>,offset=4,end=0x25b4eaf822e1 <undefined>)
    2: _transform [0x38503cda5d21] [/me...

FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory
 1: 0x89c2f0 node::Abort() [node]
 2: 0x89c33c  [node]
 3: 0xa8f05e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xa8f278 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xe7cad2  [node]
 6: 0xf88847 v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [node]
 7: 0xf9a3c8 v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::EnsureCapacity(v8::internal::Handle<v8::internal::StringTable>, int, v8::internal::PretenureFlag) [node]
 8: 0xf9a58a v8::internal::StringTable::LookupKey(v8::internal::Isolate*, v8::internal::StringTableKey*) [node]
 9: 0xfa03f4 v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [node]
10: 0xf678aa v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, bool*, v8::internal::LookupIterator::Configuration) [node]
11: 0x1112512 in the code, v8::internal::Runtime::GetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, bool*) [node]
12: 0x1112849  [node]
13: 0x1114f09 v8::internal::Runtime_KeyedGetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
14: 0x2a42914841bd

Line 94 of types.js is the entry point of the decode function, defined as follows:

  var buffer12 = struct.Buffer(12)

  function decode (buffer, offset, end) {
    var bvalue = buffer12.decode(buffer, offset, end)
    for (var stop = 0; bvalue[stop] !== 0; ++stop);
    for (var i = stop; i < bvalue.length; ++i) {
      if (bvalue[i] !== 0) throw new Error('Found a non-null byte after the first null byte in a null-padded string')
    }
    return bvalue.slice(0, stop).toString('ascii')
  }
FedFranz
  • 529
  • 1
  • 5
  • 15

2 Answers2

3

You have a typo!

The correct usage is node --max-old-space-size=4096 somescript.js

4096 is 4gb in this case, adjust it accordingly in your case...

Run node --v8-options for a list of all valid v8 options, or check this list of v8 flags

Alex Michailidis
  • 4,078
  • 1
  • 16
  • 35
  • The typo was just in the description (I've corrected it). Otherwise it wouldn't run at all...(it would yield a 'bad option' error) – FedFranz Jul 24 '18 at 16:08
  • what is the output of this `require('v8').getHeapStatistics().total_available_size` – Alex Michailidis Jul 24 '18 at 16:15
  • `require('v8').getHeapStatistics().total_available_size` varies between '12000000000' and '13000000000' along the execution – FedFranz Jul 24 '18 at 16:31
  • 1
    This is very strange... it means the limit has been increased to almost 13gb but something still causing heap to get out of memory – Alex Michailidis Jul 24 '18 at 16:35
  • Yeah...I can't see where's the problem. I tried on a different computer and it happens the same, so I guess the problem is in the code... – FedFranz Jul 24 '18 at 16:40
  • @FedFranzoni did you get anywhere here? i'm getting the same behaviour on an app i'm running node v12.7.0 with --max-old-space-size=6144 that crashes randomly at low (<2GiB) memory usage after running for a few hours (during which it sometimes exceeds 4GiB without any problems). – David Sep 05 '19 at 23:23
  • my hunch is it could be v8 type size limit, see https://stackoverflow.com/a/55474570/2269027 – David Sep 06 '19 at 00:43
  • @David not really. I just ended up following a different approach. However, I think this answer might help: https://stackoverflow.com/questions/55465821/getting-heap-out-of-memory-error-even-when-available-heap-memory-is-much-larger – FedFranz Sep 09 '19 at 14:54
0

the same error was generated but the scenerio was different

in my case, just upgraded the mysql (download and install the latest version) as the db was not connected. so i run these query.

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'new password'; FLUSH PRIVILEGES;

run this command to mysql workbench or shell.

then, you will stop getting such error.

RAHUL RAJ
  • 91
  • 4