0

I am writing a python script that processes enormous amounts of data. The problem it that is uses more memory than my computer has, so I wanted to know if there is a way to store variables on a fast ssd, rather than memory. The module I'm using to process the data is itertools, as I need to generate every possible combination for a specific string.

martineau
  • 119,623
  • 25
  • 170
  • 301
Coolcreeper
  • 134
  • 9
  • You can potentially use [`pickle`](https://docs.python.org/3/library/pickle.html). You can also split your data into chunks and process each bit separately. – MattDMo Jun 07 '22 at 21:35
  • You don't store "variables". Variables are just names that refer to *objects* in some namespace. You store *objects*. Yes, you can put objects on disk, but it won't act like an object in memory. – juanpa.arrivillaga Jun 07 '22 at 21:36
  • Would I still be able to access the objects even if it wouldn't be like it was in memory? – Coolcreeper Jun 07 '22 at 21:37
  • 2
    @Coolcreeper no, you'd have to deserialize them... back in to memory. – juanpa.arrivillaga Jun 07 '22 at 21:39
  • 2
    Look, there is probably no generic solution here, if you are trying to solve a *specific issue* you really need to give us more details – juanpa.arrivillaga Jun 07 '22 at 21:40

0 Answers0