SQL stored procedure to Matlab datatransfer - out of memory

Question

I have a stored procedure on SQLserver that is executed by Matlab. The procedure returns some 1.5million rows. My OS is XP x32 and Matlab version is 2011a 32-bit. Whenever I try to fetch this data, I get a Java Heap Memory Size error. I tried using rowLimit as 200000 but I still got the same error. Interestingly, SQL data is easily transferred when the stored procedure returns <700000 rows. (if you want to know, there are 15 cols).

clear all                                      % free all available memory
conn = database('xxx' , 'userid', 'passwrd') ; % works fine
setdbprefs('DataReturnFormat','numeric');      % works fine
curs = exec(conn,'execute xxxxxx') ;           % works fine
fetch(curs) OR fetch(curs,200000) ;     --> java.lang.OutOfMemoryError: Java heap space

Can you suggest a way to import such data sets? Thanks!

double in Matlab, mostly float in SQL. Cols with chars in them become NaNs. — Maddy, Aug 26 '11 at 18:34
What DBMS are you connecting to (e.g. Oracle/MS SQL Server/etc)? — Andrew Janke, Aug 26 '11 at 18:36

score 1 · Accepted Answer · edited May 23 '17 at 12:12

1

You can try to increase the heap size of the JVM. Refer to this article on how to do it http://www.mathworks.com/support/solutions/en/data/1-18I2C/index.html

Regarding the actual size to set, refer to this thread: Java maximum memory on Windows XP

edited May 23 '17 at 12:12

Community

1
1

answered Aug 26 '11 at 18:12

bpgergo

15,669
5
44
68

SQL stored procedure to Matlab datatransfer - out of memory

1 Answers1