Where is my memory? re-initializing DataTable

Question

I have application with 1 DataGridView, 1 DataTable etc.

My "execute" method (edited):

private void btnRunSQL_click()
{
        string strConnStr = tbConnStr.Text; // connection string from textbox
        string strSQL = tbSql.Text;         // query from textbox

        SqlDataAdapter dataAdapter = new SqlDataAdapter(strSQL, strConnStr);
        SqlCommandBuilder commandBuilder = new SqlCommandBuilder(dataAdapter);

        // clean memory
        // dtData DataTable is declared in main form class
        dtData = new DataTable(); 
        dataAdapter.Fill(dtData);

        showMemoryUsage();

 }

This is how im checking memory:

 public void showMemoryUsage()
 {
        Process proc = Process.GetCurrentProcess();
        this.Text = "Peak memory: " + proc.PeakWorkingSet64 / 1024 / 1024 + "MB";
        Application.DoEvents(); // force form refresh
 }

When I run this function multiple times - it uses more and more memory. Im working on very big data set (1000000 rows) and after few big queries I have to restart my app.

After running 1M rows query I have memory use about 900MB, second run 1100MB, 1300MB etc.

I thought re-initalizing DataTable will free my memory, but it does not. So I re-initialized BindingSource (connected with DataGridView), but it not helped too. Finally i commented my BindingSource and DataGridView.

Added later:

Disposing DataAdapter not helped.

I removed DataGridView and binding source. Not helped.

SOLUTION (I merged few answers and created test app without leaks)

using System;
using System.Data;
using System.Windows.Forms;
using System.Data.SqlClient;
using System.Diagnostics;

// to run this code you need form and controls:

// TextBox tbConnStr - textbox with SQL Server connection string
// TextBox tbSQL - for SQL query to run/test
// TextBox tbLog - for log display
// Button btnRunSQL with OnClick event set to proper method
// Button btnRunTest with OnClick event set to proper method

namespace Test_datatable
{
    public partial class Form1 : Form
    {

        DataTable dt; // i need this global

        public Form1()
        {
            InitializeComponent();
        }

        private void btnRunSQL_Click(object sender, EventArgs e)
        {
            log("Method starts.");

            string strConnStr = tbConnStr.Text;
            string strSQL = tbSQL.Text;

            using (SqlDataAdapter da = new SqlDataAdapter(strSQL, strConnStr))
            {
                using (SqlCommandBuilder cb = new SqlCommandBuilder(da))
                {

                    if (dt != null)
                    {
                        dt.Clear();
                        dt.Dispose();
                        log("DataTable cleared and disposed.");
                    }

                    dt = new DataTable();
                    da.Fill(dt);
                    log("DataTable filled.");

                }
            }

            log("Method ends.");
            tbLog.Text += Environment.NewLine;

        }

        // prints time, string and memory usage on textbox
        private void log(string text)
        {
            tbLog.Text += DateTime.Now.ToString() 
                + " "  + text + memory() + 
                Environment.NewLine;

            Application.DoEvents(); // force form refresh
        }

        // returns memory use as string, example: "Memory: 123MB"
        private string memory()
        {
            Process proc = Process.GetCurrentProcess();
            return " Peak memory: " + (proc.PeakWorkingSet64 / 1024 / 1024).ToString() + "MB ";

        }

        // test method for 10 runs
        private void btnRunTest_Click(object sender, EventArgs e)
        {
            for (int i = 0; i < 10; i++)
            {
                btnRunSQL_Click(new object(), new EventArgs());
            }
        }
    }
}

Answer 1

Best guess: there remain links from the GUI to the old DataTables through the Bindings. But the actual code is missing.

My preferred approach in this context:

if (bs != null)     bs.Clear();      // most likely solution
if (dtData != null) dtData.Clear();  // won't hurt

dtData = new DataTable(); 
bs = new BindingSource(); 

Answer 2

For starters, you should be disposing:

private void btnRunSQL_click()
{
        string strConnStr = tbConnStr.Text; // connection string from textbox
        string strSQL = tbSql.Text;         // query from textbox

        using(SqlDataAdapter dataAdapter = new SqlDataAdapter(strSQL, strConnStr))
        {
           using(SqlCommandBuilder commandBuilder = new SqlCommandBuilder(dataAdapter))
           {
               // clean memory
               // dtData DataTable and BindingSource bs are declared in main form class
               dtData = new DataTable(); 
               bs = new BindingSource(); 

               dataAdapter.Fill(dtData);
          }
      }
 }

But I believe at least part of your problem is included in the code you have not shown us.

EDIT: Please show us how you are using the BindingSource.

EDIT2: You are using PeakWorkignSet64, is your application running as 64 bit? If not this property will not be accurate. Also, try System.GC.GetTotalMemory(true) and tell us what that reports.

Answer 3

SqlDataAdapter and SqlCommandBuilder are disposable, you could try something like this

private void btnRunSQL_click()
{
        string strConnStr = tbConnStr.Text; // connection string from textbox
        string strSQL = tbSql.Text;         // query from textbox

        using(var dataAdapter = new SqlDataAdapter(strSQL, strConnStr))
        using(var commandBuilder = new SqlCommandBuilder(dataAdapter)) {
             dtData = new DataTable(); 
             bs = new BindingSource(); 

             dataAdapter.Fill(dtData);
        }
}

I think this won't fix your memory problem but since SqlDataAdapter and SqlCommandBuilder are disposable it is a good idea todo so.

Answer 4

Why don't you just re-fill your old dtData?

By creating a new instance of dtData, the old one will be flushed away by the Garbage Collector some time. You could also call the GC manually but I don't recommend that if there is a better solution.

Edit: To make this more clear, try using this function:

private void btnRunSQL_click()
{
    string strConnStr = tbConnStr.Text; // connection string from textbox
    string strSQL = tbSql.Text;         // query from textbox

    using(SqlDataAdapter dataAdapter = new SqlDataAdapter(strSQL, strConnStr))
    {
        using(SqlCommandBuilder commandBuilder = new SqlCommandBuilder(dataAdapter))
        {
            // edited after comment of  Jon Senchyna
            dtData.Clear();
            dataAdapter.Fill(dtData);
        }
     }
}