I've got a long running application which uses SQLite for its configuration database and for logging.
A new SQLite instance is opened at the beginning of the application, the application then loops for eternity, doing some pretty basic stuff, and writing the output to the SQLite database.
This is an extremely slow memory leak, over 1 day where its forced to do more usage than it does on the server, the private bytes have climbed from 30mb to 40mb, while the bytes in all heaps have remained constant. A server which does quite a lot of work with this app, takes just under 1 week to go from less than 30mb to more than 90mb. We noticed this because the service eventually started throwing errors, but didn't completely crash, after it had grown to approximately 1gb in size.
So when trying to replicate this, you're going to have to run it for a long time, before any pattern may become evident.
Here is an example of the write method used:
var insert = new Dictionary<string, object>();
insert.Add("type", description);
insert.Add("location", location);
insert.Add("message", message);
this.Sqlite.Insert("logs", insert);
This insert automatically adds a timestamp to the record, and auto increments an id, too. (Not sure if that's relevant, but that's all done automatically in SQLite)
The only reading that it does is to check that the table exists, for example:
this.Sqlite.ExecuteNonQuery("SELECT type, location, message FROM logs LIMIT 1;");
One of these instances writes out a byte[] which can contain up to something like 5kb, but usually only contains 100's of bytes. The rest of the inserted values are either integers or strings.
The reason SQLite was used was due to the required analysis we often need to do on the output, while keeping it simple and portable. However, we're now considering moving it to a proper SQL server.
This bug was introduced at sometime between 1.0.66.0 (originally hosted on phxsoftware.com) and 1.0.74.0 (which is hosted by you, and released on July 4, 2011). I have been testing this with the current version (1.0.77.0) for the past week, and it has the same problem.
Should I be calling some flush procedure? Have I structured my insert properly incorrectly? Should I be creating and destroying these connections regularly, instead of making one and keeping it open?
Thanks.
<hr /><i>mistachkin added on 2011-12-07 22:19:50 UTC:</i><br />
Do you have some C# code which can demonstrate a memory leak (no matter how small)? The current unit tests reveal no native memory leaks whatsoever (in fact, this is now checked for after each test file using the unit testing infrastructure itself). If the memory leaks in question are due to objects not being garbage collected by the GC, have you tried using GC.Collect manually to force the issue (yes, I know this is not recommended for production code)?
<hr /><i>mistachkin added on 2011-12-16 05:05:50 UTC:</i><br />
This may be fixed by check-in [419bfbd2ee]. Could you please try to reproduce the problem with the latest code on trunk?
|
