System.Data.SQLite
View Ticket
Not logged in
Ticket UUID: a1b93b2054b97f5931462c88802f42cfd39d2e11
Title: out of memory System.Data.SQLite
Status: Closed Type: Incident
Severity: Important Priority: Medium
Subsystem: Integration_Via_PInvoke Resolution: Not_A_Bug
Last Modified: 2016-03-25 03:00:29
Version Found In: 1.0.99.0
User Comments:
anonymous added on 2016-02-12 14:52:36:
i am loading 1 billion rows (aprox 76.2gb) data from sql server to sqlite (in memory db). my RAM is 32 GB and .net app is 64 bit with frame work ver. is 4.5.1. when RAM become Full it start paging data. it almost paged 25 gb data when it reached 57 gb (32gb in RAM + 25 gb Paged) out of memory exception occurred.

message: "out of memory" & vbCrLf & "out of memory"
HResult: -2147467259
Value:
at System.Data.SQLite.SQLite3.Reset(SQLiteStatement Stmt) in c:\dev\sq1ite\dotnet\System.Data.SQLite\SQLite3.cs:line 1207
at System.Data.SQLite.SQLite3.Step(SQLiteStatement Stmt) in c:\dev\sq1ite\dotnet\System.Data.SQLite\SQLite3.cs:line 1064
at System.Data.SQLite.SQLiteDataReader.NextResult() in c:\dev\sqlite\dotnet\System.Data.SQLite\SQLiteDataReader.cs:line 1451
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteConInand cmd, CommandBehavior behave) in c:\dev\sqlite\dotnet\System.Data.SQLite\SQLiteDataReader.cs:line 124
at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior) in c:\dev\sqlite\dotnet\System.Data.SQLite\SQLiteCommand.cs:line 873
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(ConmandBehavior behavior) in c:\dev\sqlite\dotnet\System.Data.SQLiteI\SQLiteCommand.cs:line 917
at SqlitePrOject.MemTest.Insert1BRecInchunksFromSOLServerToSOLITEmeM)

is this some storage limitation of SQlite?
this is the code:
Using sqltTrans = sqltCnn.BeginTransaction()
                Using sqltCmd = New SQLiteCommand(sqltCnn)

                    While sqlReader.Read()

                        sqltCmd.CommandText = "insert into sqlt1BmemDB (F1, F2, F3, F4, F5, F6, F7, F8) values ('" +
                                sqlReader.Item("F1").ToString() + "','" +
                                sqlReader.Item("F2").ToString() + "','" +
                                sqlReader.Item("F3").ToString() +
                                    "'," + sqlReader.Item("F4").ToString() +
                                    "," + sqlReader.Item("F5").ToString() +
                                    "," + sqlReader.Item("F6").ToString() +
                                    ",'" + sqlReader.Item("F7").ToString() +
                                    "','" + sqlReader.Item("F8").ToString() + "')"
                        sqltCmd.ExecuteNonQuery()
                    End While
                    sqltTrans.Commit()
                End Using
            End Using