| Ticket Hash: | a1b93b2054b97f5931462c88802f42cfd39d2e11 | ||
| Title: | out of memory System.Data.SQLite | ||
| Status: | Closed | Type: | Incident |
| Severity: | Important | Priority: | Medium |
| Subsystem: | Integration_Via_PInvoke | Resolution: | Not_A_Bug |
| Last Modified: |
2016-03-25 03:00:29 9.27 years ago |
Created: |
2016-02-12 14:52:36 9.38 years ago |
| Version Found In: | 1.0.99.0 | ||
| User Comments: | ||||
anonymous added on 2016-02-12 14:52:36:
i am loading 1 billion rows (aprox 76.2gb) data from sql server to sqlite (in memory db). my RAM is 32 GB and .net app is 64 bit with frame work ver. is 4.5.1. when RAM become Full it start paging data. it almost paged 25 gb data when it reached 57 gb (32gb in RAM + 25 gb Paged) out of memory exception occurred.
message: "out of memory" & vbCrLf & "out of memory"
HResult: -2147467259
Value:
at System.Data.SQLite.SQLite3.Reset(SQLiteStatement Stmt) in c:\dev\sq1ite\dotnet\System.Data.SQLite\SQLite3.cs:line 1207
at System.Data.SQLite.SQLite3.Step(SQLiteStatement Stmt) in c:\dev\sq1ite\dotnet\System.Data.SQLite\SQLite3.cs:line 1064
at System.Data.SQLite.SQLiteDataReader.NextResult() in c:\dev\sqlite\dotnet\System.Data.SQLite\SQLiteDataReader.cs:line 1451
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteConInand cmd, CommandBehavior behave) in c:\dev\sqlite\dotnet\System.Data.SQLite\SQLiteDataReader.cs:line 124
at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior) in c:\dev\sqlite\dotnet\System.Data.SQLite\SQLiteCommand.cs:line 873
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(ConmandBehavior behavior) in c:\dev\sqlite\dotnet\System.Data.SQLiteI\SQLiteCommand.cs:line 917
at SqlitePrOject.MemTest.Insert1BRecInchunksFromSOLServerToSOLITEmeM)
is this some storage limitation of SQlite?
this is the code:
Using sqltTrans = sqltCnn.BeginTransaction()
Using sqltCmd = New SQLiteCommand(sqltCnn)
While sqlReader.Read()
sqltCmd.CommandText = "insert into sqlt1BmemDB (F1, F2, F3, F4, F5, F6, F7, F8) values ('" +
sqlReader.Item("F1").ToString() + "','" +
sqlReader.Item("F2").ToString() + "','" +
sqlReader.Item("F3").ToString() +
"'," + sqlReader.Item("F4").ToString() +
"," + sqlReader.Item("F5").ToString() +
"," + sqlReader.Item("F6").ToString() +
",'" + sqlReader.Item("F7").ToString() +
"','" + sqlReader.Item("F8").ToString() + "')"
sqltCmd.ExecuteNonQuery()
End While
sqltTrans.Commit()
End Using
End Using
| ||||