Index: Doc/Extra/version.html ================================================================== --- Doc/Extra/version.html +++ Doc/Extra/version.html @@ -49,11 +49,12 @@
  • Add Visual Studio 2012 support to all the applicable solution/project files, their associated supporting files, and the test suite.
  • Add Visual Studio 2012 support to the redesigned designer support installer.
  • Allow opened connections to skip adding the extension functions included in the interop assembly via the new NoExtensionFunctions connection flag.
  • Support loading of SQLite extensions via the new EnableExtensions and LoadExtension methods of the SQLiteConnection class. Pursuant to [17045010df].
  • Add notifications before and after any connection is opened and closed, as well as other related notifications, via the new static Changed event.
  • -
  • Add an overload of the SQLiteLog.LogMessage method that takes a single string argument.
  • +
  • Add an overload of the SQLiteLog.LogMessage method that takes a single string parameter.
  • +
  • Add an overload of the SQLiteConnection.LogMessage method that takes a SQLiteErrorCode parameter.
  • All applicable calls into the SQLite core library now return a SQLiteErrorCode instead of an integer error code.
  • Make sure the error code of the SQLiteException class gets serialized.
  • Make the test project for the .NET Compact Framework more flexible.
  • When available, the new sqlite3_errstr function from the core library is used to get the error message for a specific return code.
  • The SetMemoryStatus, Shutdown, ResultCode, ExtendedResultCode, and SetAvRetry methods of the SQLiteConnection class now return a SQLiteErrorCode instead of an integer error code. ** Potentially Incompatible Change **
  • @@ -64,10 +65,11 @@
  • Implement more robust locking semantics for the CriticalHandle derived classes when compiled for the .NET Compact Framework.
  • Cache column indexes are they are looked up when using the SQLiteDataReader to improve performance.
  • Prevent the SQLiteConnection.Close method from throwing non-fatal exceptions during its disposal.
  • Rename the interop assembly functions sqlite3_cursor_rowid, sqlite3_context_collcompare, sqlite3_context_collseq, sqlite3_cursor_rowid, and sqlite3_table_cursor to include an "_interop" suffix. ** Potentially Incompatible Change **
  • Prevent the LastInsertRowId, MemoryUsed, and MemoryHighwater connection properties from throwing NotSupportedException when running on the .NET Compact Framework. Fix for [dd45aba387].
  • +
  • Add protection against ThreadAbortException asynchronously interrupting native resource initialization and finalization.
  • Add test automation for the Windows CE binaries.
  • 1.0.82.0 - September 3, 2012

    +** +**
  • [[SQLITE_FCNTL_BUSYHANDLER]] +** ^This file-control may be invoked by SQLite on the database file handle +** shortly after it is opened in order to provide a custom VFS with access +** to the connections busy-handler callback. The argument is of type (void **) +** - an array of two (void *) values. The first (void *) actually points +** to a function of type (int (*)(void *)). In order to invoke the connections +** busy-handler, this function should be invoked with the second (void *) in +** the array as the only argument. If it returns non-zero, then the operation +** should be retried. If it returns zero, the custom VFS should abandon the +** current operation. */ #define SQLITE_FCNTL_LOCKSTATE 1 #define SQLITE_GET_LOCKPROXYFILE 2 #define SQLITE_SET_LOCKPROXYFILE 3 #define SQLITE_LAST_ERRNO 4 @@ -870,10 +881,11 @@ #define SQLITE_FCNTL_PERSIST_WAL 10 #define SQLITE_FCNTL_OVERWRITE 11 #define SQLITE_FCNTL_VFSNAME 12 #define SQLITE_FCNTL_POWERSAFE_OVERWRITE 13 #define SQLITE_FCNTL_PRAGMA 14 +#define SQLITE_FCNTL_BUSYHANDLER 15 /* ** CAPI3REF: Mutex Handle ** ** The mutex module within SQLite defines [sqlite3_mutex] to be an @@ -4749,10 +4761,13 @@ ** successfully. An [error code] is returned otherwise.)^ ** ** ^Shared cache is disabled by default. But this might change in ** future releases of SQLite. Applications that care about shared ** cache setting should set it explicitly. +** +** This interface is threadsafe on processors where writing a +** 32-bit integer is atomic. ** ** See Also: [SQLite Shared-Cache Mode] */ SQLITE_API int sqlite3_enable_shared_cache(int); Index: System.Data.SQLite/SQLite3.cs ================================================================== --- System.Data.SQLite/SQLite3.cs +++ System.Data.SQLite/SQLite3.cs @@ -319,31 +319,37 @@ #endif } if (_sql == null) { - IntPtr db; - SQLiteErrorCode n; + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { + IntPtr db; + SQLiteErrorCode n; #if !SQLITE_STANDARD - if ((connectionFlags & SQLiteConnectionFlags.NoExtensionFunctions) != SQLiteConnectionFlags.NoExtensionFunctions) - { - n = UnsafeNativeMethods.sqlite3_open_interop(ToUTF8(strFilename), openFlags, out db); - } - else + if ((connectionFlags & SQLiteConnectionFlags.NoExtensionFunctions) != SQLiteConnectionFlags.NoExtensionFunctions) + { + n = UnsafeNativeMethods.sqlite3_open_interop(ToUTF8(strFilename), openFlags, out db); + } + else #endif - { - n = UnsafeNativeMethods.sqlite3_open_v2(ToUTF8(strFilename), out db, openFlags, IntPtr.Zero); - } + { + n = UnsafeNativeMethods.sqlite3_open_v2(ToUTF8(strFilename), out db, openFlags, IntPtr.Zero); + } #if !NET_COMPACT_20 && TRACE_CONNECTION - Trace.WriteLine(String.Format("Open: {0}", db)); + Trace.WriteLine(String.Format("Open: {0}", db)); #endif - if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); - - _sql = new SQLiteConnectionHandle(db); + if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); + _sql = new SQLiteConnectionHandle(db); + } lock (_sql) { /* HACK: Force the SyncBlock to be "created" now. */ } } // Bind functions to this connection. If any previous functions of the same name // were already bound, then the new bindings replace the old. _functionsArray = SQLiteFunction.BindFunctions(this, connectionFlags); @@ -514,24 +520,35 @@ Random rnd = null; uint starttick = (uint)Environment.TickCount; GCHandle handle = GCHandle.Alloc(b, GCHandleType.Pinned); IntPtr psql = handle.AddrOfPinnedObject(); + SQLiteStatementHandle statementHandle = null; try { while ((n == SQLiteErrorCode.Schema || n == SQLiteErrorCode.Locked || n == SQLiteErrorCode.Busy) && retries < 3) { + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { #if !SQLITE_STANDARD - n = UnsafeNativeMethods.sqlite3_prepare_interop(_sql, psql, b.Length - 1, out stmt, out ptr, out len); + n = UnsafeNativeMethods.sqlite3_prepare_interop(_sql, psql, b.Length - 1, out stmt, out ptr, out len); #else - n = UnsafeNativeMethods.sqlite3_prepare(_sql, psql, b.Length - 1, out stmt, out ptr); - len = -1; + n = UnsafeNativeMethods.sqlite3_prepare(_sql, psql, b.Length - 1, out stmt, out ptr); + len = -1; #endif #if !NET_COMPACT_20 && TRACE_STATEMENT - Trace.WriteLine(String.Format("Prepare ({0}): {1}", n, stmt)); + Trace.WriteLine(String.Format("Prepare ({0}): {1}", n, stmt)); #endif + + if ((n == SQLiteErrorCode.Ok) && (stmt != IntPtr.Zero)) + statementHandle = new SQLiteStatementHandle(_sql, stmt); + } if (n == SQLiteErrorCode.Schema) retries++; else if (n == SQLiteErrorCode.Error) { @@ -604,11 +621,11 @@ if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, GetLastError()); strRemain = UTF8ToString(ptr, len); - if (stmt != IntPtr.Zero) cmd = new SQLiteStatement(this, flags, new SQLiteStatementHandle(_sql, stmt), strSql.Substring(0, strSql.Length - strRemain.Length), previous); + if (statementHandle != null) cmd = new SQLiteStatement(this, flags, statementHandle, strSql.Substring(0, strSql.Length - strRemain.Length), previous); return cmd; } finally { @@ -1493,11 +1510,11 @@ { return UnsafeNativeMethods.sqlite3_extended_errcode(_sql); } /// Add a log message via the SQLite sqlite3_log interface. - internal override void LogMessage(int iErrCode, string zMessage) + internal override void LogMessage(SQLiteErrorCode iErrCode, string zMessage) { UnsafeNativeMethods.sqlite3_log(iErrCode, ToUTF8(zMessage)); } #if INTEROP_CODEC @@ -1597,19 +1614,30 @@ "Source connection has an invalid handle."); byte[] zDestName = ToUTF8(destName); byte[] zSourceName = ToUTF8(sourceName); - IntPtr backup = UnsafeNativeMethods.sqlite3_backup_init( - destHandle, zDestName, sourceHandle, zSourceName); + SQLiteBackupHandle backupHandle = null; + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { + IntPtr backup = UnsafeNativeMethods.sqlite3_backup_init( + destHandle, zDestName, sourceHandle, zSourceName); + + if (backup == IntPtr.Zero) + throw new SQLiteException(ResultCode(), GetLastError()); - if (backup == IntPtr.Zero) - throw new SQLiteException(ResultCode(), GetLastError()); + backupHandle = new SQLiteBackupHandle(destHandle, backup); + } return new SQLiteBackup( - this, new SQLiteBackupHandle(destHandle, backup), - destHandle, zDestName, sourceHandle, zSourceName); + this, backupHandle, destHandle, zDestName, sourceHandle, + zSourceName); } /// /// Copies up to N pages from the source database to the destination /// database associated with the specified backup object. Index: System.Data.SQLite/SQLite3_UTF16.cs ================================================================== --- System.Data.SQLite/SQLite3_UTF16.cs +++ System.Data.SQLite/SQLite3_UTF16.cs @@ -101,46 +101,52 @@ if (usePool) { _sql = SQLiteConnectionPool.Remove(strFilename, maxPoolSize, out _poolVersion); #if !NET_COMPACT_20 && TRACE_CONNECTION - Trace.WriteLine(String.Format("Open (Pool): {0}", (_sql != null) ? _sql.ToString() : "")); + Trace.WriteLine(String.Format("Open16 (Pool): {0}", (_sql != null) ? _sql.ToString() : "")); #endif } if (_sql == null) - { - IntPtr db; - SQLiteErrorCode n; - -#if !SQLITE_STANDARD - if ((connectionFlags & SQLiteConnectionFlags.NoExtensionFunctions) != SQLiteConnectionFlags.NoExtensionFunctions) - { - n = UnsafeNativeMethods.sqlite3_open16_interop(ToUTF8(strFilename), openFlags, out db); - } - else -#endif - { - // - // NOTE: This flag check is designed to enforce the constraint that opening - // a database file that does not already exist requires specifying the - // "Create" flag, even when a native API is used that does not accept - // a flags parameter. - // - if (((openFlags & SQLiteOpenFlagsEnum.Create) != SQLiteOpenFlagsEnum.Create) && !File.Exists(strFilename)) - throw new SQLiteException(SQLiteErrorCode.CantOpen, strFilename); - - n = UnsafeNativeMethods.sqlite3_open16(strFilename, out db); - } - + { + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { + IntPtr db; + SQLiteErrorCode n; + +#if !SQLITE_STANDARD + if ((connectionFlags & SQLiteConnectionFlags.NoExtensionFunctions) != SQLiteConnectionFlags.NoExtensionFunctions) + { + n = UnsafeNativeMethods.sqlite3_open16_interop(ToUTF8(strFilename), openFlags, out db); + } + else +#endif + { + // + // NOTE: This flag check is designed to enforce the constraint that opening + // a database file that does not already exist requires specifying the + // "Create" flag, even when a native API is used that does not accept + // a flags parameter. + // + if (((openFlags & SQLiteOpenFlagsEnum.Create) != SQLiteOpenFlagsEnum.Create) && !File.Exists(strFilename)) + throw new SQLiteException(SQLiteErrorCode.CantOpen, strFilename); + + n = UnsafeNativeMethods.sqlite3_open16(strFilename, out db); + } + #if !NET_COMPACT_20 && TRACE_CONNECTION - Trace.WriteLine(String.Format("Open: {0}", db)); -#endif - - if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); - - _sql = new SQLiteConnectionHandle(db); + Trace.WriteLine(String.Format("Open16: {0}", db)); +#endif + + if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); + _sql = new SQLiteConnectionHandle(db); + } lock (_sql) { /* HACK: Force the SyncBlock to be "created" now. */ } } _functionsArray = SQLiteFunction.BindFunctions(this, connectionFlags); SetTimeout(0); GC.KeepAlive(_sql); Index: System.Data.SQLite/SQLiteBase.cs ================================================================== --- System.Data.SQLite/SQLiteBase.cs +++ System.Data.SQLite/SQLiteBase.cs @@ -234,11 +234,11 @@ /// Error code to be logged with the message. /// String to be logged. Unlike the SQLite sqlite3_log() /// interface, this should be pre-formatted. Consider using the /// String.Format() function. /// - internal abstract void LogMessage(int iErrCode, string zMessage); + internal abstract void LogMessage(SQLiteErrorCode iErrCode, string zMessage); #if INTEROP_CODEC internal abstract void SetPassword(byte[] passwordBytes); internal abstract void ChangePassword(byte[] newPasswordBytes); #endif @@ -471,161 +471,227 @@ internal static string GetLastError(SQLiteConnectionHandle hdl, IntPtr db) { if ((hdl == null) || (db == IntPtr.Zero)) return "null connection or database handle"; + string result = null; + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { #if PLATFORM_COMPACTFRAMEWORK - lock (hdl.syncRoot) + lock (hdl.syncRoot) #else - lock (hdl) + lock (hdl) #endif - { - if (hdl.IsInvalid || hdl.IsClosed) - return "closed or invalid connection handle"; - + { + if (!hdl.IsInvalid && !hdl.IsClosed) + { #if !SQLITE_STANDARD - int len; - return UTF8ToString(UnsafeNativeMethods.sqlite3_errmsg_interop(db, out len), len); + int len; + result = UTF8ToString(UnsafeNativeMethods.sqlite3_errmsg_interop(db, out len), len); #else - return UTF8ToString(UnsafeNativeMethods.sqlite3_errmsg(db), -1); + result = UTF8ToString(UnsafeNativeMethods.sqlite3_errmsg(db), -1); #endif + } + else + { + result = "closed or invalid connection handle"; + } + } } - -#pragma warning disable 162 - GC.KeepAlive(hdl); /* NOTE: Unreachable code. */ -#pragma warning restore 162 + GC.KeepAlive(hdl); + return result; } internal static void FinishBackup(SQLiteConnectionHandle hdl, IntPtr backup) { if ((hdl == null) || (backup == IntPtr.Zero)) return; + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { #if PLATFORM_COMPACTFRAMEWORK - lock (hdl.syncRoot) + lock (hdl.syncRoot) #else - lock (hdl) + lock (hdl) #endif - { + { #if !SQLITE_STANDARD - SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_backup_finish_interop(backup); + SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_backup_finish_interop(backup); #else - SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_backup_finish(backup); + SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_backup_finish(backup); #endif - if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); + if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); + } } } internal static void FinalizeStatement(SQLiteConnectionHandle hdl, IntPtr stmt) { if ((hdl == null) || (stmt == IntPtr.Zero)) return; + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { #if PLATFORM_COMPACTFRAMEWORK - lock (hdl.syncRoot) + lock (hdl.syncRoot) #else - lock (hdl) + lock (hdl) #endif - { + { #if !SQLITE_STANDARD - SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_finalize_interop(stmt); + SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_finalize_interop(stmt); #else - SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_finalize(stmt); + SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_finalize(stmt); #endif - if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); + if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, null); + } } } internal static void CloseConnection(SQLiteConnectionHandle hdl, IntPtr db) { if ((hdl == null) || (db == IntPtr.Zero)) return; -#if PLATFORM_COMPACTFRAMEWORK - lock (hdl.syncRoot) -#else - lock (hdl) -#endif - { -#if !SQLITE_STANDARD - SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_close_interop(db); -#else - ResetConnection(hdl, db); - - SQLiteErrorCode n; - - try - { - n = UnsafeNativeMethods.sqlite3_close_v2(db); - } - catch (EntryPointNotFoundException) - { - n = UnsafeNativeMethods.sqlite3_close(db); - } -#endif - if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, GetLastError(hdl, db)); + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { +#if PLATFORM_COMPACTFRAMEWORK + lock (hdl.syncRoot) +#else + lock (hdl) +#endif + { +#if !SQLITE_STANDARD + SQLiteErrorCode n = UnsafeNativeMethods.sqlite3_close_interop(db); +#else + ResetConnection(hdl, db); + + SQLiteErrorCode n; + + try + { + n = UnsafeNativeMethods.sqlite3_close_v2(db); + } + catch (EntryPointNotFoundException) + { + n = UnsafeNativeMethods.sqlite3_close(db); + } +#endif + if (n != SQLiteErrorCode.Ok) throw new SQLiteException(n, GetLastError(hdl, db)); + } } } internal static bool ResetConnection(SQLiteConnectionHandle hdl, IntPtr db, bool canThrow) { if ((hdl == null) || (db == IntPtr.Zero)) return false; -#if PLATFORM_COMPACTFRAMEWORK - lock (hdl.syncRoot) -#else - lock (hdl) -#endif - { - if (canThrow && hdl.IsInvalid) - throw new InvalidOperationException("The connection handle is invalid."); - - if (canThrow && hdl.IsClosed) - throw new InvalidOperationException("The connection handle is closed."); - - if (!canThrow && (hdl.IsInvalid || hdl.IsClosed)) - return false; - - IntPtr stmt = IntPtr.Zero; - SQLiteErrorCode n; - do - { - stmt = UnsafeNativeMethods.sqlite3_next_stmt(db, stmt); - if (stmt != IntPtr.Zero) - { -#if !SQLITE_STANDARD - n = UnsafeNativeMethods.sqlite3_reset_interop(stmt); -#else - n = UnsafeNativeMethods.sqlite3_reset(stmt); -#endif - } - } while (stmt != IntPtr.Zero); - - if (IsAutocommit(hdl, db) == false) // a transaction is pending on the connection - { - n = UnsafeNativeMethods.sqlite3_exec(db, ToUTF8("ROLLBACK"), IntPtr.Zero, IntPtr.Zero, out stmt); - if (n != SQLiteErrorCode.Ok) - { - if (canThrow) - throw new SQLiteException(n, GetLastError(hdl, db)); - else - return false; + + bool result = false; + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { +#if PLATFORM_COMPACTFRAMEWORK + lock (hdl.syncRoot) +#else + lock (hdl) +#endif + { + if (canThrow && hdl.IsInvalid) + throw new InvalidOperationException("The connection handle is invalid."); + + if (canThrow && hdl.IsClosed) + throw new InvalidOperationException("The connection handle is closed."); + + if (!hdl.IsInvalid && !hdl.IsClosed) + { + IntPtr stmt = IntPtr.Zero; + SQLiteErrorCode n; + + do + { + stmt = UnsafeNativeMethods.sqlite3_next_stmt(db, stmt); + if (stmt != IntPtr.Zero) + { +#if !SQLITE_STANDARD + n = UnsafeNativeMethods.sqlite3_reset_interop(stmt); +#else + n = UnsafeNativeMethods.sqlite3_reset(stmt); +#endif + } + } while (stmt != IntPtr.Zero); + + // + // NOTE: Is a transaction NOT pending on the connection? + // + if (IsAutocommit(hdl, db)) + { + result = true; + } + else + { + n = UnsafeNativeMethods.sqlite3_exec( + db, ToUTF8("ROLLBACK"), IntPtr.Zero, IntPtr.Zero, + out stmt); + + if (n == SQLiteErrorCode.Ok) + { + result = true; + } + else if (canThrow) + { + throw new SQLiteException(n, GetLastError(hdl, db)); + } + } } } } GC.KeepAlive(hdl); - return true; + return result; } internal static bool IsAutocommit(SQLiteConnectionHandle hdl, IntPtr db) { - if (db == IntPtr.Zero) return false; + if ((hdl == null) || (db == IntPtr.Zero)) return false; + + bool result = false; + + try + { + // do nothing. + } + finally /* NOTE: Thread.Abort() protection. */ + { #if PLATFORM_COMPACTFRAMEWORK - lock (hdl.syncRoot) + lock (hdl.syncRoot) #else - lock (hdl) + lock (hdl) #endif - { - if (hdl.IsInvalid || hdl.IsClosed) return false; - return (UnsafeNativeMethods.sqlite3_get_autocommit(db) == 1); - } -#pragma warning disable 162 - GC.KeepAlive(hdl); /* NOTE: Unreachable code. */ -#pragma warning restore 162 + { + if (!hdl.IsInvalid && !hdl.IsClosed) + result = (UnsafeNativeMethods.sqlite3_get_autocommit(db) == 1); + } + } + GC.KeepAlive(hdl); /* NOTE: Unreachable code. */ + return result; } } internal interface ISQLiteSchemaExtensions { Index: System.Data.SQLite/SQLiteConnection.cs ================================================================== --- System.Data.SQLite/SQLiteConnection.cs +++ System.Data.SQLite/SQLiteConnection.cs @@ -1932,19 +1932,30 @@ throw new InvalidOperationException("Database connection not valid for getting extended result code."); return _sql.ExtendedResultCode(); } /// Add a log message via the SQLite sqlite3_log interface. - public void LogMessage(int iErrCode, string zMessage) + public void LogMessage(SQLiteErrorCode iErrCode, string zMessage) { CheckDisposed(); if (_sql == null) throw new InvalidOperationException("Database connection not valid for logging message."); _sql.LogMessage(iErrCode, zMessage); } + + /// Add a log message via the SQLite sqlite3_log interface. + public void LogMessage(int iErrCode, string zMessage) + { + CheckDisposed(); + + if (_sql == null) + throw new InvalidOperationException("Database connection not valid for logging message."); + + _sql.LogMessage((SQLiteErrorCode)iErrCode, zMessage); + } #if INTEROP_CODEC /// /// Change the password (or assign a password) to an open database. /// Index: System.Data.SQLite/UnsafeNativeMethods.cs ================================================================== --- System.Data.SQLite/UnsafeNativeMethods.cs +++ System.Data.SQLite/UnsafeNativeMethods.cs @@ -1338,11 +1338,11 @@ #if !PLATFORM_COMPACTFRAMEWORK [DllImport(SQLITE_DLL, CallingConvention = CallingConvention.Cdecl)] #else [DllImport(SQLITE_DLL)] #endif - internal static extern void sqlite3_log(int iErrCode, byte[] zFormat); + internal static extern void sqlite3_log(SQLiteErrorCode iErrCode, byte[] zFormat); #if !PLATFORM_COMPACTFRAMEWORK [DllImport(SQLITE_DLL, CallingConvention = CallingConvention.Cdecl)] #else [DllImport(SQLITE_DLL)] Index: Tests/basic.eagle ================================================================== --- Tests/basic.eagle +++ Tests/basic.eagle @@ -1443,12 +1443,11 @@ runTest {test data-1.28 {SetMemoryStatus method} -setup { # # NOTE: Make sure that SQLite core library is completely shutdown prior to # starting this test. # - object invoke -flags +NonPublic System.Data.SQLite.UnsafeNativeMethods \ - sqlite3_shutdown + shutdownSQLite $test_channel # # NOTE: Create an instance of the core SQLite library interop wrapper class. # set sqlite3 [object create -flags +NonPublic System.Data.SQLite.SQLite3 \ @@ -1501,12 +1500,11 @@ catch { # # NOTE: Make sure that SQLite core library is completely shutdown prior # to attempting to reconfigure the memory status setting. # - object invoke -flags +NonPublic System.Data.SQLite.UnsafeNativeMethods \ - sqlite3_shutdown + shutdownSQLite $test_channel # # NOTE: Attempt to make sure the default value for the process-wide # memory usage tracking setting is restored. This is not 100% # reliable because we have no idea what the original value was @@ -1518,12 +1516,12 @@ } unset -nocomplain result sqlite3 } -constraints \ {eagle monoBug28 command.sql compile.DATA SQLite System.Data.SQLite} -match \ -regexp -result {^Ok Misuse Ok Ok System#IntPtr#\d+ System#IntPtr#\d+ \d+ \d+ \d+\ -\d+ \d+ True True True True True True$}} +regexp -result {^Ok Misuse Ok Ok System#IntPtr#\d+ System#IntPtr#\d+ \d+ \d+\ +\d+ \d+ \d+ True True True True True True$}} ############################################################################### runTest {test data-1.29 {SQLiteConnection.Open with SetDefaults=False} -setup { setupDb [set fileName data-1.29.db] "" "" "" "" SetDefaults=False Index: Tests/common.eagle ================================================================== --- Tests/common.eagle +++ Tests/common.eagle @@ -686,10 +686,21 @@ # NOTE: Is the specified database file name really an in-memory database? # return [expr {$fileName eq ":memory:" || \ [string range $fileName 0 12] eq "file::memory:"}] } + + proc executeSql { sql {execute none} {fileName ""} } { + if {[string length $fileName] == 0} then {set fileName :memory:} + setupDb $fileName "" "" "" "" "" false false false false memDb + + try { + return [sql execute -execute $execute $memDb $sql] + } finally { + cleanupDb $fileName memDb false false + } + } proc setupDb { fileName {mode ""} {dateTimeFormat ""} {dateTimeKind ""} {flags ""} {extra ""} {qualify true} {delete true} {uri false} {temporary true} {varName db} } { @@ -823,13 +834,13 @@ # # NOTE: Configure the temporary directory for the newly opened database # connection now unless our caller forbids it. # - if {$temporary} then { + if {$temporary && ![info exists ::no(setTemporaryDirectory)]} then { sql execute $db [appendArgs \ - "PRAGMA temp_store_directory = \"" [getTemporaryDirectory] "\";"] + "PRAGMA temp_store_directory = \"" [getTemporaryDirectory] \"\;] } # # NOTE: Always return the connection handle upon success. # @@ -1038,10 +1049,35 @@ tputs $channel [appendArgs \ "==== WARNING: failed full garbage collection, error: " \ \n\t $error \n] } } + + proc shutdownSQLite { channel {force false} {quiet false} } { + # + # NOTE: Make sure that SQLite core library is completely shutdown. This + # is used by tests that change configuration options and/or those + # that need to make sure logging is initialized (i.e. just in case + # the SQLite core library was initialized in the process prior to + # the SQLiteLog class being able to setup its logging callback). + # + if {$force || [haveConstraint SQLite]} then { + if {[catch {object invoke -flags +NonPublic \ + System.Data.SQLite.UnsafeNativeMethods \ + sqlite3_shutdown} result] == 0} then { + if {!$quiet} then { + tputs $channel [appendArgs \ + "---- call sqlite3_shutdown()... ok: " $result \n] + } + } else { + if {!$quiet} then { + tputs $channel [appendArgs \ + "---- call sqlite3_shutdown()... error: " \n\t $result \n] + } + } + } + } proc reportSQLiteResources { channel {quiet false} {collect true} } { # # NOTE: Skip all output if we are running in "quiet" mode. # @@ -1055,11 +1091,11 @@ if {!$quiet} then { tputs $channel [appendArgs $memory " bytes\n"] } } else { # - # NOTE: Maybe the SQLite native library is unavailable? + # NOTE: Maybe the SQLite core library is unavailable? # set memory unknown if {!$quiet} then { tputs $channel [appendArgs $memory \n] @@ -1078,11 +1114,11 @@ if {!$quiet} then { tputs $channel [appendArgs $memory " bytes\n"] } } else { # - # NOTE: Maybe the SQLite native library is unavailable? + # NOTE: Maybe the SQLite core library is unavailable? # set memory unknown if {!$quiet} then { tputs $channel [appendArgs $memory \n] @@ -1117,10 +1153,111 @@ } } return $result } + + proc checkForSQLiteDirectories { channel {reset false} } { + # + # NOTE: Check if the sqlite3_win32_set_directory function is available. + # + tputs $channel \ + "---- checking for function sqlite3_win32_set_directory... " + + # + # NOTE: This call to the sqlite3_win32_set_directory function uses the + # invalid value 0 for the first argument. This code is designed + # to check if calling the function will raise an exception (i.e. + # the actual result of the function does not matter as long as no + # directory is changed). + # + if {[catch { + object invoke -flags +NonPublic \ + System.Data.SQLite.UnsafeNativeMethods \ + sqlite3_win32_set_directory 0 null}] == 0} then { + # + # NOTE: Calling the sqlite3_win32_set_directory function does not + # cause an exception; therefore, it must be available (i.e. + # even though it should return a failure return code in this + # case). + # + addConstraint sqlite3_win32_set_directory + + tputs $channel yes\n + + # + # NOTE: Does our caller want to reset the directories? + # + if {$reset} then { + # + # NOTE: Now make sure the database and temporary directories are + # reset their default values, which should be null for both. + # Since the sqlite3_win32_set_directory function is available, + # use it. + # + for {set index 1} {$index < 3} {incr index} { + if {[catch { + object invoke -flags +NonPublic \ + System.Data.SQLite.UnsafeNativeMethods \ + sqlite3_win32_set_directory $index null} \ + result] == 0} then { + tputs $channel [appendArgs \ + "---- call sqlite3_win32_set_directory(" $index \ + ", null)... ok: " $result \n] + } else { + tputs $channel [appendArgs \ + "---- call sqlite3_win32_set_directory(" $index \ + ", null)... error: " \n\t $result \n] + } + } + } + } else { + tputs $channel no\n + + # + # NOTE: Does our caller want to reset the directories? + # + if {$reset} then { + # + # NOTE: Now make sure the database and temporary directories are + # reset their default values, which should be null for both. + # Since the sqlite3_win32_set_directory function does not + # appear to be available, use the associated PRAGMA commands + # instead. + # + foreach directory [list data_store_directory temp_store_directory] { + set sql [appendArgs "PRAGMA " $directory " = \"\";"] + + if {[catch {executeSql $sql} result] == 0} then { + tputs $channel [appendArgs \ + "---- execute PRAGMA " $directory "... ok: " \ + $result \n] + } else { + tputs $channel [appendArgs \ + "---- execute PRAGMA " $directory "... error: " \ + \n\t $result \n] + } + } + } + } + + # + # NOTE: Finally, show the current value of the database and temporary + # directories. + # + foreach directory [list data_store_directory temp_store_directory] { + tputs $channel [appendArgs "---- checking " $directory "... "] + + set sql [appendArgs "PRAGMA " $directory \;] + + if {[catch {executeSql $sql scalar} result] == 0} then { + tputs $channel [appendArgs "ok: \"" $result \"\n] + } else { + tputs $channel [appendArgs "error: " \n\t $result \n] + } + } + } proc runSQLiteTestPrologue {} { # # NOTE: Skip running our custom prologue if the main one has been skipped. # @@ -1221,31 +1358,13 @@ # found it (above), this should always succeed. # checkForSQLite $::test_channel # - # NOTE: Check if the sqlite3_win32_set_directory function is available. - # - tputs $::test_channel \ - "---- checking for function sqlite3_win32_set_directory... " - - if {[catch { - object invoke -flags +NonPublic \ - System.Data.SQLite.UnsafeNativeMethods \ - sqlite3_win32_set_directory 0 null}] == 0} then { - # - # NOTE: Calling the sqlite3_win32_set_directory function does not - # cause an exception; therefore, it must be available (i.e. - # even though it should return a failure return code in this - # case). - # - addConstraint sqlite3_win32_set_directory - - tputs $::test_channel yes\n - } else { - tputs $::test_channel no\n - } + # NOTE: Check the SQLite database and temporary directories. + # + checkForSQLiteDirectories $::test_channel # # NOTE: Attempt to determine if the custom extension functions were # compiled into the SQLite interop assembly. # Index: Tests/stress.eagle ================================================================== --- Tests/stress.eagle +++ Tests/stress.eagle @@ -18,13 +18,22 @@ package require System.Data.SQLite.Test runSQLiteTestPrologue ############################################################################### +# +# NOTE: Make sure that SQLite core library is completely shutdown prior to +# starting any of the tests in this file. +# +shutdownSQLite $test_channel + +############################################################################### + runTest {test stress-1.1 {multithreaded stress testing} -setup { unset -nocomplain result thread index workload noWorkload srcDb db \ - fileName compiled options count + fileName compiled options count times logFileName logListener \ + connection ############################################################################# proc expectedError { error } { return [expr {[regexp -- {\sno such table: t1\s} $error] || \ @@ -49,26 +58,85 @@ exit Failure; # halt all testing now. } ############################################################################# + proc setupLogging { fileName } { + if {![info exists ::logListener]} then { + set ::logListener [object create -alias \ + System.Diagnostics.TextWriterTraceListener $fileName] + } + + object invoke System.Diagnostics.Trace.Listeners Add $::logListener + object invoke System.Data.SQLite.SQLiteLog Initialize + + tputs $::test_channel [appendArgs \ + "---- enabled SQLite trace logging to file \"" $fileName \"\n] + } + + ############################################################################# + + proc cleanupLogging { fileName } { + if {[info exists ::logListener]} then { + object invoke System.Diagnostics.Trace.Listeners Remove $::logListener + $::logListener Close + } + + # + # NOTE: Copy the trace listener log file to the main test log file. + # + tlog "---- BEGIN TRACE LISTENER OUTPUT\n" + tlog [readFile $fileName] + tlog "\n---- END TRACE LISTENER OUTPUT\n" + + # + # NOTE: Delete the trace listener log file because its contents have + # been copied to the main test log file. + # + cleanupFile $fileName + + tputs $::test_channel [appendArgs \ + "---- disabled SQLite trace logging to file \"" $fileName \"\n] + } + + ############################################################################# + + # + # NOTE: The trace listener used with the SQLiteLog class to capture output + # from the core SQLite library requires its own log file because the + # TextWriterTraceListener class opens and locks the log file it uses + # as the basis of the output stream. Before this test is complete, + # the entire contents of this trace log file will be copied into the + # main test log file and then deleted. + # + set logFileName [appendArgs $test_log .trace] + setupLogging $logFileName + + ############################################################################# + set count(0) 1 set count(1) 5 set count(2) 300 + set count(3) 57 + set count(4) 10000000 set noWorkload [list] if {[info exists argv] && [llength $argv] > 0} then { parse options -flags \ {-StopOnUnknownOption +IgnoreOnUnknownOption SkipOnUnknownOption} -- \ [list [list null MustHaveIntegerValue -1 -1 -count0 $count(0)] \ [list null MustHaveIntegerValue -1 -1 -count1 $count(1)] \ [list null MustHaveIntegerValue -1 -1 -count2 $count(2)] \ + [list null MustHaveIntegerValue -1 -1 -count3 $count(3)] \ + [list null MustHaveIntegerValue -1 -1 -count4 $count(4)] \ [list null MustHaveListValue -1 -1 -noWorkload $noWorkload]] $argv set count(0) $options(-count0,value) set count(1) $options(-count1,value) set count(2) $options(-count2,value) + set count(3) $options(-count3,value) + set count(4) $options(-count4,value) set noWorkload $options(-noWorkload,value) } ############################################################################# @@ -79,10 +147,16 @@ "---- workloads will have " $count(1) " iteration(s)...\n"] tputs $test_channel [appendArgs \ "---- workloads will wait at least " $count(2) " millisecond(s)...\n"] + tputs $test_channel [appendArgs \ + "---- workload \"small\" chunk size is " $count(3) " byte(s)...\n"] + + tputs $test_channel [appendArgs \ + "---- workload \"big\" chunk size is " $count(4) " byte(s)...\n"] + if {[llength $noWorkload] > 0} then { tputs $test_channel [appendArgs \ "---- workloads to be skipped... " $noWorkload \n] } @@ -118,27 +192,34 @@ setupDb $fileName(1) "" "" "" "" "" false false true true srcDb setupDb $fileName(2) ############################################################################# + set connection [getDbConnection] + + $connection LogMessage Ok [appendArgs \ + "starting stress test using database \"" $fileName(2) "\"..."] + + ############################################################################# + set workload(1) [list [list srcFileName dstFileName table count] { # # NOTE: Workload #1, CREATE TABLE statements. # lappend ::times(1) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 2} {$index <= $count} {incr index} { + for {set index 2} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs \ "CREATE TABLE IF NOT EXISTS t" $index "(x PRIMARY KEY, y, z);"] - showTest 1 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest A + } error]} then { + if {[expectedError $error]} then { + showTest a + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -149,21 +230,21 @@ # # NOTE: Workload #2, DROP TABLE statements. # lappend ::times(2) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 2} {$index <= $count} {incr index} { + for {set index 2} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs \ "DROP TABLE IF EXISTS t" $index \;] - showTest 2 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest B + } error]} then { + if {[expectedError $error]} then { + showTest b + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -174,21 +255,21 @@ # # NOTE: Workload #3, "small" SELECT statements. # lappend ::times(3) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute -execute reader $db [appendArgs \ "SELECT x, y FROM " $table " WHERE z = 'small';"] - showTest 3 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest C + } error]} then { + if {[expectedError $error]} then { + showTest c + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -199,21 +280,21 @@ # # NOTE: Workload #4, "big" SELECT statements. # lappend ::times(4) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute -execute reader $db [appendArgs \ "SELECT x, y FROM " $table " WHERE z = 'big';"] - showTest 4 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest D + } error]} then { + if {[expectedError $error]} then { + showTest d + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -224,24 +305,24 @@ # # NOTE: Workload #5, "small" INSERT statements. # lappend ::times(5) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs \ "INSERT INTO " $table "(x, y, z) VALUES('" \ [format %lX [expr {random()}]] "', '" \ - [base64 encode -- [expr {randstr(10000)}]] \ + [base64 encode -- [expr {randstr($::count(3))}]] \ "', 'small');"] - showTest 5 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest E + } error]} then { + if {[expectedError $error]} then { + showTest e + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -252,23 +333,23 @@ # # NOTE: Workload #6, "big" INSERT statements. # lappend ::times(6) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs \ "INSERT INTO " $table "(x, y, z) VALUES('" \ [format %lX [expr {random()}]] \ - "', RANDOMBLOB(10000000), 'big');"] - showTest 6 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + "', RANDOMBLOB(" $::count(4) "), 'big');"] + showTest F + } error]} then { + if {[expectedError $error]} then { + showTest f + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -279,22 +360,22 @@ # # NOTE: Workload #7, "small" UPDATE statements. # lappend ::times(7) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs "UPDATE " $table \ - " SET y = '" [base64 encode -- [expr {randstr(10000)}]] \ + " SET y = '" [base64 encode -- [expr {randstr($::count(3))}]] \ "' WHERE x LIKE '" [format %X $index] "%' AND z = 'small';"] - showTest 7 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest G + } error]} then { + if {[expectedError $error]} then { + showTest g + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -305,22 +386,23 @@ # # NOTE: Workload #8, "big" UPDATE statements. # lappend ::times(8) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs "UPDATE " $table \ - " SET y = RANDOMBLOB(10000000) WHERE x LIKE '" \ - [format %X $index] "%' AND z = 'big';"] - showTest 8 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + " SET y = RANDOMBLOB(" $::count(4) \ + ") WHERE x LIKE '" [format %X $index] \ + "%' AND z = 'big';"] + showTest H + } error]} then { + if {[expectedError $error]} then { + showTest h + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -331,21 +413,21 @@ # # NOTE: Workload #9, "small" DELETE statements. # lappend ::times(9) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs "DELETE FROM " $table \ " WHERE x LIKE '" [format %X $index] "%' AND z = 'small';"] - showTest 9 - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest I + } error]} then { + if {[expectedError $error]} then { + showTest i + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -356,21 +438,21 @@ # # NOTE: Workload #10, "big" DELETE statements. # lappend ::times(10) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db [appendArgs "DELETE FROM " $table \ " WHERE x LIKE '" [format %X $index] "%' AND z = 'big';"] - showTest A - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest J + } error]} then { + if {[expectedError $error]} then { + showTest j + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -381,20 +463,20 @@ # # NOTE: Workload #11, VACUUM statement. # lappend ::times(11) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { sql execute $db "VACUUM;" - showTest B - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest K + } error]} then { + if {[expectedError $error]} then { + showTest k + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -407,11 +489,20 @@ # lappend ::times(12) [lindex [time { for {set index 1} {$index <= $count} {incr index} { if {[string is integer -strict $::compiled(12)]} then { set id $::compiled(12); # NOTE: Already compiled. - object invoke _Dynamic${id}.Test${id} BackupAndGetData + if {[catch { + object invoke _Dynamic${id}.Test${id} BackupAndGetData + showTest L + } error]} then { + if {[expectedError $error]} then { + showTest l + } else { + failTest $error + } + } } else { set id [object invoke Interpreter.GetActive NextId] set code [compileCSharpWith [subst { using System; using System.Data.SQLite; @@ -447,16 +538,24 @@ } } }] true true true results errors System.Data.SQLite.dll] if {$code eq "Ok"} then { set ::compiled(12) $id; # NOTE: Compiled OK. - object invoke _Dynamic${id}.Test${id} BackupAndGetData + if {[catch { + object invoke _Dynamic${id}.Test${id} BackupAndGetData + showTest L + } error]} then { + if {[expectedError $error]} then { + showTest l + } else { + failTest $error + } + } } else { error $errors } } - showTest C } }] 0] }] ############################################################################# @@ -467,11 +566,20 @@ # lappend ::times(13) [lindex [time { for {set index 1} {$index <= $count} {incr index} { if {[string is integer -strict $::compiled(13)]} then { set id $::compiled(13); # NOTE: Already compiled. - object invoke _Dynamic${id}.Test${id} BackupAndGetData + if {[catch { + object invoke _Dynamic${id}.Test${id} BackupAndGetData + showTest M + } error]} then { + if {[expectedError $error]} then { + showTest m + } else { + failTest $error + } + } } else { set id [object invoke Interpreter.GetActive NextId] set code [compileCSharpWith [subst { using System; using System.Data.SQLite; @@ -507,16 +615,24 @@ } } }] true true true results errors System.Data.SQLite.dll] if {$code eq "Ok"} then { set ::compiled(13) $id; # NOTE: Compiled OK. - object invoke _Dynamic${id}.Test${id} BackupAndGetData + if {[catch { + object invoke _Dynamic${id}.Test${id} BackupAndGetData + showTest M + } error]} then { + if {[expectedError $error]} then { + showTest m + } else { + failTest $error + } + } } else { error $errors } } - showTest D } }] 0] }] ############################################################################# @@ -525,25 +641,25 @@ # # NOTE: Workload #14, PRAGMA integrity check statement. # lappend ::times(14) [lindex [time { setupDb $dstFileName "" "" "" "" "" true false - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { set result [sql execute -execute scalar $db \ "PRAGMA integrity_check;"] if {$result eq "ok"} then { - showTest E + showTest N } else { error [appendArgs "integrity check failed: " $result] } - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + } error]} then { + if {[expectedError $error]} then { + showTest n + } else { + failTest $error + } } } cleanupDb $dstFileName db false true false }] 0] }] @@ -553,20 +669,20 @@ set workload(15) [list [list srcFileName dstFileName table count] { # # NOTE: Workload #15, force managed garbage collection # lappend ::times(15) [lindex [time { - if {[catch { - for {set index 1} {$index <= $count} {incr index} { + for {set index 1} {$index <= $count} {incr index} { + if {[catch { collectGarbage $::test_channel - showTest F - } - } error]} then { - if {[expectedError $error]} then { - showTest * - } else { - failTest $error + showTest O + } error]} then { + if {[expectedError $error]} then { + showTest o + } else { + failTest $error + } } } }] 0] }] } -body { @@ -636,14 +752,22 @@ object removecallback [list apply $workload($index(0)) $fileName(1) \ $fileName(2) t1 $count(1)] } } + freeDbConnection + + cleanupLogging $logFileName + + rename cleanupLogging "" + rename setupLogging "" + unset -nocomplain result thread index workload noWorkload srcDb db \ - fileName compiled options count times + fileName compiled options count times logFileName logListener \ + connection } -constraints \ {eagle monoBug28 command.sql compile.DATA SQLite System.Data.SQLite} -result {}} ############################################################################### runSQLiteTestEpilogue runTestEpilogue ADDED Tests/thread.eagle Index: Tests/thread.eagle ================================================================== --- /dev/null +++ Tests/thread.eagle @@ -0,0 +1,437 @@ +############################################################################### +# +# thread.eagle -- +# +# Written by Joe Mistachkin. +# Released to the public domain, use at your own risk! +# +############################################################################### + +package require Eagle +package require Eagle.Library +package require Eagle.Test + +runTestPrologue + +############################################################################### + +package require System.Data.SQLite.Test +runSQLiteTestPrologue + +############################################################################### + +checkForSQLiteDirectories $test_channel true +set memory_used [reportSQLiteResources $test_channel true] + +############################################################################### + +# +# NOTE: How many test threads should be created and used for these tests? This +# value must be at least two. The first test thread (at index 0) just +# repeatedly calls the Thread.Abort() method on the other test threads +# that appear to be alive until all test threads appear to be dead. All +# other threads will attempt to open a connection, execute one or more +# queries against it, and then close it. +# +set count 10 + +############################################################################### + +runTest {test thread-1.1 {Thread.Abort() impact on native resources} -setup { + setupDb [set fileName thread-1.1.db] + + tputs $test_channel [appendArgs \ + "---- using a total of " $count " threads...\n"] +} -body { + sql execute $db { + CREATE TABLE t1(x INTEGER PRIMARY KEY, y BLOB); + INSERT INTO t1 (y) VALUES(RANDOMBLOB(1000)); + INSERT INTO t1 (y) VALUES(RANDOMBLOB(1000)); + INSERT INTO t1 (y) VALUES(RANDOMBLOB(1000)); + INSERT INTO t1 (y) VALUES(RANDOMBLOB(1000)); + INSERT INTO t1 (y) VALUES(RANDOMBLOB(1000)); + } + + # + # NOTE: The temporary directory must be reset here because it allocates + # some SQLite memory and this test requires an extremely accurate + # reading. + # + sql execute $db { + PRAGMA temp_store_directory = ""; + } + + # + # NOTE: Close the database now, freeing any native SQLite memory and/or + # resources that it may be using at this point; however, do not + # actually delete the database file because it is needed in the C# + # code for this test. + # + cleanupDb $fileName db true false false; unset -nocomplain db + + # + # NOTE: Setup the variables used in the dynamically substituted C# code + # for the main body of this test (below). + # + set id [object invoke Interpreter.GetActive NextId] + set dataSource [file join [getDatabaseDirectory] $fileName] + + set sql { \ + SELECT x, y FROM t1 ORDER BY x; \ + } + + unset -nocomplain results errors + + set code [compileCSharpWith [subst { + using System; + using System.Collections.Generic; + using System.Data.SQLite; + using System.Diagnostics; + using System.Threading; + using Eagle._Containers.Public; + + namespace _Dynamic${id} + { + public static class Test${id} + { + public static IntList RunTestThreads() + { + // + // NOTE: This is the total number of data bytes seen in the second + // column of the query result rows seen by all test threads. + // This value will vary greatly based upon how many loop + // iterations are performed prior to each test thread being + // aborted. + // + int sum = 0; + + // + // NOTE: This is the total number of query result rows seen by all + // the test threads. + // + int rows = 0; + + // + // NOTE: This is the total number of exceptions caught by all the + // test threads. + // + int errors = 0; + + // + // NOTE: This is the total number of test threads to create. + // + int count = ${count}; + + // + // NOTE: Create the array of thread objects. + // + Thread\[\] thread = new Thread\[count\]; + + // + // NOTE: Create a random number generator suitable for waiting a + // random number of milliseconds between each loop iteration + // and then selecting a random thread index. + // + Random random = new Random(); + + // + // NOTE: Create the event that will be used to synchronize all the + // created threads so that they start doing their actual test + // "work" at approximately the same time. This is also used + // to make sure that all test threads are inside of a try + // block before attempting to abort them. + // + using (ManualResetEvent goEvent = new ManualResetEvent(false)) + { + // + // NOTE: Create a (reusable) delegate that will contain the code + // that most of the created test threads are to execute. + // This code will open, use, and close a single database + // connection. Multiple commands and data readers will also + // be used. + // + ThreadStart threadStart1 = delegate() + { + try + { + // + // NOTE: Wait forever for the "GO" signal so that all threads + // can start working at approximately the same time. + // This is also used to make sure that all test threads + // are inside of a try block before attempting to abort + // them. + // + goEvent.WaitOne(); + + // + // NOTE: Create a new connection object. We purposely avoid + // putting this inside a "using" block to help test our + // cleanup via the garbage collector. + // + SQLiteConnection connection = new SQLiteConnection( + "Data Source=${dataSource};"); + + // + // NOTE: Open the connection. After this point, native memory + // and resources have been allocated by this thread. + // + connection.Open(); + + // + // NOTE: Loop forever until the first thread signals us to stop + // via calling Thread.Abort() method on our associated + // thread object. + // + while (true) + { + // + // NOTE: Create a dictionary to temporarily hold the values + // from the data reader for this loop iteration. + // + Dictionary results = + new Dictionary(); + + // + // NOTE: Create a new command object using the connection + // object for this thread. Again, avoid putting this + // inside a "using" block to help test our cleanup via + // the garbage collector. + // + SQLiteCommand command = new SQLiteCommand("${sql}", + connection); + + // + // NOTE: Execute the query and get the resulting data reader + // object. Again, avoid putting this inside a "using" + // block to help test our cleanup via the garbage + // collector. + // + SQLiteDataReader reader = command.ExecuteReader(); + + // + // NOTE: Start processing each available query result row. + // This processing (or any of the above processing) + // may be stopped at any time due to this test thread + // being aborted. + // + while (reader.Read()) + { + results.Add((long)reader\["x"\], + reader\["y"\] as byte\[\]); + + Interlocked.Add(ref sum, + results\[(long)reader\["x"\]\].Length); + + Interlocked.Increment(ref rows); + } + + // + // NOTE: Close the data reader for this loop iteration as we + // are done with it. + // + reader.Close(); + reader = null; + + // + // NOTE: Dispose the command for this loop iteration as we + // are done with it. + // + command.Dispose(); + command = null; + } + + // + // NOTE: Close the connection for this test thread as we are + // done with it. Since the above loop is infinite, it + // should only be exited via this test thread being + // aborted; therefore, execution should never reach this + // point. + // + connection.Close(); + connection = null; + } + catch (Exception e) + { + Interlocked.Increment(ref errors); + Trace.WriteLine(e); + } + }; + + // + // NOTE: Create a (reusable) delegate that will contain the code + // that half the created threads are to execute. This code + // will repeatedly call the Thread.Abort() method on all the + // other test threads until they all appear to be dead. + // + ThreadStart threadStart2 = delegate() + { + try + { + // + // NOTE: Wait forever for the "GO" signal so that all threads + // can start working at approximately the same time. + // This is also used to make sure that all test threads + // are inside of a try block before attempting to abort + // them. + // + goEvent.WaitOne(); + + // + // NOTE: Give the other test threads a slight head start to + // make sure that they are fully alive prior to trying + // to abort any of them. + // + Thread.Sleep(1000); + + // + // NOTE: Loop forever until all test threads appear to be dead. + // + while (true) + { + // + // NOTE: Wait a random number of milliseconds, up to a full + // second. + // + Thread.Sleep(random.Next(0, 1000)); + + // + // NOTE: Select a random thread to abort. + // + int index = random.Next(1, count); + + // + // NOTE: If the thread appears to be alive, try to abort + // it. + // + try + { + if (thread\[index\].IsAlive) + thread\[index\].Abort(); + } + catch (Exception e) + { + Trace.WriteLine(e); + } + + // + // NOTE: If all the other threads are dead, presumably due + // to being aborted, stop now. This check is simpler, + // and possibly more reliable, than checking if any of + // the test threads are still alive via their IsAlive + // property. + // + if (Interlocked.Increment(ref errors) == count) + break; + else + Interlocked.Decrement(ref errors); + } + } + catch (Exception e) + { + Interlocked.Increment(ref errors); + Trace.WriteLine(e); + } + }; + + // + // NOTE: Create each of the test threads with a suitable stack + // size. We must specify a stack size here because the + // default one for the process would be the same as the + // parent executable (the Eagle shell), which is 16MB, + // too large to be useful. + // + for (int index = 0; index < count; index++) + { + // + // NOTE: Figure out what kind of thread to create (i.e. one + // that uses a connection or one that calls the GC). + // + ThreadStart threadStart; + + if (index == 0) + threadStart = threadStart2; + else + threadStart = threadStart1; + + thread\[index\] = new Thread(threadStart, 1048576); + + // + // NOTE: Name each thread for a better debugging experience. + // + thread\[index\].Name = String.Format( + "[file rootname ${fileName}] #{0}", index); + } + + // + // NOTE: Start all the threads now. They should not actually do + // any of the test "work" until we signal the event. + // + for (int index = 0; index < count; index++) + thread\[index\].Start(); + + // + // NOTE: Send the signal that all threads should start doing + // their test "work" now. + // + goEvent.Set(); /* GO */ + + // + // NOTE: Wait forever for each thread to finish its test "work" + // and then die. + // + for (int index = 0; index < count; index++) + thread\[index\].Join(); + } + + // + // NOTE: Return a list of integers with total number of data bytes + // seen, total number of query result rows seen, and the total + // number of exceptions caught by all the test threads. + // + IntList counts = new IntList(); + + counts.Add(sum); + counts.Add(rows); + counts.Add(errors); + + return counts; + } + + /////////////////////////////////////////////////////////////////////// + + public static void Main() + { + // do nothing. + } + } + } + }] true true true results errors [list System.Data.SQLite.dll Eagle.dll]] + + list $code $results \ + [expr {[info exists errors] ? $errors : ""}] \ + [expr {$code eq "Ok" ? [catch { + object invoke _Dynamic${id}.Test${id} RunTestThreads + } result] : [set result ""]}] $result \ + [collectGarbage $test_channel] \ + [reportSQLiteResources $test_channel true] +} -cleanup { + cleanupDb $fileName + + unset -nocomplain result results errors code sql dataSource id db fileName +} -constraints \ +{eagle monoBug28 command.sql compile.DATA SQLite System.Data.SQLite} -match \ +regexp -result [appendArgs "^Ok System#CodeDom#Compiler#CompilerResults#\\d+\ +\\{\\} 0 \\{\\d+ \\d+ " $count "\\} \\{\\} " $memory_used \$]} + +############################################################################### + +unset -nocomplain count + +############################################################################### + +unset -nocomplain memory_used + +############################################################################### + +runSQLiteTestEpilogue +runTestEpilogue Index: Tests/tkt-2ce0870fad.eagle ================================================================== --- Tests/tkt-2ce0870fad.eagle +++ Tests/tkt-2ce0870fad.eagle @@ -22,14 +22,11 @@ # # NOTE: Make sure that SQLite core library is completely shutdown prior to # starting any of the tests in this file. # -if {[haveConstraint SQLite]} then { - object invoke -flags +NonPublic System.Data.SQLite.UnsafeNativeMethods \ - sqlite3_shutdown -} +shutdownSQLite $test_channel ############################################################################### for {set i 1} {$i < 3} {incr i} { runTest {test [appendArgs tkt-2ce0870fad-1. $i] {logging setup} -setup \ Index: Tests/tkt-996d13cd87.eagle ================================================================== --- Tests/tkt-996d13cd87.eagle +++ Tests/tkt-996d13cd87.eagle @@ -124,13 +124,14 @@ // using (ManualResetEvent goEvent = new ManualResetEvent(false)) { // // NOTE: Create a (reusable) delegate that will contain the code - // that half the created threads are to execute. This code - // will repeatedly create, open, and close a single database - // connection with (or without) pooling enabled. + // that most of the created test threads are going to + // execute. The code in this delegate will create, open, + // use, and close a single database connection with (or + // without) pooling enabled. // ThreadStart threadStart1 = delegate() { try { @@ -139,13 +140,13 @@ // can start working at approximately the same time. // goEvent.WaitOne(); // - // NOTE: Repeatedly try to create, open, and close a pooled - // database connection and then wait a random number of - // milliseconds before doing it again. + // NOTE: Try to create, open, and close a database connection + // and then wait a random number of milliseconds before + // doing it again. // Thread.Sleep(random.Next(0, 500)); SQLiteConnection connection = new SQLiteConnection( "Data Source=${dataSource};Pooling=${pooling};"); @@ -168,12 +169,13 @@ } }; // // NOTE: Create a (reusable) delegate that will contain the code - // that half the created threads are to execute. This code - // will repeatedly force a full garbage collection. + // that the garbage collection thread is to execute. The + // code in this delegate will attempt to force a full round + // of garbage collection several times. // ThreadStart threadStart2 = delegate() { try { Index: Tests/tkt-b4a7ddc83f.eagle ================================================================== --- Tests/tkt-b4a7ddc83f.eagle +++ Tests/tkt-b4a7ddc83f.eagle @@ -22,14 +22,11 @@ # # NOTE: Make sure that SQLite core library is completely shutdown prior to # starting any of the tests in this file. # -if {[haveConstraint SQLite]} then { - object invoke -flags +NonPublic System.Data.SQLite.UnsafeNativeMethods \ - sqlite3_shutdown -} +shutdownSQLite $test_channel ############################################################################### for {set i 1} {$i < 3} {incr i} { runTest {test [appendArgs tkt-b4a7ddc83f-1. $i] {logging shutdown} -setup \ Index: readme.htm ================================================================== --- readme.htm +++ readme.htm @@ -194,11 +194,12 @@
  • Add Visual Studio 2012 support to all the applicable solution/project files, their associated supporting files, and the test suite.
  • Add Visual Studio 2012 support to the redesigned designer support installer.
  • Allow opened connections to skip adding the extension functions included in the interop assembly via the new NoExtensionFunctions connection flag.
  • Support loading of SQLite extensions via the new EnableExtensions and LoadExtension methods of the SQLiteConnection class. Pursuant to [17045010df].
  • Add notifications before and after any connection is opened and closed, as well as other related notifications, via the new static Changed event.
  • -
  • Add an overload of the SQLiteLog.LogMessage method that takes a single string argument.
  • +
  • Add an overload of the SQLiteLog.LogMessage method that takes a single string parameter.
  • +
  • Add an overload of the SQLiteConnection.LogMessage method that takes a SQLiteErrorCode parameter.
  • All applicable calls into the SQLite core library now return a SQLiteErrorCode instead of an integer error code.
  • Make sure the error code of the SQLiteException class gets serialized.
  • Make the test project for the .NET Compact Framework more flexible.
  • When available, the new sqlite3_errstr function from the core library is used to get the error message for a specific return code.
  • The SetMemoryStatus, Shutdown, ResultCode, ExtendedResultCode, and SetAvRetry methods of the SQLiteConnection class now return a SQLiteErrorCode instead of an integer error code. ** Potentially Incompatible Change **
  • @@ -209,10 +210,11 @@
  • Implement more robust locking semantics for the CriticalHandle derived classes when compiled for the .NET Compact Framework.
  • Cache column indexes are they are looked up when using the SQLiteDataReader to improve performance.
  • Prevent the SQLiteConnection.Close method from throwing non-fatal exceptions during its disposal.
  • Rename the interop assembly functions sqlite3_cursor_rowid, sqlite3_context_collcompare, sqlite3_context_collseq, sqlite3_cursor_rowid, and sqlite3_table_cursor to include an "_interop" suffix. ** Potentially Incompatible Change **
  • Prevent the LastInsertRowId, MemoryUsed, and MemoryHighwater connection properties from throwing NotSupportedException when running on the .NET Compact Framework. Fix for [dd45aba387].
  • +
  • Add protection against ThreadAbortException asynchronously interrupting native resource initialization and finalization.
  • Add test automation for the Windows CE binaries.
  • 1.0.82.0 - September 3, 2012

    Index: test/TestCases.cs ================================================================== --- test/TestCases.cs +++ test/TestCases.cs @@ -1655,11 +1655,11 @@ cnn.Open(); logevents = 0; - cnn.LogMessage(1, "test log event"); + cnn.LogMessage(SQLiteErrorCode.Error, "test log event"); if (logevents != 1) throw new Exception(String.Format( "Log event count {0} incorrect.", logevents)); Index: www/news.wiki ================================================================== --- www/news.wiki +++ www/news.wiki @@ -10,11 +10,12 @@
  • Add Visual Studio 2012 support to all the applicable solution/project files, their associated supporting files, and the test suite.
  • Add Visual Studio 2012 support to the redesigned designer support installer.
  • Allow opened connections to skip adding the extension functions included in the interop assembly via the new NoExtensionFunctions connection flag.
  • Support loading of SQLite extensions via the new EnableExtensions and LoadExtension methods of the SQLiteConnection class. Pursuant to [17045010df].
  • Add notifications before and after any connection is opened and closed, as well as other related notifications, via the new static Changed event.
  • -
  • Add an overload of the SQLiteLog.LogMessage method that takes a single string argument.
  • +
  • Add an overload of the SQLiteLog.LogMessage method that takes a single string parameter.
  • +
  • Add an overload of the SQLiteConnection.LogMessage method that takes a SQLiteErrorCode parameter.
  • All applicable calls into the SQLite core library now return a SQLiteErrorCode instead of an integer error code.
  • Make sure the error code of the SQLiteException class gets serialized.
  • Make the test project for the .NET Compact Framework more flexible.
  • When available, the new sqlite3_errstr function from the core library is used to get the error message for a specific return code.
  • The SetMemoryStatus, Shutdown, ResultCode, ExtendedResultCode, and SetAvRetry methods of the SQLiteConnection class now return a SQLiteErrorCode instead of an integer error code. ** Potentially Incompatible Change **
  • @@ -25,10 +26,11 @@
  • Implement more robust locking semantics for the CriticalHandle derived classes when compiled for the .NET Compact Framework.
  • Cache column indexes are they are looked up when using the SQLiteDataReader to improve performance.
  • Prevent the SQLiteConnection.Close method from throwing non-fatal exceptions during its disposal.
  • Rename the interop assembly functions sqlite3_cursor_rowid, sqlite3_context_collcompare, sqlite3_context_collseq, sqlite3_cursor_rowid, and sqlite3_table_cursor to include an "_interop" suffix. ** Potentially Incompatible Change **
  • Prevent the LastInsertRowId, MemoryUsed, and MemoryHighwater connection properties from throwing NotSupportedException when running on the .NET Compact Framework. Fix for [dd45aba387].
  • +
  • Add protection against ThreadAbortException asynchronously interrupting native resource initialization and finalization.
  • Add test automation for the Windows CE binaries.
  • 1.0.82.0 - September 3, 2012