Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Update core SQLite to 3.7.7.1 release. |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | build-enhancements |
| Files: | files | file ages | folders |
| SHA1: |
f35ae2653cf9f8b665be5e76d153c0b9 |
| User & Date: | mistachkin 2011-06-30 02:43:23 |
Context
|
2011-06-30
| ||
| 06:17 | Update the VS property files to SQLite version 3.7.7.1. check-in: f10d138b59 user: mistachkin tags: build-enhancements | |
| 02:43 | Update core SQLite to 3.7.7.1 release. check-in: f35ae2653c user: mistachkin tags: build-enhancements | |
| 01:27 | When building the 'NativeOnly' solution configurations via the command line using the *.MSBuild.sln files, skip building any managed code. Revise naming of the setup packages. check-in: 0c50bcf546 user: mistachkin tags: build-enhancements | |
Changes
Changes to SQLite.Interop/src/core/sqlite3.c.
1 1 /****************************************************************************** 2 2 ** This file is an amalgamation of many separate C source files from SQLite 3 -** version 3.7.6.3. By combining all the individual C code files into this 3 +** version 3.7.7.1. By combining all the individual C code files into this 4 4 ** single large file, the entire code can be compiled as a single translation 5 5 ** unit. This allows many compilers to do optimizations that would not be 6 6 ** possible if the files were compiled separately. Performance improvements 7 7 ** of 5% or more are commonly seen when SQLite is compiled as a single 8 8 ** translation unit. 9 9 ** 10 10 ** This file is all you need to compile SQLite. To use SQLite in other ................................................................................ 646 646 ** string contains the date and time of the check-in (UTC) and an SHA1 647 647 ** hash of the entire source tree. 648 648 ** 649 649 ** See also: [sqlite3_libversion()], 650 650 ** [sqlite3_libversion_number()], [sqlite3_sourceid()], 651 651 ** [sqlite_version()] and [sqlite_source_id()]. 652 652 */ 653 -#define SQLITE_VERSION "3.7.6.3" 654 -#define SQLITE_VERSION_NUMBER 3007006 655 -#define SQLITE_SOURCE_ID "2011-05-19 13:26:54 ed1da510a239ea767a01dc332b667119fa3c908e" 653 +#define SQLITE_VERSION "3.7.7.1" 654 +#define SQLITE_VERSION_NUMBER 3007007 655 +#define SQLITE_SOURCE_ID "2011-06-28 17:39:05 af0d91adf497f5f36ec3813f04235a6e195a605f" 656 656 657 657 /* 658 658 ** CAPI3REF: Run-Time Library Version Numbers 659 659 ** KEYWORDS: sqlite3_version, sqlite3_sourceid 660 660 ** 661 661 ** These interfaces provide the same information as the [SQLITE_VERSION], 662 662 ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros ................................................................................ 849 849 ** 850 850 ** ^The sqlite3_exec() interface runs zero or more UTF-8 encoded, 851 851 ** semicolon-separate SQL statements passed into its 2nd argument, 852 852 ** in the context of the [database connection] passed in as its 1st 853 853 ** argument. ^If the callback function of the 3rd argument to 854 854 ** sqlite3_exec() is not NULL, then it is invoked for each result row 855 855 ** coming out of the evaluated SQL statements. ^The 4th argument to 856 -** to sqlite3_exec() is relayed through to the 1st argument of each 856 +** sqlite3_exec() is relayed through to the 1st argument of each 857 857 ** callback invocation. ^If the callback pointer to sqlite3_exec() 858 858 ** is NULL, then no callback is ever invoked and result rows are 859 859 ** ignored. 860 860 ** 861 861 ** ^If an error occurs while evaluating the SQL statements passed into 862 862 ** sqlite3_exec(), then execution of the current statement stops and 863 863 ** subsequent statements are skipped. ^If the 5th parameter to sqlite3_exec() ................................................................................ 914 914 ** KEYWORDS: {result code} {result codes} 915 915 ** 916 916 ** Many SQLite functions return an integer result code from the set shown 917 917 ** here in order to indicates success or failure. 918 918 ** 919 919 ** New error codes may be added in future versions of SQLite. 920 920 ** 921 -** See also: [SQLITE_IOERR_READ | extended result codes] 921 +** See also: [SQLITE_IOERR_READ | extended result codes], 922 +** [sqlite3_vtab_on_conflict()] [SQLITE_ROLLBACK | result codes]. 922 923 */ 923 924 #define SQLITE_OK 0 /* Successful result */ 924 925 /* beginning-of-error-codes */ 925 926 #define SQLITE_ERROR 1 /* SQL error or missing database */ 926 927 #define SQLITE_INTERNAL 2 /* Internal logic error in SQLite */ 927 928 #define SQLITE_PERM 3 /* Access permission denied */ 928 929 #define SQLITE_ABORT 4 /* Callback routine requested an abort */ ................................................................................ 991 992 #define SQLITE_IOERR_CHECKRESERVEDLOCK (SQLITE_IOERR | (14<<8)) 992 993 #define SQLITE_IOERR_LOCK (SQLITE_IOERR | (15<<8)) 993 994 #define SQLITE_IOERR_CLOSE (SQLITE_IOERR | (16<<8)) 994 995 #define SQLITE_IOERR_DIR_CLOSE (SQLITE_IOERR | (17<<8)) 995 996 #define SQLITE_IOERR_SHMOPEN (SQLITE_IOERR | (18<<8)) 996 997 #define SQLITE_IOERR_SHMSIZE (SQLITE_IOERR | (19<<8)) 997 998 #define SQLITE_IOERR_SHMLOCK (SQLITE_IOERR | (20<<8)) 999 +#define SQLITE_IOERR_SHMMAP (SQLITE_IOERR | (21<<8)) 1000 +#define SQLITE_IOERR_SEEK (SQLITE_IOERR | (22<<8)) 998 1001 #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) 999 1002 #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) 1000 1003 #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) 1004 +#define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) 1005 +#define SQLITE_READONLY_RECOVERY (SQLITE_READONLY | (1<<8)) 1006 +#define SQLITE_READONLY_CANTLOCK (SQLITE_READONLY | (2<<8)) 1001 1007 1002 1008 /* 1003 1009 ** CAPI3REF: Flags For File Open Operations 1004 1010 ** 1005 1011 ** These bit values are intended for use in the 1006 1012 ** 3rd parameter to the [sqlite3_open_v2()] interface and 1007 -** in the 4th parameter to the xOpen method of the 1008 -** [sqlite3_vfs] object. 1013 +** in the 4th parameter to the [sqlite3_vfs.xOpen] method. 1009 1014 */ 1010 1015 #define SQLITE_OPEN_READONLY 0x00000001 /* Ok for sqlite3_open_v2() */ 1011 1016 #define SQLITE_OPEN_READWRITE 0x00000002 /* Ok for sqlite3_open_v2() */ 1012 1017 #define SQLITE_OPEN_CREATE 0x00000004 /* Ok for sqlite3_open_v2() */ 1013 1018 #define SQLITE_OPEN_DELETEONCLOSE 0x00000008 /* VFS only */ 1014 1019 #define SQLITE_OPEN_EXCLUSIVE 0x00000010 /* VFS only */ 1015 1020 #define SQLITE_OPEN_AUTOPROXY 0x00000020 /* VFS only */ 1021 +#define SQLITE_OPEN_URI 0x00000040 /* Ok for sqlite3_open_v2() */ 1016 1022 #define SQLITE_OPEN_MAIN_DB 0x00000100 /* VFS only */ 1017 1023 #define SQLITE_OPEN_TEMP_DB 0x00000200 /* VFS only */ 1018 1024 #define SQLITE_OPEN_TRANSIENT_DB 0x00000400 /* VFS only */ 1019 1025 #define SQLITE_OPEN_MAIN_JOURNAL 0x00000800 /* VFS only */ 1020 1026 #define SQLITE_OPEN_TEMP_JOURNAL 0x00001000 /* VFS only */ 1021 1027 #define SQLITE_OPEN_SUBJOURNAL 0x00002000 /* VFS only */ 1022 1028 #define SQLITE_OPEN_MASTER_JOURNAL 0x00004000 /* VFS only */ ................................................................................ 1119 1125 struct sqlite3_file { 1120 1126 const struct sqlite3_io_methods *pMethods; /* Methods for an open file */ 1121 1127 }; 1122 1128 1123 1129 /* 1124 1130 ** CAPI3REF: OS Interface File Virtual Methods Object 1125 1131 ** 1126 -** Every file opened by the [sqlite3_vfs] xOpen method populates an 1132 +** Every file opened by the [sqlite3_vfs.xOpen] method populates an 1127 1133 ** [sqlite3_file] object (or, more commonly, a subclass of the 1128 1134 ** [sqlite3_file] object) with a pointer to an instance of this object. 1129 1135 ** This object defines the methods used to perform various operations 1130 1136 ** against the open file represented by the [sqlite3_file] object. 1131 1137 ** 1132 -** If the xOpen method sets the sqlite3_file.pMethods element 1138 +** If the [sqlite3_vfs.xOpen] method sets the sqlite3_file.pMethods element 1133 1139 ** to a non-NULL pointer, then the sqlite3_io_methods.xClose method 1134 -** may be invoked even if the xOpen reported that it failed. The 1135 -** only way to prevent a call to xClose following a failed xOpen 1136 -** is for the xOpen to set the sqlite3_file.pMethods element to NULL. 1140 +** may be invoked even if the [sqlite3_vfs.xOpen] reported that it failed. The 1141 +** only way to prevent a call to xClose following a failed [sqlite3_vfs.xOpen] 1142 +** is for the [sqlite3_vfs.xOpen] to set the sqlite3_file.pMethods element 1143 +** to NULL. 1137 1144 ** 1138 1145 ** The flags argument to xSync may be one of [SQLITE_SYNC_NORMAL] or 1139 1146 ** [SQLITE_SYNC_FULL]. The first choice is the normal fsync(). 1140 1147 ** The second choice is a Mac OS X style fullsync. The [SQLITE_SYNC_DATAONLY] 1141 1148 ** flag may be ORed in to indicate that only the data of the file 1142 1149 ** and not its inode needs to be synced. 1143 1150 ** ................................................................................ 1301 1308 typedef struct sqlite3_mutex sqlite3_mutex; 1302 1309 1303 1310 /* 1304 1311 ** CAPI3REF: OS Interface Object 1305 1312 ** 1306 1313 ** An instance of the sqlite3_vfs object defines the interface between 1307 1314 ** the SQLite core and the underlying operating system. The "vfs" 1308 -** in the name of the object stands for "virtual file system". 1315 +** in the name of the object stands for "virtual file system". See 1316 +** the [VFS | VFS documentation] for further information. 1309 1317 ** 1310 1318 ** The value of the iVersion field is initially 1 but may be larger in 1311 1319 ** future versions of SQLite. Additional fields may be appended to this 1312 1320 ** object when the iVersion value is increased. Note that the structure 1313 1321 ** of the sqlite3_vfs object changes in the transaction between 1314 1322 ** SQLite version 3.5.9 and 3.6.0 and yet the iVersion field was not 1315 1323 ** modified. ................................................................................ 1330 1338 ** or modify this field while holding a particular static mutex. 1331 1339 ** The application should never modify anything within the sqlite3_vfs 1332 1340 ** object once the object has been registered. 1333 1341 ** 1334 1342 ** The zName field holds the name of the VFS module. The name must 1335 1343 ** be unique across all VFS modules. 1336 1344 ** 1345 +** [[sqlite3_vfs.xOpen]] 1337 1346 ** ^SQLite guarantees that the zFilename parameter to xOpen 1338 1347 ** is either a NULL pointer or string obtained 1339 1348 ** from xFullPathname() with an optional suffix added. 1340 1349 ** ^If a suffix is added to the zFilename parameter, it will 1341 1350 ** consist of a single "-" character followed by no more than 1342 1351 ** 10 alphanumeric and/or "-" characters. 1343 1352 ** ^SQLite further guarantees that ................................................................................ 1407 1416 ** allocate the structure; it should just fill it in. Note that 1408 1417 ** the xOpen method must set the sqlite3_file.pMethods to either 1409 1418 ** a valid [sqlite3_io_methods] object or to NULL. xOpen must do 1410 1419 ** this even if the open fails. SQLite expects that the sqlite3_file.pMethods 1411 1420 ** element will be valid after xOpen returns regardless of the success 1412 1421 ** or failure of the xOpen call. 1413 1422 ** 1423 +** [[sqlite3_vfs.xAccess]] 1414 1424 ** ^The flags argument to xAccess() may be [SQLITE_ACCESS_EXISTS] 1415 1425 ** to test for the existence of a file, or [SQLITE_ACCESS_READWRITE] to 1416 1426 ** test whether a file is readable and writable, or [SQLITE_ACCESS_READ] 1417 1427 ** to test whether a file is at least readable. The file can be a 1418 1428 ** directory. 1419 1429 ** 1420 1430 ** ^SQLite will always allocate at least mxPathname+1 bytes for the ................................................................................ 1431 1441 ** of good-quality randomness into zOut. The return value is 1432 1442 ** the actual number of bytes of randomness obtained. 1433 1443 ** The xSleep() method causes the calling thread to sleep for at 1434 1444 ** least the number of microseconds given. ^The xCurrentTime() 1435 1445 ** method returns a Julian Day Number for the current date and time as 1436 1446 ** a floating point value. 1437 1447 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian 1438 -** Day Number multipled by 86400000 (the number of milliseconds in 1448 +** Day Number multiplied by 86400000 (the number of milliseconds in 1439 1449 ** a 24-hour day). 1440 1450 ** ^SQLite will use the xCurrentTimeInt64() method to get the current 1441 1451 ** date and time if that method is available (if iVersion is 2 or 1442 1452 ** greater and the function pointer is not NULL) and will fall back 1443 1453 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable. 1444 1454 ** 1445 1455 ** ^The xSetSystemCall(), xGetSystemCall(), and xNestSystemCall() interfaces ................................................................................ 1653 1663 ** [sqlite3_initialize()] or after shutdown by [sqlite3_shutdown()]. 1654 1664 ** ^If sqlite3_config() is called after [sqlite3_initialize()] and before 1655 1665 ** [sqlite3_shutdown()] then it will return SQLITE_MISUSE. 1656 1666 ** Note, however, that ^sqlite3_config() can be called as part of the 1657 1667 ** implementation of an application-defined [sqlite3_os_init()]. 1658 1668 ** 1659 1669 ** The first argument to sqlite3_config() is an integer 1660 -** [SQLITE_CONFIG_SINGLETHREAD | configuration option] that determines 1670 +** [configuration option] that determines 1661 1671 ** what property of SQLite is to be configured. Subsequent arguments 1662 -** vary depending on the [SQLITE_CONFIG_SINGLETHREAD | configuration option] 1672 +** vary depending on the [configuration option] 1663 1673 ** in the first argument. 1664 1674 ** 1665 1675 ** ^When a configuration option is set, sqlite3_config() returns [SQLITE_OK]. 1666 1676 ** ^If the option is unknown or SQLite is unable to set the option 1667 1677 ** then this routine returns a non-zero [error code]. 1668 1678 */ 1669 1679 SQLITE_API int sqlite3_config(int, ...); ................................................................................ 1765 1775 int (*xInit)(void*); /* Initialize the memory allocator */ 1766 1776 void (*xShutdown)(void*); /* Deinitialize the memory allocator */ 1767 1777 void *pAppData; /* Argument to xInit() and xShutdown() */ 1768 1778 }; 1769 1779 1770 1780 /* 1771 1781 ** CAPI3REF: Configuration Options 1782 +** KEYWORDS: {configuration option} 1772 1783 ** 1773 1784 ** These constants are the available integer configuration options that 1774 1785 ** can be passed as the first argument to the [sqlite3_config()] interface. 1775 1786 ** 1776 1787 ** New configuration options may be added in future releases of SQLite. 1777 1788 ** Existing configuration options might be discontinued. Applications 1778 1789 ** should check the return code from [sqlite3_config()] to make sure that 1779 1790 ** the call worked. The [sqlite3_config()] interface will return a 1780 1791 ** non-zero [error code] if a discontinued or unsupported configuration option 1781 1792 ** is invoked. 1782 1793 ** 1783 1794 ** <dl> 1784 -** <dt>SQLITE_CONFIG_SINGLETHREAD</dt> 1795 +** [[SQLITE_CONFIG_SINGLETHREAD]] <dt>SQLITE_CONFIG_SINGLETHREAD</dt> 1785 1796 ** <dd>There are no arguments to this option. ^This option sets the 1786 1797 ** [threading mode] to Single-thread. In other words, it disables 1787 1798 ** all mutexing and puts SQLite into a mode where it can only be used 1788 1799 ** by a single thread. ^If SQLite is compiled with 1789 1800 ** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then 1790 1801 ** it is not possible to change the [threading mode] from its default 1791 1802 ** value of Single-thread and so [sqlite3_config()] will return 1792 1803 ** [SQLITE_ERROR] if called with the SQLITE_CONFIG_SINGLETHREAD 1793 1804 ** configuration option.</dd> 1794 1805 ** 1795 -** <dt>SQLITE_CONFIG_MULTITHREAD</dt> 1806 +** [[SQLITE_CONFIG_MULTITHREAD]] <dt>SQLITE_CONFIG_MULTITHREAD</dt> 1796 1807 ** <dd>There are no arguments to this option. ^This option sets the 1797 1808 ** [threading mode] to Multi-thread. In other words, it disables 1798 1809 ** mutexing on [database connection] and [prepared statement] objects. 1799 1810 ** The application is responsible for serializing access to 1800 1811 ** [database connections] and [prepared statements]. But other mutexes 1801 1812 ** are enabled so that SQLite will be safe to use in a multi-threaded 1802 1813 ** environment as long as no two threads attempt to use the same 1803 1814 ** [database connection] at the same time. ^If SQLite is compiled with 1804 1815 ** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then 1805 1816 ** it is not possible to set the Multi-thread [threading mode] and 1806 1817 ** [sqlite3_config()] will return [SQLITE_ERROR] if called with the 1807 1818 ** SQLITE_CONFIG_MULTITHREAD configuration option.</dd> 1808 1819 ** 1809 -** <dt>SQLITE_CONFIG_SERIALIZED</dt> 1820 +** [[SQLITE_CONFIG_SERIALIZED]] <dt>SQLITE_CONFIG_SERIALIZED</dt> 1810 1821 ** <dd>There are no arguments to this option. ^This option sets the 1811 1822 ** [threading mode] to Serialized. In other words, this option enables 1812 1823 ** all mutexes including the recursive 1813 1824 ** mutexes on [database connection] and [prepared statement] objects. 1814 1825 ** In this mode (which is the default when SQLite is compiled with 1815 1826 ** [SQLITE_THREADSAFE=1]) the SQLite library will itself serialize access 1816 1827 ** to [database connections] and [prepared statements] so that the ................................................................................ 1818 1829 ** same [prepared statement] in different threads at the same time. 1819 1830 ** ^If SQLite is compiled with 1820 1831 ** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then 1821 1832 ** it is not possible to set the Serialized [threading mode] and 1822 1833 ** [sqlite3_config()] will return [SQLITE_ERROR] if called with the 1823 1834 ** SQLITE_CONFIG_SERIALIZED configuration option.</dd> 1824 1835 ** 1825 -** <dt>SQLITE_CONFIG_MALLOC</dt> 1836 +** [[SQLITE_CONFIG_MALLOC]] <dt>SQLITE_CONFIG_MALLOC</dt> 1826 1837 ** <dd> ^(This option takes a single argument which is a pointer to an 1827 1838 ** instance of the [sqlite3_mem_methods] structure. The argument specifies 1828 1839 ** alternative low-level memory allocation routines to be used in place of 1829 1840 ** the memory allocation routines built into SQLite.)^ ^SQLite makes 1830 1841 ** its own private copy of the content of the [sqlite3_mem_methods] structure 1831 1842 ** before the [sqlite3_config()] call returns.</dd> 1832 1843 ** 1833 -** <dt>SQLITE_CONFIG_GETMALLOC</dt> 1844 +** [[SQLITE_CONFIG_GETMALLOC]] <dt>SQLITE_CONFIG_GETMALLOC</dt> 1834 1845 ** <dd> ^(This option takes a single argument which is a pointer to an 1835 1846 ** instance of the [sqlite3_mem_methods] structure. The [sqlite3_mem_methods] 1836 1847 ** structure is filled with the currently defined memory allocation routines.)^ 1837 1848 ** This option can be used to overload the default memory allocation 1838 1849 ** routines with a wrapper that simulations memory allocation failure or 1839 1850 ** tracks memory usage, for example. </dd> 1840 1851 ** 1841 -** <dt>SQLITE_CONFIG_MEMSTATUS</dt> 1852 +** [[SQLITE_CONFIG_MEMSTATUS]] <dt>SQLITE_CONFIG_MEMSTATUS</dt> 1842 1853 ** <dd> ^This option takes single argument of type int, interpreted as a 1843 1854 ** boolean, which enables or disables the collection of memory allocation 1844 1855 ** statistics. ^(When memory allocation statistics are disabled, the 1845 1856 ** following SQLite interfaces become non-operational: 1846 1857 ** <ul> 1847 1858 ** <li> [sqlite3_memory_used()] 1848 1859 ** <li> [sqlite3_memory_highwater()] ................................................................................ 1850 1861 ** <li> [sqlite3_status()] 1851 1862 ** </ul>)^ 1852 1863 ** ^Memory allocation statistics are enabled by default unless SQLite is 1853 1864 ** compiled with [SQLITE_DEFAULT_MEMSTATUS]=0 in which case memory 1854 1865 ** allocation statistics are disabled by default. 1855 1866 ** </dd> 1856 1867 ** 1857 -** <dt>SQLITE_CONFIG_SCRATCH</dt> 1868 +** [[SQLITE_CONFIG_SCRATCH]] <dt>SQLITE_CONFIG_SCRATCH</dt> 1858 1869 ** <dd> ^This option specifies a static memory buffer that SQLite can use for 1859 1870 ** scratch memory. There are three arguments: A pointer an 8-byte 1860 1871 ** aligned memory buffer from which the scratch allocations will be 1861 1872 ** drawn, the size of each scratch allocation (sz), 1862 1873 ** and the maximum number of scratch allocations (N). The sz 1863 1874 ** argument must be a multiple of 16. 1864 1875 ** The first argument must be a pointer to an 8-byte aligned buffer ................................................................................ 1866 1877 ** ^SQLite will use no more than two scratch buffers per thread. So 1867 1878 ** N should be set to twice the expected maximum number of threads. 1868 1879 ** ^SQLite will never require a scratch buffer that is more than 6 1869 1880 ** times the database page size. ^If SQLite needs needs additional 1870 1881 ** scratch memory beyond what is provided by this configuration option, then 1871 1882 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd> 1872 1883 ** 1873 -** <dt>SQLITE_CONFIG_PAGECACHE</dt> 1884 +** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt> 1874 1885 ** <dd> ^This option specifies a static memory buffer that SQLite can use for 1875 -** the database page cache with the default page cache implemenation. 1886 +** the database page cache with the default page cache implementation. 1876 1887 ** This configuration should not be used if an application-define page 1877 1888 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option. 1878 1889 ** There are three arguments to this option: A pointer to 8-byte aligned 1879 1890 ** memory, the size of each page buffer (sz), and the number of pages (N). 1880 1891 ** The sz argument should be the size of the largest database page 1881 1892 ** (a power of two between 512 and 32768) plus a little extra for each 1882 1893 ** page header. ^The page header size is 20 to 40 bytes depending on ................................................................................ 1887 1898 ** memory needs for the first N pages that it adds to cache. ^If additional 1888 1899 ** page cache memory is needed beyond what is provided by this option, then 1889 1900 ** SQLite goes to [sqlite3_malloc()] for the additional storage space. 1890 1901 ** The pointer in the first argument must 1891 1902 ** be aligned to an 8-byte boundary or subsequent behavior of SQLite 1892 1903 ** will be undefined.</dd> 1893 1904 ** 1894 -** <dt>SQLITE_CONFIG_HEAP</dt> 1905 +** [[SQLITE_CONFIG_HEAP]] <dt>SQLITE_CONFIG_HEAP</dt> 1895 1906 ** <dd> ^This option specifies a static memory buffer that SQLite will use 1896 1907 ** for all of its dynamic memory allocation needs beyond those provided 1897 1908 ** for by [SQLITE_CONFIG_SCRATCH] and [SQLITE_CONFIG_PAGECACHE]. 1898 1909 ** There are three arguments: An 8-byte aligned pointer to the memory, 1899 1910 ** the number of bytes in the memory buffer, and the minimum allocation size. 1900 1911 ** ^If the first pointer (the memory pointer) is NULL, then SQLite reverts 1901 1912 ** to using its default memory allocator (the system malloc() implementation), ................................................................................ 1904 1915 ** [SQLITE_ENABLE_MEMSYS5] are defined, then the alternative memory 1905 1916 ** allocator is engaged to handle all of SQLites memory allocation needs. 1906 1917 ** The first pointer (the memory pointer) must be aligned to an 8-byte 1907 1918 ** boundary or subsequent behavior of SQLite will be undefined. 1908 1919 ** The minimum allocation size is capped at 2^12. Reasonable values 1909 1920 ** for the minimum allocation size are 2^5 through 2^8.</dd> 1910 1921 ** 1911 -** <dt>SQLITE_CONFIG_MUTEX</dt> 1922 +** [[SQLITE_CONFIG_MUTEX]] <dt>SQLITE_CONFIG_MUTEX</dt> 1912 1923 ** <dd> ^(This option takes a single argument which is a pointer to an 1913 1924 ** instance of the [sqlite3_mutex_methods] structure. The argument specifies 1914 1925 ** alternative low-level mutex routines to be used in place 1915 1926 ** the mutex routines built into SQLite.)^ ^SQLite makes a copy of the 1916 1927 ** content of the [sqlite3_mutex_methods] structure before the call to 1917 1928 ** [sqlite3_config()] returns. ^If SQLite is compiled with 1918 1929 ** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then 1919 1930 ** the entire mutexing subsystem is omitted from the build and hence calls to 1920 1931 ** [sqlite3_config()] with the SQLITE_CONFIG_MUTEX configuration option will 1921 1932 ** return [SQLITE_ERROR].</dd> 1922 1933 ** 1923 -** <dt>SQLITE_CONFIG_GETMUTEX</dt> 1934 +** [[SQLITE_CONFIG_GETMUTEX]] <dt>SQLITE_CONFIG_GETMUTEX</dt> 1924 1935 ** <dd> ^(This option takes a single argument which is a pointer to an 1925 1936 ** instance of the [sqlite3_mutex_methods] structure. The 1926 1937 ** [sqlite3_mutex_methods] 1927 1938 ** structure is filled with the currently defined mutex routines.)^ 1928 1939 ** This option can be used to overload the default mutex allocation 1929 1940 ** routines with a wrapper used to track mutex usage for performance 1930 1941 ** profiling or testing, for example. ^If SQLite is compiled with 1931 1942 ** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then 1932 1943 ** the entire mutexing subsystem is omitted from the build and hence calls to 1933 1944 ** [sqlite3_config()] with the SQLITE_CONFIG_GETMUTEX configuration option will 1934 1945 ** return [SQLITE_ERROR].</dd> 1935 1946 ** 1936 -** <dt>SQLITE_CONFIG_LOOKASIDE</dt> 1947 +** [[SQLITE_CONFIG_LOOKASIDE]] <dt>SQLITE_CONFIG_LOOKASIDE</dt> 1937 1948 ** <dd> ^(This option takes two arguments that determine the default 1938 1949 ** memory allocation for the lookaside memory allocator on each 1939 1950 ** [database connection]. The first argument is the 1940 1951 ** size of each lookaside buffer slot and the second is the number of 1941 1952 ** slots allocated to each database connection.)^ ^(This option sets the 1942 1953 ** <i>default</i> lookaside size. The [SQLITE_DBCONFIG_LOOKASIDE] 1943 1954 ** verb to [sqlite3_db_config()] can be used to change the lookaside 1944 1955 ** configuration on individual connections.)^ </dd> 1945 1956 ** 1946 -** <dt>SQLITE_CONFIG_PCACHE</dt> 1957 +** [[SQLITE_CONFIG_PCACHE]] <dt>SQLITE_CONFIG_PCACHE</dt> 1947 1958 ** <dd> ^(This option takes a single argument which is a pointer to 1948 1959 ** an [sqlite3_pcache_methods] object. This object specifies the interface 1949 1960 ** to a custom page cache implementation.)^ ^SQLite makes a copy of the 1950 1961 ** object and uses it for page cache memory allocations.</dd> 1951 1962 ** 1952 -** <dt>SQLITE_CONFIG_GETPCACHE</dt> 1963 +** [[SQLITE_CONFIG_GETPCACHE]] <dt>SQLITE_CONFIG_GETPCACHE</dt> 1953 1964 ** <dd> ^(This option takes a single argument which is a pointer to an 1954 1965 ** [sqlite3_pcache_methods] object. SQLite copies of the current 1955 1966 ** page cache implementation into that object.)^ </dd> 1956 1967 ** 1957 -** <dt>SQLITE_CONFIG_LOG</dt> 1968 +** [[SQLITE_CONFIG_LOG]] <dt>SQLITE_CONFIG_LOG</dt> 1958 1969 ** <dd> ^The SQLITE_CONFIG_LOG option takes two arguments: a pointer to a 1959 1970 ** function with a call signature of void(*)(void*,int,const char*), 1960 1971 ** and a pointer to void. ^If the function pointer is not NULL, it is 1961 1972 ** invoked by [sqlite3_log()] to process each logging event. ^If the 1962 1973 ** function pointer is NULL, the [sqlite3_log()] interface becomes a no-op. 1963 1974 ** ^The void pointer that is the second argument to SQLITE_CONFIG_LOG is 1964 1975 ** passed through as the first parameter to the application-defined logger ................................................................................ 1968 1979 ** [extended result code]. ^The third parameter passed to the logger is 1969 1980 ** log message after formatting via [sqlite3_snprintf()]. 1970 1981 ** The SQLite logging interface is not reentrant; the logger function 1971 1982 ** supplied by the application must not invoke any SQLite interface. 1972 1983 ** In a multi-threaded application, the application-defined logger 1973 1984 ** function must be threadsafe. </dd> 1974 1985 ** 1986 +** [[SQLITE_CONFIG_URI]] <dt>SQLITE_CONFIG_URI 1987 +** <dd> This option takes a single argument of type int. If non-zero, then 1988 +** URI handling is globally enabled. If the parameter is zero, then URI handling 1989 +** is globally disabled. If URI handling is globally enabled, all filenames 1990 +** passed to [sqlite3_open()], [sqlite3_open_v2()], [sqlite3_open16()] or 1991 +** specified as part of [ATTACH] commands are interpreted as URIs, regardless 1992 +** of whether or not the [SQLITE_OPEN_URI] flag is set when the database 1993 +** connection is opened. If it is globally disabled, filenames are 1994 +** only interpreted as URIs if the SQLITE_OPEN_URI flag is set when the 1995 +** database connection is opened. By default, URI handling is globally 1996 +** disabled. The default value may be changed by compiling with the 1997 +** [SQLITE_USE_URI] symbol defined. 1975 1998 ** </dl> 1976 1999 */ 1977 2000 #define SQLITE_CONFIG_SINGLETHREAD 1 /* nil */ 1978 2001 #define SQLITE_CONFIG_MULTITHREAD 2 /* nil */ 1979 2002 #define SQLITE_CONFIG_SERIALIZED 3 /* nil */ 1980 2003 #define SQLITE_CONFIG_MALLOC 4 /* sqlite3_mem_methods* */ 1981 2004 #define SQLITE_CONFIG_GETMALLOC 5 /* sqlite3_mem_methods* */ ................................................................................ 1986 2009 #define SQLITE_CONFIG_MUTEX 10 /* sqlite3_mutex_methods* */ 1987 2010 #define SQLITE_CONFIG_GETMUTEX 11 /* sqlite3_mutex_methods* */ 1988 2011 /* previously SQLITE_CONFIG_CHUNKALLOC 12 which is now unused. */ 1989 2012 #define SQLITE_CONFIG_LOOKASIDE 13 /* int int */ 1990 2013 #define SQLITE_CONFIG_PCACHE 14 /* sqlite3_pcache_methods* */ 1991 2014 #define SQLITE_CONFIG_GETPCACHE 15 /* sqlite3_pcache_methods* */ 1992 2015 #define SQLITE_CONFIG_LOG 16 /* xFunc, void* */ 2016 +#define SQLITE_CONFIG_URI 17 /* int */ 1993 2017 1994 2018 /* 1995 2019 ** CAPI3REF: Database Connection Configuration Options 1996 2020 ** 1997 2021 ** These constants are the available integer configuration options that 1998 2022 ** can be passed as the second argument to the [sqlite3_db_config()] interface. 1999 2023 ** ................................................................................ 2071 2095 ** as an undeclared column named ROWID, OID, or _ROWID_ as long as those 2072 2096 ** names are not also used by explicitly declared columns. ^If 2073 2097 ** the table has a column of type [INTEGER PRIMARY KEY] then that column 2074 2098 ** is another alias for the rowid. 2075 2099 ** 2076 2100 ** ^This routine returns the [rowid] of the most recent 2077 2101 ** successful [INSERT] into the database from the [database connection] 2078 -** in the first argument. ^If no successful [INSERT]s 2102 +** in the first argument. ^As of SQLite version 3.7.7, this routines 2103 +** records the last insert rowid of both ordinary tables and [virtual tables]. 2104 +** ^If no successful [INSERT]s 2079 2105 ** have ever occurred on that database connection, zero is returned. 2080 2106 ** 2081 -** ^(If an [INSERT] occurs within a trigger, then the [rowid] of the inserted 2082 -** row is returned by this routine as long as the trigger is running. 2083 -** But once the trigger terminates, the value returned by this routine 2084 -** reverts to the last value inserted before the trigger fired.)^ 2107 +** ^(If an [INSERT] occurs within a trigger or within a [virtual table] 2108 +** method, then this routine will return the [rowid] of the inserted 2109 +** row as long as the trigger or virtual table method is running. 2110 +** But once the trigger or virtual table method ends, the value returned 2111 +** by this routine reverts to what it was before the trigger or virtual 2112 +** table method began.)^ 2085 2113 ** 2086 2114 ** ^An [INSERT] that fails due to a constraint violation is not a 2087 2115 ** successful [INSERT] and does not change the value returned by this 2088 2116 ** routine. ^Thus INSERT OR FAIL, INSERT OR IGNORE, INSERT OR ROLLBACK, 2089 2117 ** and INSERT OR ABORT make no changes to the return value of this 2090 2118 ** routine when their insertion fails. ^(When INSERT OR REPLACE 2091 2119 ** encounters a constraint violation, it does not fail. The ................................................................................ 2740 2768 ** CAPI3REF: Authorizer Return Codes 2741 2769 ** 2742 2770 ** The [sqlite3_set_authorizer | authorizer callback function] must 2743 2771 ** return either [SQLITE_OK] or one of these two constants in order 2744 2772 ** to signal SQLite whether or not the action is permitted. See the 2745 2773 ** [sqlite3_set_authorizer | authorizer documentation] for additional 2746 2774 ** information. 2775 +** 2776 +** Note that SQLITE_IGNORE is also used as a [SQLITE_ROLLBACK | return code] 2777 +** from the [sqlite3_vtab_on_conflict()] interface. 2747 2778 */ 2748 2779 #define SQLITE_DENY 1 /* Abort the SQL statement with an error */ 2749 2780 #define SQLITE_IGNORE 2 /* Don't allow access, but don't generate an error */ 2750 2781 2751 2782 /* 2752 2783 ** CAPI3REF: Authorizer Action Codes 2753 2784 ** ................................................................................ 2862 2893 ** 2863 2894 */ 2864 2895 SQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*); 2865 2896 2866 2897 /* 2867 2898 ** CAPI3REF: Opening A New Database Connection 2868 2899 ** 2869 -** ^These routines open an SQLite database file whose name is given by the 2900 +** ^These routines open an SQLite database file as specified by the 2870 2901 ** filename argument. ^The filename argument is interpreted as UTF-8 for 2871 2902 ** sqlite3_open() and sqlite3_open_v2() and as UTF-16 in the native byte 2872 2903 ** order for sqlite3_open16(). ^(A [database connection] handle is usually 2873 2904 ** returned in *ppDb, even if an error occurs. The only exception is that 2874 2905 ** if SQLite is unable to allocate memory to hold the [sqlite3] object, 2875 2906 ** a NULL will be written into *ppDb instead of a pointer to the [sqlite3] 2876 2907 ** object.)^ ^(If the database is opened (and/or created) successfully, then ................................................................................ 2889 2920 ** 2890 2921 ** The sqlite3_open_v2() interface works like sqlite3_open() 2891 2922 ** except that it accepts two additional parameters for additional control 2892 2923 ** over the new database connection. ^(The flags parameter to 2893 2924 ** sqlite3_open_v2() can take one of 2894 2925 ** the following three values, optionally combined with the 2895 2926 ** [SQLITE_OPEN_NOMUTEX], [SQLITE_OPEN_FULLMUTEX], [SQLITE_OPEN_SHAREDCACHE], 2896 -** and/or [SQLITE_OPEN_PRIVATECACHE] flags:)^ 2927 +** [SQLITE_OPEN_PRIVATECACHE], and/or [SQLITE_OPEN_URI] flags:)^ 2897 2928 ** 2898 2929 ** <dl> 2899 2930 ** ^(<dt>[SQLITE_OPEN_READONLY]</dt> 2900 2931 ** <dd>The database is opened in read-only mode. If the database does not 2901 2932 ** already exist, an error is returned.</dd>)^ 2902 2933 ** 2903 2934 ** ^(<dt>[SQLITE_OPEN_READWRITE]</dt> ................................................................................ 2908 2939 ** ^(<dt>[SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]</dt> 2909 2940 ** <dd>The database is opened for reading and writing, and is created if 2910 2941 ** it does not already exist. This is the behavior that is always used for 2911 2942 ** sqlite3_open() and sqlite3_open16().</dd>)^ 2912 2943 ** </dl> 2913 2944 ** 2914 2945 ** If the 3rd parameter to sqlite3_open_v2() is not one of the 2915 -** combinations shown above or one of the combinations shown above combined 2916 -** with the [SQLITE_OPEN_NOMUTEX], [SQLITE_OPEN_FULLMUTEX], 2917 -** [SQLITE_OPEN_SHAREDCACHE] and/or [SQLITE_OPEN_PRIVATECACHE] flags, 2946 +** combinations shown above optionally combined with other 2947 +** [SQLITE_OPEN_READONLY | SQLITE_OPEN_* bits] 2918 2948 ** then the behavior is undefined. 2919 2949 ** 2920 2950 ** ^If the [SQLITE_OPEN_NOMUTEX] flag is set, then the database connection 2921 2951 ** opens in the multi-thread [threading mode] as long as the single-thread 2922 2952 ** mode has not been set at compile-time or start-time. ^If the 2923 2953 ** [SQLITE_OPEN_FULLMUTEX] flag is set then the database connection opens 2924 2954 ** in the serialized [threading mode] unless single-thread was 2925 2955 ** previously selected at compile-time or start-time. 2926 2956 ** ^The [SQLITE_OPEN_SHAREDCACHE] flag causes the database connection to be 2927 2957 ** eligible to use [shared cache mode], regardless of whether or not shared 2928 2958 ** cache is enabled using [sqlite3_enable_shared_cache()]. ^The 2929 2959 ** [SQLITE_OPEN_PRIVATECACHE] flag causes the database connection to not 2930 2960 ** participate in [shared cache mode] even if it is enabled. 2961 +** 2962 +** ^The fourth parameter to sqlite3_open_v2() is the name of the 2963 +** [sqlite3_vfs] object that defines the operating system interface that 2964 +** the new database connection should use. ^If the fourth parameter is 2965 +** a NULL pointer then the default [sqlite3_vfs] object is used. 2931 2966 ** 2932 2967 ** ^If the filename is ":memory:", then a private, temporary in-memory database 2933 2968 ** is created for the connection. ^This in-memory database will vanish when 2934 2969 ** the database connection is closed. Future versions of SQLite might 2935 2970 ** make use of additional special filenames that begin with the ":" character. 2936 2971 ** It is recommended that when a database filename actually does begin with 2937 2972 ** a ":" character you should prefix the filename with a pathname such as 2938 2973 ** "./" to avoid ambiguity. 2939 2974 ** 2940 2975 ** ^If the filename is an empty string, then a private, temporary 2941 2976 ** on-disk database will be created. ^This private database will be 2942 2977 ** automatically deleted as soon as the database connection is closed. 2943 2978 ** 2944 -** ^The fourth parameter to sqlite3_open_v2() is the name of the 2945 -** [sqlite3_vfs] object that defines the operating system interface that 2946 -** the new database connection should use. ^If the fourth parameter is 2947 -** a NULL pointer then the default [sqlite3_vfs] object is used. 2979 +** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3> 2980 +** 2981 +** ^If [URI filename] interpretation is enabled, and the filename argument 2982 +** begins with "file:", then the filename is interpreted as a URI. ^URI 2983 +** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is 2984 +** set in the fourth argument to sqlite3_open_v2(), or if it has 2985 +** been enabled globally using the [SQLITE_CONFIG_URI] option with the 2986 +** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option. 2987 +** As of SQLite version 3.7.7, URI filename interpretation is turned off 2988 +** by default, but future releases of SQLite might enable URI filename 2989 +** interpretation by default. See "[URI filenames]" for additional 2990 +** information. 2991 +** 2992 +** URI filenames are parsed according to RFC 3986. ^If the URI contains an 2993 +** authority, then it must be either an empty string or the string 2994 +** "localhost". ^If the authority is not an empty string or "localhost", an 2995 +** error is returned to the caller. ^The fragment component of a URI, if 2996 +** present, is ignored. 2997 +** 2998 +** ^SQLite uses the path component of the URI as the name of the disk file 2999 +** which contains the database. ^If the path begins with a '/' character, 3000 +** then it is interpreted as an absolute path. ^If the path does not begin 3001 +** with a '/' (meaning that the authority section is omitted from the URI) 3002 +** then the path is interpreted as a relative path. 3003 +** ^On windows, the first component of an absolute path 3004 +** is a drive specification (e.g. "C:"). 3005 +** 3006 +** [[core URI query parameters]] 3007 +** The query component of a URI may contain parameters that are interpreted 3008 +** either by SQLite itself, or by a [VFS | custom VFS implementation]. 3009 +** SQLite interprets the following three query parameters: 3010 +** 3011 +** <ul> 3012 +** <li> <b>vfs</b>: ^The "vfs" parameter may be used to specify the name of 3013 +** a VFS object that provides the operating system interface that should 3014 +** be used to access the database file on disk. ^If this option is set to 3015 +** an empty string the default VFS object is used. ^Specifying an unknown 3016 +** VFS is an error. ^If sqlite3_open_v2() is used and the vfs option is 3017 +** present, then the VFS specified by the option takes precedence over 3018 +** the value passed as the fourth parameter to sqlite3_open_v2(). 3019 +** 3020 +** <li> <b>mode</b>: ^(The mode parameter may be set to either "ro", "rw" or 3021 +** "rwc". Attempting to set it to any other value is an error)^. 3022 +** ^If "ro" is specified, then the database is opened for read-only 3023 +** access, just as if the [SQLITE_OPEN_READONLY] flag had been set in the 3024 +** third argument to sqlite3_prepare_v2(). ^If the mode option is set to 3025 +** "rw", then the database is opened for read-write (but not create) 3026 +** access, as if SQLITE_OPEN_READWRITE (but not SQLITE_OPEN_CREATE) had 3027 +** been set. ^Value "rwc" is equivalent to setting both 3028 +** SQLITE_OPEN_READWRITE and SQLITE_OPEN_CREATE. ^If sqlite3_open_v2() is 3029 +** used, it is an error to specify a value for the mode parameter that is 3030 +** less restrictive than that specified by the flags passed as the third 3031 +** parameter. 3032 +** 3033 +** <li> <b>cache</b>: ^The cache parameter may be set to either "shared" or 3034 +** "private". ^Setting it to "shared" is equivalent to setting the 3035 +** SQLITE_OPEN_SHAREDCACHE bit in the flags argument passed to 3036 +** sqlite3_open_v2(). ^Setting the cache parameter to "private" is 3037 +** equivalent to setting the SQLITE_OPEN_PRIVATECACHE bit. 3038 +** ^If sqlite3_open_v2() is used and the "cache" parameter is present in 3039 +** a URI filename, its value overrides any behaviour requested by setting 3040 +** SQLITE_OPEN_PRIVATECACHE or SQLITE_OPEN_SHAREDCACHE flag. 3041 +** </ul> 3042 +** 3043 +** ^Specifying an unknown parameter in the query component of a URI is not an 3044 +** error. Future versions of SQLite might understand additional query 3045 +** parameters. See "[query parameters with special meaning to SQLite]" for 3046 +** additional information. 3047 +** 3048 +** [[URI filename examples]] <h3>URI filename examples</h3> 3049 +** 3050 +** <table border="1" align=center cellpadding=5> 3051 +** <tr><th> URI filenames <th> Results 3052 +** <tr><td> file:data.db <td> 3053 +** Open the file "data.db" in the current directory. 3054 +** <tr><td> file:/home/fred/data.db<br> 3055 +** file:///home/fred/data.db <br> 3056 +** file://localhost/home/fred/data.db <br> <td> 3057 +** Open the database file "/home/fred/data.db". 3058 +** <tr><td> file://darkstar/home/fred/data.db <td> 3059 +** An error. "darkstar" is not a recognized authority. 3060 +** <tr><td style="white-space:nowrap"> 3061 +** file:///C:/Documents%20and%20Settings/fred/Desktop/data.db 3062 +** <td> Windows only: Open the file "data.db" on fred's desktop on drive 3063 +** C:. Note that the %20 escaping in this example is not strictly 3064 +** necessary - space characters can be used literally 3065 +** in URI filenames. 3066 +** <tr><td> file:data.db?mode=ro&cache=private <td> 3067 +** Open file "data.db" in the current directory for read-only access. 3068 +** Regardless of whether or not shared-cache mode is enabled by 3069 +** default, use a private cache. 3070 +** <tr><td> file:/home/fred/data.db?vfs=unix-nolock <td> 3071 +** Open file "/home/fred/data.db". Use the special VFS "unix-nolock". 3072 +** <tr><td> file:data.db?mode=readonly <td> 3073 +** An error. "readonly" is not a valid option for the "mode" parameter. 3074 +** </table> 3075 +** 3076 +** ^URI hexadecimal escape sequences (%HH) are supported within the path and 3077 +** query components of a URI. A hexadecimal escape sequence consists of a 3078 +** percent sign - "%" - followed by exactly two hexadecimal digits 3079 +** specifying an octet value. ^Before the path or query components of a 3080 +** URI filename are interpreted, they are encoded using UTF-8 and all 3081 +** hexadecimal escape sequences replaced by a single byte containing the 3082 +** corresponding octet. If this process generates an invalid UTF-8 encoding, 3083 +** the results are undefined. 2948 3084 ** 2949 3085 ** <b>Note to Windows users:</b> The encoding used for the filename argument 2950 3086 ** of sqlite3_open() and sqlite3_open_v2() must be UTF-8, not whatever 2951 3087 ** codepage is currently defined. Filenames containing international 2952 3088 ** characters must be converted to UTF-8 prior to passing them into 2953 3089 ** sqlite3_open() or sqlite3_open_v2(). 2954 3090 */ ................................................................................ 2962 3098 ); 2963 3099 SQLITE_API int sqlite3_open_v2( 2964 3100 const char *filename, /* Database filename (UTF-8) */ 2965 3101 sqlite3 **ppDb, /* OUT: SQLite db handle */ 2966 3102 int flags, /* Flags */ 2967 3103 const char *zVfs /* Name of VFS module to use */ 2968 3104 ); 3105 + 3106 +/* 3107 +** CAPI3REF: Obtain Values For URI Parameters 3108 +** 3109 +** This is a utility routine, useful to VFS implementations, that checks 3110 +** to see if a database file was a URI that contained a specific query 3111 +** parameter, and if so obtains the value of the query parameter. 3112 +** 3113 +** The zFilename argument is the filename pointer passed into the xOpen() 3114 +** method of a VFS implementation. The zParam argument is the name of the 3115 +** query parameter we seek. This routine returns the value of the zParam 3116 +** parameter if it exists. If the parameter does not exist, this routine 3117 +** returns a NULL pointer. 3118 +** 3119 +** If the zFilename argument to this function is not a pointer that SQLite 3120 +** passed into the xOpen VFS method, then the behavior of this routine 3121 +** is undefined and probably undesirable. 3122 +*/ 3123 +SQLITE_API const char *sqlite3_uri_parameter(const char *zFilename, const char *zParam); 3124 + 2969 3125 2970 3126 /* 2971 3127 ** CAPI3REF: Error Codes And Messages 2972 3128 ** 2973 3129 ** ^The sqlite3_errcode() interface returns the numeric [result code] or 2974 3130 ** [extended result code] for the most recent failed sqlite3_* API call 2975 3131 ** associated with a [database connection]. If a prior API call failed ................................................................................ 3078 3234 ** 3079 3235 ** These constants define various performance limits 3080 3236 ** that can be lowered at run-time using [sqlite3_limit()]. 3081 3237 ** The synopsis of the meanings of the various limits is shown below. 3082 3238 ** Additional information is available at [limits | Limits in SQLite]. 3083 3239 ** 3084 3240 ** <dl> 3085 -** ^(<dt>SQLITE_LIMIT_LENGTH</dt> 3241 +** [[SQLITE_LIMIT_LENGTH]] ^(<dt>SQLITE_LIMIT_LENGTH</dt> 3086 3242 ** <dd>The maximum size of any string or BLOB or table row, in bytes.<dd>)^ 3087 3243 ** 3088 -** ^(<dt>SQLITE_LIMIT_SQL_LENGTH</dt> 3244 +** [[SQLITE_LIMIT_SQL_LENGTH]] ^(<dt>SQLITE_LIMIT_SQL_LENGTH</dt> 3089 3245 ** <dd>The maximum length of an SQL statement, in bytes.</dd>)^ 3090 3246 ** 3091 -** ^(<dt>SQLITE_LIMIT_COLUMN</dt> 3247 +** [[SQLITE_LIMIT_COLUMN]] ^(<dt>SQLITE_LIMIT_COLUMN</dt> 3092 3248 ** <dd>The maximum number of columns in a table definition or in the 3093 3249 ** result set of a [SELECT] or the maximum number of columns in an index 3094 3250 ** or in an ORDER BY or GROUP BY clause.</dd>)^ 3095 3251 ** 3096 -** ^(<dt>SQLITE_LIMIT_EXPR_DEPTH</dt> 3252 +** [[SQLITE_LIMIT_EXPR_DEPTH]] ^(<dt>SQLITE_LIMIT_EXPR_DEPTH</dt> 3097 3253 ** <dd>The maximum depth of the parse tree on any expression.</dd>)^ 3098 3254 ** 3099 -** ^(<dt>SQLITE_LIMIT_COMPOUND_SELECT</dt> 3255 +** [[SQLITE_LIMIT_COMPOUND_SELECT]] ^(<dt>SQLITE_LIMIT_COMPOUND_SELECT</dt> 3100 3256 ** <dd>The maximum number of terms in a compound SELECT statement.</dd>)^ 3101 3257 ** 3102 -** ^(<dt>SQLITE_LIMIT_VDBE_OP</dt> 3258 +** [[SQLITE_LIMIT_VDBE_OP]] ^(<dt>SQLITE_LIMIT_VDBE_OP</dt> 3103 3259 ** <dd>The maximum number of instructions in a virtual machine program 3104 3260 ** used to implement an SQL statement. This limit is not currently 3105 3261 ** enforced, though that might be added in some future release of 3106 3262 ** SQLite.</dd>)^ 3107 3263 ** 3108 -** ^(<dt>SQLITE_LIMIT_FUNCTION_ARG</dt> 3264 +** [[SQLITE_LIMIT_FUNCTION_ARG]] ^(<dt>SQLITE_LIMIT_FUNCTION_ARG</dt> 3109 3265 ** <dd>The maximum number of arguments on a function.</dd>)^ 3110 3266 ** 3111 -** ^(<dt>SQLITE_LIMIT_ATTACHED</dt> 3267 +** [[SQLITE_LIMIT_ATTACHED]] ^(<dt>SQLITE_LIMIT_ATTACHED</dt> 3112 3268 ** <dd>The maximum number of [ATTACH | attached databases].)^</dd> 3113 3269 ** 3270 +** [[SQLITE_LIMIT_LIKE_PATTERN_LENGTH]] 3114 3271 ** ^(<dt>SQLITE_LIMIT_LIKE_PATTERN_LENGTH</dt> 3115 3272 ** <dd>The maximum length of the pattern argument to the [LIKE] or 3116 3273 ** [GLOB] operators.</dd>)^ 3117 3274 ** 3275 +** [[SQLITE_LIMIT_VARIABLE_NUMBER]] 3118 3276 ** ^(<dt>SQLITE_LIMIT_VARIABLE_NUMBER</dt> 3119 3277 ** <dd>The maximum index number of any [parameter] in an SQL statement.)^ 3120 3278 ** 3121 -** ^(<dt>SQLITE_LIMIT_TRIGGER_DEPTH</dt> 3279 +** [[SQLITE_LIMIT_TRIGGER_DEPTH]] ^(<dt>SQLITE_LIMIT_TRIGGER_DEPTH</dt> 3122 3280 ** <dd>The maximum depth of recursion for triggers.</dd>)^ 3123 3281 ** </dl> 3124 3282 */ 3125 3283 #define SQLITE_LIMIT_LENGTH 0 3126 3284 #define SQLITE_LIMIT_SQL_LENGTH 1 3127 3285 #define SQLITE_LIMIT_COLUMN 2 3128 3286 #define SQLITE_LIMIT_EXPR_DEPTH 3 ................................................................................ 3643 3801 ** [SQLITE_DONE], [SQLITE_ROW], [SQLITE_ERROR], or [SQLITE_MISUSE]. 3644 3802 ** ^With the "v2" interface, any of the other [result codes] or 3645 3803 ** [extended result codes] might be returned as well. 3646 3804 ** 3647 3805 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the 3648 3806 ** database locks it needs to do its job. ^If the statement is a [COMMIT] 3649 3807 ** or occurs outside of an explicit transaction, then you can retry the 3650 -** statement. If the statement is not a [COMMIT] and occurs within a 3808 +** statement. If the statement is not a [COMMIT] and occurs within an 3651 3809 ** explicit transaction then you should rollback the transaction before 3652 3810 ** continuing. 3653 3811 ** 3654 3812 ** ^[SQLITE_DONE] means that the statement has finished executing 3655 3813 ** successfully. sqlite3_step() should not be called again on this virtual 3656 3814 ** machine without first calling [sqlite3_reset()] to reset the virtual 3657 3815 ** machine back to its initial state. ................................................................................ 3922 4080 SQLITE_API int sqlite3_column_type(sqlite3_stmt*, int iCol); 3923 4081 SQLITE_API sqlite3_value *sqlite3_column_value(sqlite3_stmt*, int iCol); 3924 4082 3925 4083 /* 3926 4084 ** CAPI3REF: Destroy A Prepared Statement Object 3927 4085 ** 3928 4086 ** ^The sqlite3_finalize() function is called to delete a [prepared statement]. 3929 -** ^If the most recent evaluation of the statement encountered no errors or 4087 +** ^If the most recent evaluation of the statement encountered no errors 3930 4088 ** or if the statement is never been evaluated, then sqlite3_finalize() returns 3931 4089 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then 3932 4090 ** sqlite3_finalize(S) returns the appropriate [error code] or 3933 4091 ** [extended error code]. 3934 4092 ** 3935 4093 ** ^The sqlite3_finalize(S) routine can be called at any point during 3936 4094 ** the life cycle of [prepared statement] S: ................................................................................ 5149 5307 int (*xSync)(sqlite3_vtab *pVTab); 5150 5308 int (*xCommit)(sqlite3_vtab *pVTab); 5151 5309 int (*xRollback)(sqlite3_vtab *pVTab); 5152 5310 int (*xFindFunction)(sqlite3_vtab *pVtab, int nArg, const char *zName, 5153 5311 void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), 5154 5312 void **ppArg); 5155 5313 int (*xRename)(sqlite3_vtab *pVtab, const char *zNew); 5314 + /* The methods above are in version 1 of the sqlite_module object. Those 5315 + ** below are for version 2 and greater. */ 5316 + int (*xSavepoint)(sqlite3_vtab *pVTab, int); 5317 + int (*xRelease)(sqlite3_vtab *pVTab, int); 5318 + int (*xRollbackTo)(sqlite3_vtab *pVTab, int); 5156 5319 }; 5157 5320 5158 5321 /* 5159 5322 ** CAPI3REF: Virtual Table Indexing Information 5160 5323 ** KEYWORDS: sqlite3_index_info 5161 5324 ** 5162 5325 ** The sqlite3_index_info structure and its substructures is used as part ................................................................................ 5831 5994 ** ^The implementation is not required to provided versions of these 5832 5995 ** routines that actually work. If the implementation does not provide working 5833 5996 ** versions of these routines, it should at least provide stubs that always 5834 5997 ** return true so that one does not get spurious assertion failures. 5835 5998 ** 5836 5999 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then 5837 6000 ** the routine should return 1. This seems counter-intuitive since 5838 -** clearly the mutex cannot be held if it does not exist. But the 6001 +** clearly the mutex cannot be held if it does not exist. But 5839 6002 ** the reason the mutex does not exist is because the build is not 5840 6003 ** using mutexes. And we do not want the assert() containing the 5841 6004 ** call to sqlite3_mutex_held() to fail, so a non-zero return is 5842 6005 ** the appropriate thing to do. ^The sqlite3_mutex_notheld() 5843 6006 ** interface should also return 1 when given a NULL pointer. 5844 6007 */ 5845 6008 #ifndef NDEBUG ................................................................................ 5954 6117 #define SQLITE_TESTCTRL_ASSERT 12 5955 6118 #define SQLITE_TESTCTRL_ALWAYS 13 5956 6119 #define SQLITE_TESTCTRL_RESERVE 14 5957 6120 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 5958 6121 #define SQLITE_TESTCTRL_ISKEYWORD 16 5959 6122 #define SQLITE_TESTCTRL_PGHDRSZ 17 5960 6123 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18 5961 -#define SQLITE_TESTCTRL_LAST 18 6124 +#define SQLITE_TESTCTRL_LOCALTIME_FAULT 19 6125 +#define SQLITE_TESTCTRL_LAST 19 5962 6126 5963 6127 /* 5964 6128 ** CAPI3REF: SQLite Runtime Status 5965 6129 ** 5966 6130 ** ^This interface is used to retrieve runtime status information 5967 6131 ** about the performance of SQLite, and optionally to reset various 5968 6132 ** highwater marks. ^The first argument is an integer code for 5969 6133 ** the specific parameter to measure. ^(Recognized integer codes 5970 -** are of the form [SQLITE_STATUS_MEMORY_USED | SQLITE_STATUS_...].)^ 6134 +** are of the form [status parameters | SQLITE_STATUS_...].)^ 5971 6135 ** ^The current value of the parameter is returned into *pCurrent. 5972 6136 ** ^The highest recorded value is returned in *pHighwater. ^If the 5973 6137 ** resetFlag is true, then the highest record value is reset after 5974 6138 ** *pHighwater is written. ^(Some parameters do not record the highest 5975 6139 ** value. For those parameters 5976 6140 ** nothing is written into *pHighwater and the resetFlag is ignored.)^ 5977 6141 ** ^(Other parameters record only the highwater mark and not the current ................................................................................ 5990 6154 ** See also: [sqlite3_db_status()] 5991 6155 */ 5992 6156 SQLITE_API int sqlite3_status(int op, int *pCurrent, int *pHighwater, int resetFlag); 5993 6157 5994 6158 5995 6159 /* 5996 6160 ** CAPI3REF: Status Parameters 6161 +** KEYWORDS: {status parameters} 5997 6162 ** 5998 6163 ** These integer constants designate various run-time status parameters 5999 6164 ** that can be returned by [sqlite3_status()]. 6000 6165 ** 6001 6166 ** <dl> 6002 -** ^(<dt>SQLITE_STATUS_MEMORY_USED</dt> 6167 +** [[SQLITE_STATUS_MEMORY_USED]] ^(<dt>SQLITE_STATUS_MEMORY_USED</dt> 6003 6168 ** <dd>This parameter is the current amount of memory checked out 6004 6169 ** using [sqlite3_malloc()], either directly or indirectly. The 6005 6170 ** figure includes calls made to [sqlite3_malloc()] by the application 6006 6171 ** and internal memory usage by the SQLite library. Scratch memory 6007 6172 ** controlled by [SQLITE_CONFIG_SCRATCH] and auxiliary page-cache 6008 6173 ** memory controlled by [SQLITE_CONFIG_PAGECACHE] is not included in 6009 6174 ** this parameter. The amount returned is the sum of the allocation 6010 6175 ** sizes as reported by the xSize method in [sqlite3_mem_methods].</dd>)^ 6011 6176 ** 6012 -** ^(<dt>SQLITE_STATUS_MALLOC_SIZE</dt> 6177 +** [[SQLITE_STATUS_MALLOC_SIZE]] ^(<dt>SQLITE_STATUS_MALLOC_SIZE</dt> 6013 6178 ** <dd>This parameter records the largest memory allocation request 6014 6179 ** handed to [sqlite3_malloc()] or [sqlite3_realloc()] (or their 6015 6180 ** internal equivalents). Only the value returned in the 6016 6181 ** *pHighwater parameter to [sqlite3_status()] is of interest. 6017 6182 ** The value written into the *pCurrent parameter is undefined.</dd>)^ 6018 6183 ** 6019 -** ^(<dt>SQLITE_STATUS_MALLOC_COUNT</dt> 6184 +** [[SQLITE_STATUS_MALLOC_COUNT]] ^(<dt>SQLITE_STATUS_MALLOC_COUNT</dt> 6020 6185 ** <dd>This parameter records the number of separate memory allocations 6021 6186 ** currently checked out.</dd>)^ 6022 6187 ** 6023 -** ^(<dt>SQLITE_STATUS_PAGECACHE_USED</dt> 6188 +** [[SQLITE_STATUS_PAGECACHE_USED]] ^(<dt>SQLITE_STATUS_PAGECACHE_USED</dt> 6024 6189 ** <dd>This parameter returns the number of pages used out of the 6025 6190 ** [pagecache memory allocator] that was configured using 6026 6191 ** [SQLITE_CONFIG_PAGECACHE]. The 6027 6192 ** value returned is in pages, not in bytes.</dd>)^ 6028 6193 ** 6194 +** [[SQLITE_STATUS_PAGECACHE_OVERFLOW]] 6029 6195 ** ^(<dt>SQLITE_STATUS_PAGECACHE_OVERFLOW</dt> 6030 6196 ** <dd>This parameter returns the number of bytes of page cache 6031 6197 ** allocation which could not be satisfied by the [SQLITE_CONFIG_PAGECACHE] 6032 6198 ** buffer and where forced to overflow to [sqlite3_malloc()]. The 6033 6199 ** returned value includes allocations that overflowed because they 6034 6200 ** where too large (they were larger than the "sz" parameter to 6035 6201 ** [SQLITE_CONFIG_PAGECACHE]) and allocations that overflowed because 6036 6202 ** no space was left in the page cache.</dd>)^ 6037 6203 ** 6038 -** ^(<dt>SQLITE_STATUS_PAGECACHE_SIZE</dt> 6204 +** [[SQLITE_STATUS_PAGECACHE_SIZE]] ^(<dt>SQLITE_STATUS_PAGECACHE_SIZE</dt> 6039 6205 ** <dd>This parameter records the largest memory allocation request 6040 6206 ** handed to [pagecache memory allocator]. Only the value returned in the 6041 6207 ** *pHighwater parameter to [sqlite3_status()] is of interest. 6042 6208 ** The value written into the *pCurrent parameter is undefined.</dd>)^ 6043 6209 ** 6044 -** ^(<dt>SQLITE_STATUS_SCRATCH_USED</dt> 6210 +** [[SQLITE_STATUS_SCRATCH_USED]] ^(<dt>SQLITE_STATUS_SCRATCH_USED</dt> 6045 6211 ** <dd>This parameter returns the number of allocations used out of the 6046 6212 ** [scratch memory allocator] configured using 6047 6213 ** [SQLITE_CONFIG_SCRATCH]. The value returned is in allocations, not 6048 6214 ** in bytes. Since a single thread may only have one scratch allocation 6049 6215 ** outstanding at time, this parameter also reports the number of threads 6050 6216 ** using scratch memory at the same time.</dd>)^ 6051 6217 ** 6052 -** ^(<dt>SQLITE_STATUS_SCRATCH_OVERFLOW</dt> 6218 +** [[SQLITE_STATUS_SCRATCH_OVERFLOW]] ^(<dt>SQLITE_STATUS_SCRATCH_OVERFLOW</dt> 6053 6219 ** <dd>This parameter returns the number of bytes of scratch memory 6054 6220 ** allocation which could not be satisfied by the [SQLITE_CONFIG_SCRATCH] 6055 6221 ** buffer and where forced to overflow to [sqlite3_malloc()]. The values 6056 6222 ** returned include overflows because the requested allocation was too 6057 6223 ** larger (that is, because the requested allocation was larger than the 6058 6224 ** "sz" parameter to [SQLITE_CONFIG_SCRATCH]) and because no scratch buffer 6059 6225 ** slots were available. 6060 6226 ** </dd>)^ 6061 6227 ** 6062 -** ^(<dt>SQLITE_STATUS_SCRATCH_SIZE</dt> 6228 +** [[SQLITE_STATUS_SCRATCH_SIZE]] ^(<dt>SQLITE_STATUS_SCRATCH_SIZE</dt> 6063 6229 ** <dd>This parameter records the largest memory allocation request 6064 6230 ** handed to [scratch memory allocator]. Only the value returned in the 6065 6231 ** *pHighwater parameter to [sqlite3_status()] is of interest. 6066 6232 ** The value written into the *pCurrent parameter is undefined.</dd>)^ 6067 6233 ** 6068 -** ^(<dt>SQLITE_STATUS_PARSER_STACK</dt> 6234 +** [[SQLITE_STATUS_PARSER_STACK]] ^(<dt>SQLITE_STATUS_PARSER_STACK</dt> 6069 6235 ** <dd>This parameter records the deepest parser stack. It is only 6070 6236 ** meaningful if SQLite is compiled with [YYTRACKMAXSTACKDEPTH].</dd>)^ 6071 6237 ** </dl> 6072 6238 ** 6073 6239 ** New status parameters may be added from time to time. 6074 6240 */ 6075 6241 #define SQLITE_STATUS_MEMORY_USED 0 ................................................................................ 6086 6252 /* 6087 6253 ** CAPI3REF: Database Connection Status 6088 6254 ** 6089 6255 ** ^This interface is used to retrieve runtime status information 6090 6256 ** about a single [database connection]. ^The first argument is the 6091 6257 ** database connection object to be interrogated. ^The second argument 6092 6258 ** is an integer constant, taken from the set of 6093 -** [SQLITE_DBSTATUS_LOOKASIDE_USED | SQLITE_DBSTATUS_*] macros, that 6259 +** [SQLITE_DBSTATUS options], that 6094 6260 ** determines the parameter to interrogate. The set of 6095 -** [SQLITE_DBSTATUS_LOOKASIDE_USED | SQLITE_DBSTATUS_*] macros is likely 6261 +** [SQLITE_DBSTATUS options] is likely 6096 6262 ** to grow in future releases of SQLite. 6097 6263 ** 6098 6264 ** ^The current value of the requested parameter is written into *pCur 6099 6265 ** and the highest instantaneous value is written into *pHiwtr. ^If 6100 6266 ** the resetFlg is true, then the highest instantaneous value is 6101 6267 ** reset back down to the current value. 6102 6268 ** ................................................................................ 6105 6271 ** 6106 6272 ** See also: [sqlite3_status()] and [sqlite3_stmt_status()]. 6107 6273 */ 6108 6274 SQLITE_API int sqlite3_db_status(sqlite3*, int op, int *pCur, int *pHiwtr, int resetFlg); 6109 6275 6110 6276 /* 6111 6277 ** CAPI3REF: Status Parameters for database connections 6278 +** KEYWORDS: {SQLITE_DBSTATUS options} 6112 6279 ** 6113 6280 ** These constants are the available integer "verbs" that can be passed as 6114 6281 ** the second argument to the [sqlite3_db_status()] interface. 6115 6282 ** 6116 6283 ** New verbs may be added in future releases of SQLite. Existing verbs 6117 6284 ** might be discontinued. Applications should check the return code from 6118 6285 ** [sqlite3_db_status()] to make sure that the call worked. 6119 6286 ** The [sqlite3_db_status()] interface will return a non-zero error code 6120 6287 ** if a discontinued or unsupported verb is invoked. 6121 6288 ** 6122 6289 ** <dl> 6123 -** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_USED</dt> 6290 +** [[SQLITE_DBSTATUS_LOOKASIDE_USED]] ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_USED</dt> 6124 6291 ** <dd>This parameter returns the number of lookaside memory slots currently 6125 6292 ** checked out.</dd>)^ 6126 6293 ** 6127 -** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_HIT</dt> 6294 +** [[SQLITE_DBSTATUS_LOOKASIDE_HIT]] ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_HIT</dt> 6128 6295 ** <dd>This parameter returns the number malloc attempts that were 6129 6296 ** satisfied using lookaside memory. Only the high-water value is meaningful; 6130 6297 ** the current value is always zero.)^ 6131 6298 ** 6299 +** [[SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE]] 6132 6300 ** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE</dt> 6133 6301 ** <dd>This parameter returns the number malloc attempts that might have 6134 6302 ** been satisfied using lookaside memory but failed due to the amount of 6135 6303 ** memory requested being larger than the lookaside slot size. 6136 6304 ** Only the high-water value is meaningful; 6137 6305 ** the current value is always zero.)^ 6138 6306 ** 6307 +** [[SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL]] 6139 6308 ** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL</dt> 6140 6309 ** <dd>This parameter returns the number malloc attempts that might have 6141 6310 ** been satisfied using lookaside memory but failed due to all lookaside 6142 6311 ** memory already being in use. 6143 6312 ** Only the high-water value is meaningful; 6144 6313 ** the current value is always zero.)^ 6145 6314 ** 6146 -** ^(<dt>SQLITE_DBSTATUS_CACHE_USED</dt> 6315 +** [[SQLITE_DBSTATUS_CACHE_USED]] ^(<dt>SQLITE_DBSTATUS_CACHE_USED</dt> 6147 6316 ** <dd>This parameter returns the approximate number of of bytes of heap 6148 6317 ** memory used by all pager caches associated with the database connection.)^ 6149 6318 ** ^The highwater mark associated with SQLITE_DBSTATUS_CACHE_USED is always 0. 6150 6319 ** 6151 -** ^(<dt>SQLITE_DBSTATUS_SCHEMA_USED</dt> 6320 +** [[SQLITE_DBSTATUS_SCHEMA_USED]] ^(<dt>SQLITE_DBSTATUS_SCHEMA_USED</dt> 6152 6321 ** <dd>This parameter returns the approximate number of of bytes of heap 6153 6322 ** memory used to store the schema for all databases associated 6154 6323 ** with the connection - main, temp, and any [ATTACH]-ed databases.)^ 6155 6324 ** ^The full amount of memory used by the schemas is reported, even if the 6156 6325 ** schema memory is shared with other database connections due to 6157 6326 ** [shared cache mode] being enabled. 6158 6327 ** ^The highwater mark associated with SQLITE_DBSTATUS_SCHEMA_USED is always 0. 6159 6328 ** 6160 -** ^(<dt>SQLITE_DBSTATUS_STMT_USED</dt> 6329 +** [[SQLITE_DBSTATUS_STMT_USED]] ^(<dt>SQLITE_DBSTATUS_STMT_USED</dt> 6161 6330 ** <dd>This parameter returns the approximate number of of bytes of heap 6162 6331 ** and lookaside memory used by all prepared statements associated with 6163 6332 ** the database connection.)^ 6164 6333 ** ^The highwater mark associated with SQLITE_DBSTATUS_STMT_USED is always 0. 6165 6334 ** </dd> 6166 6335 ** </dl> 6167 6336 */ ................................................................................ 6175 6344 #define SQLITE_DBSTATUS_MAX 6 /* Largest defined DBSTATUS */ 6176 6345 6177 6346 6178 6347 /* 6179 6348 ** CAPI3REF: Prepared Statement Status 6180 6349 ** 6181 6350 ** ^(Each prepared statement maintains various 6182 -** [SQLITE_STMTSTATUS_SORT | counters] that measure the number 6351 +** [SQLITE_STMTSTATUS counters] that measure the number 6183 6352 ** of times it has performed specific operations.)^ These counters can 6184 6353 ** be used to monitor the performance characteristics of the prepared 6185 6354 ** statements. For example, if the number of table steps greatly exceeds 6186 6355 ** the number of table searches or result rows, that would tend to indicate 6187 6356 ** that the prepared statement is using a full table scan rather than 6188 6357 ** an index. 6189 6358 ** 6190 6359 ** ^(This interface is used to retrieve and reset counter values from 6191 6360 ** a [prepared statement]. The first argument is the prepared statement 6192 6361 ** object to be interrogated. The second argument 6193 -** is an integer code for a specific [SQLITE_STMTSTATUS_SORT | counter] 6362 +** is an integer code for a specific [SQLITE_STMTSTATUS counter] 6194 6363 ** to be interrogated.)^ 6195 6364 ** ^The current value of the requested counter is returned. 6196 6365 ** ^If the resetFlg is true, then the counter is reset to zero after this 6197 6366 ** interface call returns. 6198 6367 ** 6199 6368 ** See also: [sqlite3_status()] and [sqlite3_db_status()]. 6200 6369 */ 6201 6370 SQLITE_API int sqlite3_stmt_status(sqlite3_stmt*, int op,int resetFlg); 6202 6371 6203 6372 /* 6204 6373 ** CAPI3REF: Status Parameters for prepared statements 6374 +** KEYWORDS: {SQLITE_STMTSTATUS counter} {SQLITE_STMTSTATUS counters} 6205 6375 ** 6206 6376 ** These preprocessor macros define integer codes that name counter 6207 6377 ** values associated with the [sqlite3_stmt_status()] interface. 6208 6378 ** The meanings of the various counters are as follows: 6209 6379 ** 6210 6380 ** <dl> 6211 -** <dt>SQLITE_STMTSTATUS_FULLSCAN_STEP</dt> 6381 +** [[SQLITE_STMTSTATUS_FULLSCAN_STEP]] <dt>SQLITE_STMTSTATUS_FULLSCAN_STEP</dt> 6212 6382 ** <dd>^This is the number of times that SQLite has stepped forward in 6213 6383 ** a table as part of a full table scan. Large numbers for this counter 6214 6384 ** may indicate opportunities for performance improvement through 6215 6385 ** careful use of indices.</dd> 6216 6386 ** 6217 -** <dt>SQLITE_STMTSTATUS_SORT</dt> 6387 +** [[SQLITE_STMTSTATUS_SORT]] <dt>SQLITE_STMTSTATUS_SORT</dt> 6218 6388 ** <dd>^This is the number of sort operations that have occurred. 6219 6389 ** A non-zero value in this counter may indicate an opportunity to 6220 6390 ** improvement performance through careful use of indices.</dd> 6221 6391 ** 6222 -** <dt>SQLITE_STMTSTATUS_AUTOINDEX</dt> 6392 +** [[SQLITE_STMTSTATUS_AUTOINDEX]] <dt>SQLITE_STMTSTATUS_AUTOINDEX</dt> 6223 6393 ** <dd>^This is the number of rows inserted into transient indices that 6224 6394 ** were created automatically in order to help joins run faster. 6225 6395 ** A non-zero value in this counter may indicate an opportunity to 6226 6396 ** improvement performance by adding permanent indices that do not 6227 6397 ** need to be reinitialized each time the statement is run.</dd> 6228 6398 ** 6229 6399 ** </dl> ................................................................................ 6266 6436 ** The built-in page cache is recommended for most uses. 6267 6437 ** 6268 6438 ** ^(The contents of the sqlite3_pcache_methods structure are copied to an 6269 6439 ** internal buffer by SQLite within the call to [sqlite3_config]. Hence 6270 6440 ** the application may discard the parameter after the call to 6271 6441 ** [sqlite3_config()] returns.)^ 6272 6442 ** 6443 +** [[the xInit() page cache method]] 6273 6444 ** ^(The xInit() method is called once for each effective 6274 6445 ** call to [sqlite3_initialize()])^ 6275 6446 ** (usually only once during the lifetime of the process). ^(The xInit() 6276 6447 ** method is passed a copy of the sqlite3_pcache_methods.pArg value.)^ 6277 6448 ** The intent of the xInit() method is to set up global data structures 6278 6449 ** required by the custom page cache implementation. 6279 6450 ** ^(If the xInit() method is NULL, then the 6280 6451 ** built-in default page cache is used instead of the application defined 6281 6452 ** page cache.)^ 6282 6453 ** 6454 +** [[the xShutdown() page cache method]] 6283 6455 ** ^The xShutdown() method is called by [sqlite3_shutdown()]. 6284 6456 ** It can be used to clean up 6285 6457 ** any outstanding resources before process shutdown, if required. 6286 6458 ** ^The xShutdown() method may be NULL. 6287 6459 ** 6288 6460 ** ^SQLite automatically serializes calls to the xInit method, 6289 6461 ** so the xInit method need not be threadsafe. ^The ................................................................................ 6290 6462 ** xShutdown method is only called from [sqlite3_shutdown()] so it does 6291 6463 ** not need to be threadsafe either. All other methods must be threadsafe 6292 6464 ** in multithreaded applications. 6293 6465 ** 6294 6466 ** ^SQLite will never invoke xInit() more than once without an intervening 6295 6467 ** call to xShutdown(). 6296 6468 ** 6469 +** [[the xCreate() page cache methods]] 6297 6470 ** ^SQLite invokes the xCreate() method to construct a new cache instance. 6298 6471 ** SQLite will typically create one cache instance for each open database file, 6299 6472 ** though this is not guaranteed. ^The 6300 6473 ** first parameter, szPage, is the size in bytes of the pages that must 6301 6474 ** be allocated by the cache. ^szPage will not be a power of two. ^szPage 6302 6475 ** will the page size of the database file that is to be cached plus an 6303 6476 ** increment (here called "R") of less than 250. SQLite will use the ................................................................................ 6314 6487 ** it is purely advisory. ^On a cache where bPurgeable is false, SQLite will 6315 6488 ** never invoke xUnpin() except to deliberately delete a page. 6316 6489 ** ^In other words, calls to xUnpin() on a cache with bPurgeable set to 6317 6490 ** false will always have the "discard" flag set to true. 6318 6491 ** ^Hence, a cache created with bPurgeable false will 6319 6492 ** never contain any unpinned pages. 6320 6493 ** 6494 +** [[the xCachesize() page cache method]] 6321 6495 ** ^(The xCachesize() method may be called at any time by SQLite to set the 6322 6496 ** suggested maximum cache-size (number of pages stored by) the cache 6323 6497 ** instance passed as the first argument. This is the value configured using 6324 6498 ** the SQLite "[PRAGMA cache_size]" command.)^ As with the bPurgeable 6325 6499 ** parameter, the implementation is not required to do anything with this 6326 6500 ** value; it is advisory only. 6327 6501 ** 6502 +** [[the xPagecount() page cache methods]] 6328 6503 ** The xPagecount() method must return the number of pages currently 6329 6504 ** stored in the cache, both pinned and unpinned. 6330 6505 ** 6506 +** [[the xFetch() page cache methods]] 6331 6507 ** The xFetch() method locates a page in the cache and returns a pointer to 6332 6508 ** the page, or a NULL pointer. 6333 6509 ** A "page", in this context, means a buffer of szPage bytes aligned at an 6334 6510 ** 8-byte boundary. The page to be fetched is determined by the key. ^The 6335 -** mimimum key value is 1. After it has been retrieved using xFetch, the page 6511 +** minimum key value is 1. After it has been retrieved using xFetch, the page 6336 6512 ** is considered to be "pinned". 6337 6513 ** 6338 6514 ** If the requested page is already in the page cache, then the page cache 6339 6515 ** implementation must return a pointer to the page buffer with its content 6340 6516 ** intact. If the requested page is not already in the cache, then the 6341 6517 ** cache implementation should use the value of the createFlag 6342 6518 ** parameter to help it determined what action to take: ................................................................................ 6352 6528 ** 6353 6529 ** ^(SQLite will normally invoke xFetch() with a createFlag of 0 or 1. SQLite 6354 6530 ** will only use a createFlag of 2 after a prior call with a createFlag of 1 6355 6531 ** failed.)^ In between the to xFetch() calls, SQLite may 6356 6532 ** attempt to unpin one or more cache pages by spilling the content of 6357 6533 ** pinned pages to disk and synching the operating system disk cache. 6358 6534 ** 6535 +** [[the xUnpin() page cache method]] 6359 6536 ** ^xUnpin() is called by SQLite with a pointer to a currently pinned page 6360 6537 ** as its second argument. If the third parameter, discard, is non-zero, 6361 6538 ** then the page must be evicted from the cache. 6362 6539 ** ^If the discard parameter is 6363 6540 ** zero, then the page may be discarded or retained at the discretion of 6364 6541 ** page cache implementation. ^The page cache implementation 6365 6542 ** may choose to evict unpinned pages at any time. 6366 6543 ** 6367 6544 ** The cache must not perform any reference counting. A single 6368 6545 ** call to xUnpin() unpins the page regardless of the number of prior calls 6369 6546 ** to xFetch(). 6370 6547 ** 6548 +** [[the xRekey() page cache methods]] 6371 6549 ** The xRekey() method is used to change the key value associated with the 6372 6550 ** page passed as the second argument. If the cache 6373 6551 ** previously contains an entry associated with newKey, it must be 6374 6552 ** discarded. ^Any prior cache entry associated with newKey is guaranteed not 6375 6553 ** to be pinned. 6376 6554 ** 6377 6555 ** When SQLite calls the xTruncate() method, the cache must discard all 6378 6556 ** existing cache entries with page numbers (keys) greater than or equal 6379 6557 ** to the value of the iLimit parameter passed to xTruncate(). If any 6380 6558 ** of these pages are pinned, they are implicitly unpinned, meaning that 6381 6559 ** they can be safely discarded. 6382 6560 ** 6561 +** [[the xDestroy() page cache method]] 6383 6562 ** ^The xDestroy() method is used to delete a cache allocated by xCreate(). 6384 6563 ** All resources associated with the specified cache should be freed. ^After 6385 6564 ** calling the xDestroy() method, SQLite considers the [sqlite3_pcache*] 6386 6565 ** handle invalid, and will not use it with any other sqlite3_pcache_methods 6387 6566 ** functions. 6388 6567 */ 6389 6568 typedef struct sqlite3_pcache_methods sqlite3_pcache_methods; ................................................................................ 6438 6617 ** the data between the two databases, and finally 6439 6618 ** <li><b>sqlite3_backup_finish()</b> is called to release all resources 6440 6619 ** associated with the backup operation. 6441 6620 ** </ol>)^ 6442 6621 ** There should be exactly one call to sqlite3_backup_finish() for each 6443 6622 ** successful call to sqlite3_backup_init(). 6444 6623 ** 6445 -** <b>sqlite3_backup_init()</b> 6624 +** [[sqlite3_backup_init()]] <b>sqlite3_backup_init()</b> 6446 6625 ** 6447 6626 ** ^The D and N arguments to sqlite3_backup_init(D,N,S,M) are the 6448 6627 ** [database connection] associated with the destination database 6449 6628 ** and the database name, respectively. 6450 6629 ** ^The database name is "main" for the main database, "temp" for the 6451 6630 ** temporary database, or the name specified after the AS keyword in 6452 6631 ** an [ATTACH] statement for an attached database. ................................................................................ 6465 6644 ** [sqlite3_errmsg16()] functions. 6466 6645 ** ^A successful call to sqlite3_backup_init() returns a pointer to an 6467 6646 ** [sqlite3_backup] object. 6468 6647 ** ^The [sqlite3_backup] object may be used with the sqlite3_backup_step() and 6469 6648 ** sqlite3_backup_finish() functions to perform the specified backup 6470 6649 ** operation. 6471 6650 ** 6472 -** <b>sqlite3_backup_step()</b> 6651 +** [[sqlite3_backup_step()]] <b>sqlite3_backup_step()</b> 6473 6652 ** 6474 6653 ** ^Function sqlite3_backup_step(B,N) will copy up to N pages between 6475 6654 ** the source and destination databases specified by [sqlite3_backup] object B. 6476 6655 ** ^If N is negative, all remaining source pages are copied. 6477 6656 ** ^If sqlite3_backup_step(B,N) successfully copies N pages and there 6478 6657 ** are still more pages to be copied, then the function returns [SQLITE_OK]. 6479 6658 ** ^If sqlite3_backup_step(B,N) successfully finishes copying all pages ................................................................................ 6522 6701 ** external process or via a database connection other than the one being 6523 6702 ** used by the backup operation, then the backup will be automatically 6524 6703 ** restarted by the next call to sqlite3_backup_step(). ^If the source 6525 6704 ** database is modified by the using the same database connection as is used 6526 6705 ** by the backup operation, then the backup database is automatically 6527 6706 ** updated at the same time. 6528 6707 ** 6529 -** <b>sqlite3_backup_finish()</b> 6708 +** [[sqlite3_backup_finish()]] <b>sqlite3_backup_finish()</b> 6530 6709 ** 6531 6710 ** When sqlite3_backup_step() has returned [SQLITE_DONE], or when the 6532 6711 ** application wishes to abandon the backup operation, the application 6533 6712 ** should destroy the [sqlite3_backup] by passing it to sqlite3_backup_finish(). 6534 6713 ** ^The sqlite3_backup_finish() interfaces releases all 6535 6714 ** resources associated with the [sqlite3_backup] object. 6536 6715 ** ^If sqlite3_backup_step() has not yet returned [SQLITE_DONE], then any ................................................................................ 6545 6724 ** sqlite3_backup_step() call on the same [sqlite3_backup] object, then 6546 6725 ** sqlite3_backup_finish() returns the corresponding [error code]. 6547 6726 ** 6548 6727 ** ^A return of [SQLITE_BUSY] or [SQLITE_LOCKED] from sqlite3_backup_step() 6549 6728 ** is not a permanent error and does not affect the return value of 6550 6729 ** sqlite3_backup_finish(). 6551 6730 ** 6552 -** <b>sqlite3_backup_remaining(), sqlite3_backup_pagecount()</b> 6731 +** [[sqlite3_backup__remaining()]] [[sqlite3_backup_pagecount()]] 6732 +** <b>sqlite3_backup_remaining() and sqlite3_backup_pagecount()</b> 6553 6733 ** 6554 6734 ** ^Each call to sqlite3_backup_step() sets two values inside 6555 6735 ** the [sqlite3_backup] object: the number of pages still to be backed 6556 6736 ** up and the total number of pages in the source database file. 6557 6737 ** The sqlite3_backup_remaining() and sqlite3_backup_pagecount() interfaces 6558 6738 ** retrieve these two values, respectively. 6559 6739 ** ................................................................................ 6930 7110 ** [sqlite3_wal_checkpoint_v2()]. See the [sqlite3_wal_checkpoint_v2()] 6931 7111 ** documentation for additional information about the meaning and use of 6932 7112 ** each of these values. 6933 7113 */ 6934 7114 #define SQLITE_CHECKPOINT_PASSIVE 0 6935 7115 #define SQLITE_CHECKPOINT_FULL 1 6936 7116 #define SQLITE_CHECKPOINT_RESTART 2 7117 + 7118 +/* 7119 +** CAPI3REF: Virtual Table Interface Configuration 7120 +** 7121 +** This function may be called by either the [xConnect] or [xCreate] method 7122 +** of a [virtual table] implementation to configure 7123 +** various facets of the virtual table interface. 7124 +** 7125 +** If this interface is invoked outside the context of an xConnect or 7126 +** xCreate virtual table method then the behavior is undefined. 7127 +** 7128 +** At present, there is only one option that may be configured using 7129 +** this function. (See [SQLITE_VTAB_CONSTRAINT_SUPPORT].) Further options 7130 +** may be added in the future. 7131 +*/ 7132 +SQLITE_API int sqlite3_vtab_config(sqlite3*, int op, ...); 7133 + 7134 +/* 7135 +** CAPI3REF: Virtual Table Configuration Options 7136 +** 7137 +** These macros define the various options to the 7138 +** [sqlite3_vtab_config()] interface that [virtual table] implementations 7139 +** can use to customize and optimize their behavior. 7140 +** 7141 +** <dl> 7142 +** <dt>SQLITE_VTAB_CONSTRAINT_SUPPORT 7143 +** <dd>Calls of the form 7144 +** [sqlite3_vtab_config](db,SQLITE_VTAB_CONSTRAINT_SUPPORT,X) are supported, 7145 +** where X is an integer. If X is zero, then the [virtual table] whose 7146 +** [xCreate] or [xConnect] method invoked [sqlite3_vtab_config()] does not 7147 +** support constraints. In this configuration (which is the default) if 7148 +** a call to the [xUpdate] method returns [SQLITE_CONSTRAINT], then the entire 7149 +** statement is rolled back as if [ON CONFLICT | OR ABORT] had been 7150 +** specified as part of the users SQL statement, regardless of the actual 7151 +** ON CONFLICT mode specified. 7152 +** 7153 +** If X is non-zero, then the virtual table implementation guarantees 7154 +** that if [xUpdate] returns [SQLITE_CONSTRAINT], it will do so before 7155 +** any modifications to internal or persistent data structures have been made. 7156 +** If the [ON CONFLICT] mode is ABORT, FAIL, IGNORE or ROLLBACK, SQLite 7157 +** is able to roll back a statement or database transaction, and abandon 7158 +** or continue processing the current SQL statement as appropriate. 7159 +** If the ON CONFLICT mode is REPLACE and the [xUpdate] method returns 7160 +** [SQLITE_CONSTRAINT], SQLite handles this as if the ON CONFLICT mode 7161 +** had been ABORT. 7162 +** 7163 +** Virtual table implementations that are required to handle OR REPLACE 7164 +** must do so within the [xUpdate] method. If a call to the 7165 +** [sqlite3_vtab_on_conflict()] function indicates that the current ON 7166 +** CONFLICT policy is REPLACE, the virtual table implementation should 7167 +** silently replace the appropriate rows within the xUpdate callback and 7168 +** return SQLITE_OK. Or, if this is not possible, it may return 7169 +** SQLITE_CONSTRAINT, in which case SQLite falls back to OR ABORT 7170 +** constraint handling. 7171 +** </dl> 7172 +*/ 7173 +#define SQLITE_VTAB_CONSTRAINT_SUPPORT 1 7174 + 7175 +/* 7176 +** CAPI3REF: Determine The Virtual Table Conflict Policy 7177 +** 7178 +** This function may only be called from within a call to the [xUpdate] method 7179 +** of a [virtual table] implementation for an INSERT or UPDATE operation. ^The 7180 +** value returned is one of [SQLITE_ROLLBACK], [SQLITE_IGNORE], [SQLITE_FAIL], 7181 +** [SQLITE_ABORT], or [SQLITE_REPLACE], according to the [ON CONFLICT] mode 7182 +** of the SQL statement that triggered the call to the [xUpdate] method of the 7183 +** [virtual table]. 7184 +*/ 7185 +SQLITE_API int sqlite3_vtab_on_conflict(sqlite3 *); 7186 + 7187 +/* 7188 +** CAPI3REF: Conflict resolution modes 7189 +** 7190 +** These constants are returned by [sqlite3_vtab_on_conflict()] to 7191 +** inform a [virtual table] implementation what the [ON CONFLICT] mode 7192 +** is for the SQL statement being evaluated. 7193 +** 7194 +** Note that the [SQLITE_IGNORE] constant is also used as a potential 7195 +** return value from the [sqlite3_set_authorizer()] callback and that 7196 +** [SQLITE_ABORT] is also a [result code]. 7197 +*/ 7198 +#define SQLITE_ROLLBACK 1 7199 +/* #define SQLITE_IGNORE 2 // Also used by sqlite3_authorizer() callback */ 7200 +#define SQLITE_FAIL 3 7201 +/* #define SQLITE_ABORT 4 // Also an error code */ 7202 +#define SQLITE_REPLACE 5 7203 + 6937 7204 6938 7205 6939 7206 /* 6940 7207 ** Undo the hack that converts floating point types to integer for 6941 7208 ** builds on processors without floating point support. 6942 7209 */ 6943 7210 #ifdef SQLITE_OMIT_FLOATING_POINT ................................................................................ 7597 7864 typedef struct TableLock TableLock; 7598 7865 typedef struct Token Token; 7599 7866 typedef struct Trigger Trigger; 7600 7867 typedef struct TriggerPrg TriggerPrg; 7601 7868 typedef struct TriggerStep TriggerStep; 7602 7869 typedef struct UnpackedRecord UnpackedRecord; 7603 7870 typedef struct VTable VTable; 7871 +typedef struct VtabCtx VtabCtx; 7604 7872 typedef struct Walker Walker; 7605 7873 typedef struct WherePlan WherePlan; 7606 7874 typedef struct WhereInfo WhereInfo; 7607 7875 typedef struct WhereLevel WhereLevel; 7608 7876 7609 7877 /* 7610 7878 ** Defer sourcing vdbe.h and btree.h until after the "u8" and ................................................................................ 7653 7921 */ 7654 7922 typedef struct Btree Btree; 7655 7923 typedef struct BtCursor BtCursor; 7656 7924 typedef struct BtShared BtShared; 7657 7925 7658 7926 7659 7927 SQLITE_PRIVATE int sqlite3BtreeOpen( 7928 + sqlite3_vfs *pVfs, /* VFS to use with this b-tree */ 7660 7929 const char *zFilename, /* Name of database file to open */ 7661 7930 sqlite3 *db, /* Associated database connection */ 7662 7931 Btree **ppBtree, /* Return open Btree* here */ 7663 7932 int flags, /* Flags */ 7664 7933 int vfsFlags /* Flags passed through to VFS open */ 7665 7934 ); 7666 7935 ................................................................................ 8213 8482 SQLITE_PRIVATE int sqlite3VdbeAddOp0(Vdbe*,int); 8214 8483 SQLITE_PRIVATE int sqlite3VdbeAddOp1(Vdbe*,int,int); 8215 8484 SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int); 8216 8485 SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int); 8217 8486 SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int); 8218 8487 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int); 8219 8488 SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp); 8489 +SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe*,int,char*); 8220 8490 SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1); 8221 8491 SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2); 8222 8492 SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3); 8223 8493 SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u8 P5); 8224 8494 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr); 8225 8495 SQLITE_PRIVATE void sqlite3VdbeChangeToNoop(Vdbe*, int addr, int N); 8226 8496 SQLITE_PRIVATE void sqlite3VdbeChangeP4(Vdbe*, int addr, const char *zP4, int N); 8227 8497 SQLITE_PRIVATE void sqlite3VdbeUsesBtree(Vdbe*, int); 8228 8498 SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetOp(Vdbe*, int); 8229 8499 SQLITE_PRIVATE int sqlite3VdbeMakeLabel(Vdbe*); 8230 8500 SQLITE_PRIVATE void sqlite3VdbeRunOnlyOnce(Vdbe*); 8231 8501 SQLITE_PRIVATE void sqlite3VdbeDelete(Vdbe*); 8232 8502 SQLITE_PRIVATE void sqlite3VdbeDeleteObject(sqlite3*,Vdbe*); 8233 -SQLITE_PRIVATE void sqlite3VdbeMakeReady(Vdbe*,int,int,int,int,int,int); 8503 +SQLITE_PRIVATE void sqlite3VdbeMakeReady(Vdbe*,Parse*); 8234 8504 SQLITE_PRIVATE int sqlite3VdbeFinalize(Vdbe*); 8235 8505 SQLITE_PRIVATE void sqlite3VdbeResolveLabel(Vdbe*, int); 8236 8506 SQLITE_PRIVATE int sqlite3VdbeCurrentAddr(Vdbe*); 8237 8507 #ifdef SQLITE_DEBUG 8238 8508 SQLITE_PRIVATE int sqlite3VdbeAssertMayAbort(Vdbe *, int); 8239 8509 SQLITE_PRIVATE void sqlite3VdbeTrace(Vdbe*,FILE*); 8240 8510 #endif 8241 8511 SQLITE_PRIVATE void sqlite3VdbeResetStepResult(Vdbe*); 8512 +SQLITE_PRIVATE void sqlite3VdbeRewind(Vdbe*); 8242 8513 SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe*); 8243 8514 SQLITE_PRIVATE void sqlite3VdbeSetNumCols(Vdbe*,int); 8244 8515 SQLITE_PRIVATE int sqlite3VdbeSetColName(Vdbe*, int, int, const char *, void(*)(void*)); 8245 8516 SQLITE_PRIVATE void sqlite3VdbeCountChanges(Vdbe*); 8246 8517 SQLITE_PRIVATE sqlite3 *sqlite3VdbeDb(Vdbe*); 8247 8518 SQLITE_PRIVATE void sqlite3VdbeSetSql(Vdbe*, const char *z, int n, int); 8248 8519 SQLITE_PRIVATE void sqlite3VdbeSwap(Vdbe*,Vdbe*); ................................................................................ 9011 9282 ** Schema objects are automatically deallocated when the last Btree that 9012 9283 ** references them is destroyed. The TEMP Schema is manually freed by 9013 9284 ** sqlite3_close(). 9014 9285 * 9015 9286 ** A thread must be holding a mutex on the corresponding Btree in order 9016 9287 ** to access Schema content. This implies that the thread must also be 9017 9288 ** holding a mutex on the sqlite3 connection pointer that owns the Btree. 9018 -** For a TEMP Schema, on the connection mutex is required. 9289 +** For a TEMP Schema, only the connection mutex is required. 9019 9290 */ 9020 9291 struct Schema { 9021 9292 int schema_cookie; /* Database schema version number for this file */ 9022 9293 int iGeneration; /* Generation counter. Incremented with each change */ 9023 9294 Hash tblHash; /* All tables indexed by name */ 9024 9295 Hash idxHash; /* All (named) indices indexed by name */ 9025 9296 Hash trigHash; /* All triggers indexed by name */ ................................................................................ 9132 9403 ** consistently. 9133 9404 */ 9134 9405 struct sqlite3 { 9135 9406 sqlite3_vfs *pVfs; /* OS Interface */ 9136 9407 int nDb; /* Number of backends currently in use */ 9137 9408 Db *aDb; /* All backends */ 9138 9409 int flags; /* Miscellaneous flags. See below */ 9139 - int openFlags; /* Flags passed to sqlite3_vfs.xOpen() */ 9410 + unsigned int openFlags; /* Flags passed to sqlite3_vfs.xOpen() */ 9140 9411 int errCode; /* Most recent error code (SQLITE_*) */ 9141 9412 int errMask; /* & result codes with this before returning */ 9142 9413 u8 autoCommit; /* The auto-commit flag. */ 9143 9414 u8 temp_store; /* 1: file 2: memory 0: default */ 9144 9415 u8 mallocFailed; /* True if we have seen a malloc failure */ 9145 9416 u8 dfltLockMode; /* Default locking-mode for attached dbs */ 9146 9417 signed char nextAutovac; /* Autovac setting after VACUUM if >=0 */ 9147 9418 u8 suppressErr; /* Do not issue error messages if true */ 9419 + u8 vtabOnConflict; /* Value to return for s3_vtab_on_conflict() */ 9148 9420 int nextPagesize; /* Pagesize after VACUUM if >0 */ 9149 9421 int nTable; /* Number of tables in the database */ 9150 9422 CollSeq *pDfltColl; /* The default collating sequence (BINARY) */ 9151 9423 i64 lastRowid; /* ROWID of most recent insert (see above) */ 9152 9424 u32 magic; /* Magic number for detect library misuse */ 9153 9425 int nChange; /* Value returned by sqlite3_changes() */ 9154 9426 int nTotalChange; /* Value returned by sqlite3_total_changes() */ ................................................................................ 9199 9471 #ifndef SQLITE_OMIT_PROGRESS_CALLBACK 9200 9472 int (*xProgress)(void *); /* The progress callback */ 9201 9473 void *pProgressArg; /* Argument to the progress callback */ 9202 9474 int nProgressOps; /* Number of opcodes for progress callback */ 9203 9475 #endif 9204 9476 #ifndef SQLITE_OMIT_VIRTUALTABLE 9205 9477 Hash aModule; /* populated by sqlite3_create_module() */ 9206 - Table *pVTab; /* vtab with active Connect/Create method */ 9478 + VtabCtx *pVtabCtx; /* Context for active vtab connect/create */ 9207 9479 VTable **aVTrans; /* Virtual tables with open transactions */ 9208 9480 int nVTrans; /* Allocated size of aVTrans */ 9209 9481 VTable *pDisconnect; /* Disconnect these in next sqlite3_prepare() */ 9210 9482 #endif 9211 9483 FuncDefHash aFunc; /* Hash table of connection functions */ 9212 9484 Hash aCollSeq; /* All collating sequences */ 9213 9485 BusyHandler busyHandler; /* Busy callback */ ................................................................................ 9283 9555 #define SQLITE_QueryFlattener 0x01 /* Disable query flattening */ 9284 9556 #define SQLITE_ColumnCache 0x02 /* Disable the column cache */ 9285 9557 #define SQLITE_IndexSort 0x04 /* Disable indexes for sorting */ 9286 9558 #define SQLITE_IndexSearch 0x08 /* Disable indexes for searching */ 9287 9559 #define SQLITE_IndexCover 0x10 /* Disable index covering table */ 9288 9560 #define SQLITE_GroupByOrder 0x20 /* Disable GROUPBY cover of ORDERBY */ 9289 9561 #define SQLITE_FactorOutConst 0x40 /* Disable factoring out constants */ 9562 +#define SQLITE_IdxRealAsInt 0x80 /* Store REAL as INT in indices */ 9290 9563 #define SQLITE_OptMask 0xff /* Mask of all disablable opts */ 9291 9564 9292 9565 /* 9293 9566 ** Possible values for the sqlite.magic field. 9294 9567 ** The numbers are obtained at random and have no special meaning, other 9295 9568 ** than being distinct from one another. 9296 9569 */ ................................................................................ 9562 9835 ** the first argument. 9563 9836 */ 9564 9837 struct VTable { 9565 9838 sqlite3 *db; /* Database connection associated with this table */ 9566 9839 Module *pMod; /* Pointer to module implementation */ 9567 9840 sqlite3_vtab *pVtab; /* Pointer to vtab instance */ 9568 9841 int nRef; /* Number of pointers to this structure */ 9842 + u8 bConstraint; /* True if constraints are supported */ 9843 + int iSavepoint; /* Depth of the SAVEPOINT stack */ 9569 9844 VTable *pNext; /* Next in linked list (see above) */ 9570 9845 }; 9571 9846 9572 9847 /* 9573 9848 ** Each SQL table is represented in memory by an instance of the 9574 9849 ** following structure. 9575 9850 ** ................................................................................ 10556 10831 u8 disableTriggers; /* True to disable triggers */ 10557 10832 double nQueryLoop; /* Estimated number of iterations of a query */ 10558 10833 10559 10834 /* Above is constant between recursions. Below is reset before and after 10560 10835 ** each recursion */ 10561 10836 10562 10837 int nVar; /* Number of '?' variables seen in the SQL so far */ 10563 - int nVarExpr; /* Number of used slots in apVarExpr[] */ 10564 - int nVarExprAlloc; /* Number of allocated slots in apVarExpr[] */ 10565 - Expr **apVarExpr; /* Pointers to :aaa and $aaaa wildcard expressions */ 10838 + int nzVar; /* Number of available slots in azVar[] */ 10839 + char **azVar; /* Pointers to names of parameters */ 10566 10840 Vdbe *pReprepare; /* VM being reprepared (sqlite3Reprepare()) */ 10567 10841 int nAlias; /* Number of aliased result set columns */ 10568 10842 int nAliasAlloc; /* Number of allocated slots for aAlias[] */ 10569 10843 int *aAlias; /* Register used to hold aliased result */ 10570 10844 u8 explain; /* True if the EXPLAIN flag is found on the query */ 10571 10845 Token sNameToken; /* Token with unqualified schema object name */ 10572 10846 Token sLastToken; /* The last token parsed */ ................................................................................ 10750 11024 ** 10751 11025 ** This structure also contains some state information. 10752 11026 */ 10753 11027 struct Sqlite3Config { 10754 11028 int bMemstat; /* True to enable memory status */ 10755 11029 int bCoreMutex; /* True to enable core mutexing */ 10756 11030 int bFullMutex; /* True to enable full mutexing */ 11031 + int bOpenUri; /* True to interpret filenames as URIs */ 10757 11032 int mxStrlen; /* Maximum string length */ 10758 11033 int szLookaside; /* Default lookaside buffer size */ 10759 11034 int nLookaside; /* Default lookaside buffer count */ 10760 11035 sqlite3_mem_methods m; /* Low-level memory allocation interface */ 10761 11036 sqlite3_mutex_methods mutex; /* Low-level mutex interface */ 10762 11037 sqlite3_pcache_methods pcache; /* Low-level page-cache interface */ 10763 11038 void *pHeap; /* Heap storage space */ ................................................................................ 10778 11053 int isMutexInit; /* True after mutexes are initialized */ 10779 11054 int isMallocInit; /* True after malloc is initialized */ 10780 11055 int isPCacheInit; /* True after malloc is initialized */ 10781 11056 sqlite3_mutex *pInitMutex; /* Mutex used by sqlite3_initialize() */ 10782 11057 int nRefInitMutex; /* Number of users of pInitMutex */ 10783 11058 void (*xLog)(void*,int,const char*); /* Function for logging */ 10784 11059 void *pLogArg; /* First argument to xLog() */ 11060 + int bLocaltimeFault; /* True to fail localtime() calls */ 10785 11061 }; 10786 11062 10787 11063 /* 10788 11064 ** Context pointer passed down through the tree-walk. 10789 11065 */ 10790 11066 struct Walker { 10791 11067 int (*xExprCallback)(Walker*, Expr*); /* Callback for expressions */ ................................................................................ 10999 11275 SQLITE_PRIVATE void sqlite3AddNotNull(Parse*, int); 11000 11276 SQLITE_PRIVATE void sqlite3AddPrimaryKey(Parse*, ExprList*, int, int, int); 11001 11277 SQLITE_PRIVATE void sqlite3AddCheckConstraint(Parse*, Expr*); 11002 11278 SQLITE_PRIVATE void sqlite3AddColumnType(Parse*,Token*); 11003 11279 SQLITE_PRIVATE void sqlite3AddDefaultValue(Parse*,ExprSpan*); 11004 11280 SQLITE_PRIVATE void sqlite3AddCollateType(Parse*, Token*); 11005 11281 SQLITE_PRIVATE void sqlite3EndTable(Parse*,Token*,Token*,Select*); 11282 +SQLITE_PRIVATE int sqlite3ParseUri(const char*,const char*,unsigned int*, 11283 + sqlite3_vfs**,char**,char **); 11006 11284 11007 11285 SQLITE_PRIVATE Bitvec *sqlite3BitvecCreate(u32); 11008 11286 SQLITE_PRIVATE int sqlite3BitvecTest(Bitvec*, u32); 11009 11287 SQLITE_PRIVATE int sqlite3BitvecSet(Bitvec*, u32); 11010 11288 SQLITE_PRIVATE void sqlite3BitvecClear(Bitvec*, u32, void*); 11011 11289 SQLITE_PRIVATE void sqlite3BitvecDestroy(Bitvec*); 11012 11290 SQLITE_PRIVATE u32 sqlite3BitvecSize(Bitvec*); ................................................................................ 11203 11481 SQLITE_PRIVATE int sqlite3FixExprList(DbFixer*, ExprList*); 11204 11482 SQLITE_PRIVATE int sqlite3FixTriggerStep(DbFixer*, TriggerStep*); 11205 11483 SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8); 11206 11484 SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*); 11207 11485 SQLITE_PRIVATE int sqlite3Atoi(const char*); 11208 11486 SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar); 11209 11487 SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte); 11210 -SQLITE_PRIVATE int sqlite3Utf8Read(const u8*, const u8**); 11488 +SQLITE_PRIVATE u32 sqlite3Utf8Read(const u8*, const u8**); 11211 11489 11212 11490 /* 11213 11491 ** Routines to read and write variable-length integers. These used to 11214 11492 ** be defined locally, but now we use the varint routines in the util.c 11215 11493 ** file. Code should use the MACRO forms below, as the Varint32 versions 11216 11494 ** are coded to assume the single byte case is already handled (which 11217 11495 ** the MACRO form does). ................................................................................ 11249 11527 SQLITE_PRIVATE void sqlite3TableAffinityStr(Vdbe *, Table *); 11250 11528 SQLITE_PRIVATE char sqlite3CompareAffinity(Expr *pExpr, char aff2); 11251 11529 SQLITE_PRIVATE int sqlite3IndexAffinityOk(Expr *pExpr, char idx_affinity); 11252 11530 SQLITE_PRIVATE char sqlite3ExprAffinity(Expr *pExpr); 11253 11531 SQLITE_PRIVATE int sqlite3Atoi64(const char*, i64*, int, u8); 11254 11532 SQLITE_PRIVATE void sqlite3Error(sqlite3*, int, const char*,...); 11255 11533 SQLITE_PRIVATE void *sqlite3HexToBlob(sqlite3*, const char *z, int n); 11534 +SQLITE_PRIVATE u8 sqlite3HexToInt(int h); 11256 11535 SQLITE_PRIVATE int sqlite3TwoPartName(Parse *, Token *, Token *, Token **); 11257 11536 SQLITE_PRIVATE const char *sqlite3ErrStr(int); 11258 11537 SQLITE_PRIVATE int sqlite3ReadSchema(Parse *pParse); 11259 11538 SQLITE_PRIVATE CollSeq *sqlite3FindCollSeq(sqlite3*,u8 enc, const char*,int); 11260 11539 SQLITE_PRIVATE CollSeq *sqlite3LocateCollSeq(Parse *pParse, const char*zName); 11261 11540 SQLITE_PRIVATE CollSeq *sqlite3ExprCollSeq(Parse *pParse, Expr *pExpr); 11262 11541 SQLITE_PRIVATE Expr *sqlite3ExprSetColl(Expr*, CollSeq*); ................................................................................ 11264 11543 SQLITE_PRIVATE int sqlite3CheckCollSeq(Parse *, CollSeq *); 11265 11544 SQLITE_PRIVATE int sqlite3CheckObjectName(Parse *, const char *); 11266 11545 SQLITE_PRIVATE void sqlite3VdbeSetChanges(sqlite3 *, int); 11267 11546 SQLITE_PRIVATE int sqlite3AddInt64(i64*,i64); 11268 11547 SQLITE_PRIVATE int sqlite3SubInt64(i64*,i64); 11269 11548 SQLITE_PRIVATE int sqlite3MulInt64(i64*,i64); 11270 11549 SQLITE_PRIVATE int sqlite3AbsInt32(int); 11550 +#ifdef SQLITE_ENABLE_8_3_NAMES 11551 +SQLITE_PRIVATE void sqlite3FileSuffix3(const char*, char*); 11552 +#else 11553 +# define sqlite3FileSuffix3(X,Y) 11554 +#endif 11555 +SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z); 11271 11556 11272 11557 SQLITE_PRIVATE const void *sqlite3ValueText(sqlite3_value*, u8); 11273 11558 SQLITE_PRIVATE int sqlite3ValueBytes(sqlite3_value*, u8); 11274 11559 SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8, 11275 11560 void(*)(void*)); 11276 11561 SQLITE_PRIVATE void sqlite3ValueFree(sqlite3_value*); 11277 11562 SQLITE_PRIVATE sqlite3_value *sqlite3ValueNew(sqlite3 *); ................................................................................ 11373 11658 # define sqlite3VtabSync(X,Y) SQLITE_OK 11374 11659 # define sqlite3VtabRollback(X) 11375 11660 # define sqlite3VtabCommit(X) 11376 11661 # define sqlite3VtabInSync(db) 0 11377 11662 # define sqlite3VtabLock(X) 11378 11663 # define sqlite3VtabUnlock(X) 11379 11664 # define sqlite3VtabUnlockList(X) 11665 +# define sqlite3VtabSavepoint(X, Y, Z) SQLITE_OK 11380 11666 #else 11381 11667 SQLITE_PRIVATE void sqlite3VtabClear(sqlite3 *db, Table*); 11382 11668 SQLITE_PRIVATE int sqlite3VtabSync(sqlite3 *db, char **); 11383 11669 SQLITE_PRIVATE int sqlite3VtabRollback(sqlite3 *db); 11384 11670 SQLITE_PRIVATE int sqlite3VtabCommit(sqlite3 *db); 11385 11671 SQLITE_PRIVATE void sqlite3VtabLock(VTable *); 11386 11672 SQLITE_PRIVATE void sqlite3VtabUnlock(VTable *); 11387 11673 SQLITE_PRIVATE void sqlite3VtabUnlockList(sqlite3*); 11674 +SQLITE_PRIVATE int sqlite3VtabSavepoint(sqlite3 *, int, int); 11388 11675 # define sqlite3VtabInSync(db) ((db)->nVTrans>0 && (db)->aVTrans==0) 11389 11676 #endif 11390 11677 SQLITE_PRIVATE void sqlite3VtabMakeWritable(Parse*,Table*); 11391 11678 SQLITE_PRIVATE void sqlite3VtabBeginParse(Parse*, Token*, Token*, Token*); 11392 11679 SQLITE_PRIVATE void sqlite3VtabFinishParse(Parse*, Token*); 11393 11680 SQLITE_PRIVATE void sqlite3VtabArgInit(Parse*); 11394 11681 SQLITE_PRIVATE void sqlite3VtabArgExtend(Parse*, Token*); ................................................................................ 11687 11974 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* e0..e7 ........ */ 11688 11975 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* e8..ef ........ */ 11689 11976 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* f0..f7 ........ */ 11690 11977 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40 /* f8..ff ........ */ 11691 11978 }; 11692 11979 #endif 11693 11980 11694 - 11981 +#ifndef SQLITE_USE_URI 11982 +# define SQLITE_USE_URI 0 11983 +#endif 11695 11984 11696 11985 /* 11697 11986 ** The following singleton contains the global configuration for 11698 11987 ** the SQLite library. 11699 11988 */ 11700 11989 SQLITE_PRIVATE SQLITE_WSD struct Sqlite3Config sqlite3Config = { 11701 11990 SQLITE_DEFAULT_MEMSTATUS, /* bMemstat */ 11702 11991 1, /* bCoreMutex */ 11703 11992 SQLITE_THREADSAFE==1, /* bFullMutex */ 11993 + SQLITE_USE_URI, /* bOpenUri */ 11704 11994 0x7ffffffe, /* mxStrlen */ 11705 11995 100, /* szLookaside */ 11706 11996 500, /* nLookaside */ 11707 11997 {0,0,0,0,0,0,0,0}, /* m */ 11708 11998 {0,0,0,0,0,0,0,0,0}, /* mutex */ 11709 11999 {0,0,0,0,0,0,0,0,0,0,0}, /* pcache */ 11710 12000 (void*)0, /* pHeap */ ................................................................................ 11724 12014 0, /* isMutexInit */ 11725 12015 0, /* isMallocInit */ 11726 12016 0, /* isPCacheInit */ 11727 12017 0, /* pInitMutex */ 11728 12018 0, /* nRefInitMutex */ 11729 12019 0, /* xLog */ 11730 12020 0, /* pLogArg */ 12021 + 0, /* bLocaltimeFault */ 11731 12022 }; 11732 12023 11733 12024 11734 12025 /* 11735 12026 ** Hash table for global functions - functions common to all 11736 12027 ** database connections. After initialization, this table is 11737 12028 ** read-only. ................................................................................ 12476 12767 u32 magic; /* Magic number for sanity checking */ 12477 12768 char *zErrMsg; /* Error message written here */ 12478 12769 Vdbe *pPrev,*pNext; /* Linked list of VDBEs with the same Vdbe.db */ 12479 12770 VdbeCursor **apCsr; /* One element of this array for each open cursor */ 12480 12771 Mem *aVar; /* Values for the OP_Variable opcode. */ 12481 12772 char **azVar; /* Name of variables */ 12482 12773 ynVar nVar; /* Number of entries in aVar[] */ 12774 + ynVar nzVar; /* Number of entries in azVar[] */ 12483 12775 u32 cacheCtr; /* VdbeCursor row cache generation counter */ 12484 12776 int pc; /* The program counter */ 12485 12777 int rc; /* Value to return */ 12486 12778 u8 errorAction; /* Recovery action to do in case of an error */ 12487 - u8 okVar; /* True if azVar[] has been initialized */ 12488 12779 u8 explain; /* True if EXPLAIN present on SQL command */ 12489 12780 u8 changeCntOn; /* True to update the change-counter */ 12490 12781 u8 expired; /* True if the VM needs to be recompiled */ 12491 12782 u8 runOnlyOnce; /* Automatically expire on reset */ 12492 12783 u8 minWriteFileFormat; /* Minimum file format for writable database files */ 12493 12784 u8 inVtabMethod; /* See comments above */ 12494 12785 u8 usesStmtJournal; /* True if uses a statement journal */ ................................................................................ 12874 13165 ** Willmann-Bell, Inc 12875 13166 ** Richmond, Virginia (USA) 12876 13167 */ 12877 13168 #include <time.h> 12878 13169 12879 13170 #ifndef SQLITE_OMIT_DATETIME_FUNCS 12880 13171 12881 -/* 12882 -** On recent Windows platforms, the localtime_s() function is available 12883 -** as part of the "Secure CRT". It is essentially equivalent to 12884 -** localtime_r() available under most POSIX platforms, except that the 12885 -** order of the parameters is reversed. 12886 -** 12887 -** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx. 12888 -** 12889 -** If the user has not indicated to use localtime_r() or localtime_s() 12890 -** already, check for an MSVC build environment that provides 12891 -** localtime_s(). 12892 -*/ 12893 -#if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \ 12894 - defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE) 12895 -#define HAVE_LOCALTIME_S 1 12896 -#endif 12897 13172 12898 13173 /* 12899 13174 ** A structure for holding a single date and time. 12900 13175 */ 12901 13176 typedef struct DateTime DateTime; 12902 13177 struct DateTime { 12903 13178 sqlite3_int64 iJD; /* The julian day number times 86400000 */ ................................................................................ 13234 13509 ** Clear the YMD and HMS and the TZ 13235 13510 */ 13236 13511 static void clearYMD_HMS_TZ(DateTime *p){ 13237 13512 p->validYMD = 0; 13238 13513 p->validHMS = 0; 13239 13514 p->validTZ = 0; 13240 13515 } 13516 + 13517 +/* 13518 +** On recent Windows platforms, the localtime_s() function is available 13519 +** as part of the "Secure CRT". It is essentially equivalent to 13520 +** localtime_r() available under most POSIX platforms, except that the 13521 +** order of the parameters is reversed. 13522 +** 13523 +** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx. 13524 +** 13525 +** If the user has not indicated to use localtime_r() or localtime_s() 13526 +** already, check for an MSVC build environment that provides 13527 +** localtime_s(). 13528 +*/ 13529 +#if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \ 13530 + defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE) 13531 +#define HAVE_LOCALTIME_S 1 13532 +#endif 13533 + 13534 +#ifndef SQLITE_OMIT_LOCALTIME 13535 +/* 13536 +** The following routine implements the rough equivalent of localtime_r() 13537 +** using whatever operating-system specific localtime facility that 13538 +** is available. This routine returns 0 on success and 13539 +** non-zero on any kind of error. 13540 +** 13541 +** If the sqlite3GlobalConfig.bLocaltimeFault variable is true then this 13542 +** routine will always fail. 13543 +*/ 13544 +static int osLocaltime(time_t *t, struct tm *pTm){ 13545 + int rc; 13546 +#if (!defined(HAVE_LOCALTIME_R) || !HAVE_LOCALTIME_R) \ 13547 + && (!defined(HAVE_LOCALTIME_S) || !HAVE_LOCALTIME_S) 13548 + struct tm *pX; 13549 + sqlite3_mutex *mutex = sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER); 13550 + sqlite3_mutex_enter(mutex); 13551 + pX = localtime(t); 13552 +#ifndef SQLITE_OMIT_BUILTIN_TEST 13553 + if( sqlite3GlobalConfig.bLocaltimeFault ) pX = 0; 13554 +#endif 13555 + if( pX ) *pTm = *pX; 13556 + sqlite3_mutex_leave(mutex); 13557 + rc = pX==0; 13558 +#else 13559 +#ifndef SQLITE_OMIT_BUILTIN_TEST 13560 + if( sqlite3GlobalConfig.bLocaltimeFault ) return 1; 13561 +#endif 13562 +#if defined(HAVE_LOCALTIME_R) && HAVE_LOCALTIME_R 13563 + rc = localtime_r(t, pTm)==0; 13564 +#else 13565 + rc = localtime_s(pTm, t); 13566 +#endif /* HAVE_LOCALTIME_R */ 13567 +#endif /* HAVE_LOCALTIME_R || HAVE_LOCALTIME_S */ 13568 + return rc; 13569 +} 13570 +#endif /* SQLITE_OMIT_LOCALTIME */ 13571 + 13241 13572 13242 13573 #ifndef SQLITE_OMIT_LOCALTIME 13243 13574 /* 13244 -** Compute the difference (in milliseconds) 13245 -** between localtime and UTC (a.k.a. GMT) 13246 -** for the time value p where p is in UTC. 13575 +** Compute the difference (in milliseconds) between localtime and UTC 13576 +** (a.k.a. GMT) for the time value p where p is in UTC. If no error occurs, 13577 +** return this value and set *pRc to SQLITE_OK. 13578 +** 13579 +** Or, if an error does occur, set *pRc to SQLITE_ERROR. The returned value 13580 +** is undefined in this case. 13247 13581 */ 13248 -static sqlite3_int64 localtimeOffset(DateTime *p){ 13582 +static sqlite3_int64 localtimeOffset( 13583 + DateTime *p, /* Date at which to calculate offset */ 13584 + sqlite3_context *pCtx, /* Write error here if one occurs */ 13585 + int *pRc /* OUT: Error code. SQLITE_OK or ERROR */ 13586 +){ 13249 13587 DateTime x, y; 13250 13588 time_t t; 13589 + struct tm sLocal; 13590 + 13591 + /* Initialize the contents of sLocal to avoid a compiler warning. */ 13592 + memset(&sLocal, 0, sizeof(sLocal)); 13593 + 13251 13594 x = *p; 13252 13595 computeYMD_HMS(&x); 13253 13596 if( x.Y<1971 || x.Y>=2038 ){ 13254 13597 x.Y = 2000; 13255 13598 x.M = 1; 13256 13599 x.D = 1; 13257 13600 x.h = 0; ................................................................................ 13261 13604 int s = (int)(x.s + 0.5); 13262 13605 x.s = s; 13263 13606 } 13264 13607 x.tz = 0; 13265 13608 x.validJD = 0; 13266 13609 computeJD(&x); 13267 13610 t = (time_t)(x.iJD/1000 - 21086676*(i64)10000); 13268 -#ifdef HAVE_LOCALTIME_R 13269 - { 13270 - struct tm sLocal; 13271 - localtime_r(&t, &sLocal); 13272 - y.Y = sLocal.tm_year + 1900; 13273 - y.M = sLocal.tm_mon + 1; 13274 - y.D = sLocal.tm_mday; 13275 - y.h = sLocal.tm_hour; 13276 - y.m = sLocal.tm_min; 13277 - y.s = sLocal.tm_sec; 13611 + if( osLocaltime(&t, &sLocal) ){ 13612 + sqlite3_result_error(pCtx, "local time unavailable", -1); 13613 + *pRc = SQLITE_ERROR; 13614 + return 0; 13278 13615 } 13279 -#elif defined(HAVE_LOCALTIME_S) && HAVE_LOCALTIME_S 13280 - { 13281 - struct tm sLocal; 13282 - localtime_s(&sLocal, &t); 13283 - y.Y = sLocal.tm_year + 1900; 13284 - y.M = sLocal.tm_mon + 1; 13285 - y.D = sLocal.tm_mday; 13286 - y.h = sLocal.tm_hour; 13287 - y.m = sLocal.tm_min; 13288 - y.s = sLocal.tm_sec; 13289 - } 13290 -#else 13291 - { 13292 - struct tm *pTm; 13293 - sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER)); 13294 - pTm = localtime(&t); 13295 - y.Y = pTm->tm_year + 1900; 13296 - y.M = pTm->tm_mon + 1; 13297 - y.D = pTm->tm_mday; 13298 - y.h = pTm->tm_hour; 13299 - y.m = pTm->tm_min; 13300 - y.s = pTm->tm_sec; 13301 - sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER)); 13302 - } 13303 -#endif 13616 + y.Y = sLocal.tm_year + 1900; 13617 + y.M = sLocal.tm_mon + 1; 13618 + y.D = sLocal.tm_mday; 13619 + y.h = sLocal.tm_hour; 13620 + y.m = sLocal.tm_min; 13621 + y.s = sLocal.tm_sec; 13304 13622 y.validYMD = 1; 13305 13623 y.validHMS = 1; 13306 13624 y.validJD = 0; 13307 13625 y.validTZ = 0; 13308 13626 computeJD(&y); 13627 + *pRc = SQLITE_OK; 13309 13628 return y.iJD - x.iJD; 13310 13629 } 13311 13630 #endif /* SQLITE_OMIT_LOCALTIME */ 13312 13631 13313 13632 /* 13314 13633 ** Process a modifier to a date-time stamp. The modifiers are 13315 13634 ** as follows: ................................................................................ 13325 13644 ** start of week 13326 13645 ** start of day 13327 13646 ** weekday N 13328 13647 ** unixepoch 13329 13648 ** localtime 13330 13649 ** utc 13331 13650 ** 13332 -** Return 0 on success and 1 if there is any kind of error. 13651 +** Return 0 on success and 1 if there is any kind of error. If the error 13652 +** is in a system call (i.e. localtime()), then an error message is written 13653 +** to context pCtx. If the error is an unrecognized modifier, no error is 13654 +** written to pCtx. 13333 13655 */ 13334 -static int parseModifier(const char *zMod, DateTime *p){ 13656 +static int parseModifier(sqlite3_context *pCtx, const char *zMod, DateTime *p){ 13335 13657 int rc = 1; 13336 13658 int n; 13337 13659 double r; 13338 13660 char *z, zBuf[30]; 13339 13661 z = zBuf; 13340 13662 for(n=0; n<ArraySize(zBuf)-1 && zMod[n]; n++){ 13341 13663 z[n] = (char)sqlite3UpperToLower[(u8)zMod[n]]; ................................................................................ 13347 13669 /* localtime 13348 13670 ** 13349 13671 ** Assuming the current time value is UTC (a.k.a. GMT), shift it to 13350 13672 ** show local time. 13351 13673 */ 13352 13674 if( strcmp(z, "localtime")==0 ){ 13353 13675 computeJD(p); 13354 - p->iJD += localtimeOffset(p); 13676 + p->iJD += localtimeOffset(p, pCtx, &rc); 13355 13677 clearYMD_HMS_TZ(p); 13356 - rc = 0; 13357 13678 } 13358 13679 break; 13359 13680 } 13360 13681 #endif 13361 13682 case 'u': { 13362 13683 /* 13363 13684 ** unixepoch ................................................................................ 13370 13691 clearYMD_HMS_TZ(p); 13371 13692 rc = 0; 13372 13693 } 13373 13694 #ifndef SQLITE_OMIT_LOCALTIME 13374 13695 else if( strcmp(z, "utc")==0 ){ 13375 13696 sqlite3_int64 c1; 13376 13697 computeJD(p); 13377 - c1 = localtimeOffset(p); 13378 - p->iJD -= c1; 13379 - clearYMD_HMS_TZ(p); 13380 - p->iJD += c1 - localtimeOffset(p); 13381 - rc = 0; 13698 + c1 = localtimeOffset(p, pCtx, &rc); 13699 + if( rc==SQLITE_OK ){ 13700 + p->iJD -= c1; 13701 + clearYMD_HMS_TZ(p); 13702 + p->iJD += c1 - localtimeOffset(p, pCtx, &rc); 13703 + } 13382 13704 } 13383 13705 #endif 13384 13706 break; 13385 13707 } 13386 13708 case 'w': { 13387 13709 /* 13388 13710 ** weekday N ................................................................................ 13555 13877 }else{ 13556 13878 z = sqlite3_value_text(argv[0]); 13557 13879 if( !z || parseDateOrTime(context, (char*)z, p) ){ 13558 13880 return 1; 13559 13881 } 13560 13882 } 13561 13883 for(i=1; i<argc; i++){ 13562 - if( (z = sqlite3_value_text(argv[i]))==0 || parseModifier((char*)z, p) ){ 13563 - return 1; 13564 - } 13884 + z = sqlite3_value_text(argv[i]); 13885 + if( z==0 || parseModifier(context, (char*)z, p) ) return 1; 13565 13886 } 13566 13887 return 0; 13567 13888 } 13568 13889 13569 13890 13570 13891 /* 13571 13892 ** The following routines implement the various date and time functions ................................................................................ 17946 18267 int nFull; 17947 18268 void *p; 17948 18269 assert( sqlite3_mutex_held(mem0.mutex) ); 17949 18270 nFull = sqlite3GlobalConfig.m.xRoundup(n); 17950 18271 sqlite3StatusSet(SQLITE_STATUS_MALLOC_SIZE, n); 17951 18272 if( mem0.alarmCallback!=0 ){ 17952 18273 int nUsed = sqlite3StatusValue(SQLITE_STATUS_MEMORY_USED); 17953 - if( nUsed+nFull >= mem0.alarmThreshold ){ 18274 + if( nUsed >= mem0.alarmThreshold - nFull ){ 17954 18275 mem0.nearlyFull = 1; 17955 18276 sqlite3MallocAlarm(nFull); 17956 18277 }else{ 17957 18278 mem0.nearlyFull = 0; 17958 18279 } 17959 18280 } 17960 18281 p = sqlite3GlobalConfig.m.xMalloc(nFull); ................................................................................ 18187 18508 sqlite3_free(p); 18188 18509 } 18189 18510 18190 18511 /* 18191 18512 ** Change the size of an existing memory allocation 18192 18513 */ 18193 18514 SQLITE_PRIVATE void *sqlite3Realloc(void *pOld, int nBytes){ 18194 - int nOld, nNew; 18515 + int nOld, nNew, nDiff; 18195 18516 void *pNew; 18196 18517 if( pOld==0 ){ 18197 18518 return sqlite3Malloc(nBytes); /* IMP: R-28354-25769 */ 18198 18519 } 18199 18520 if( nBytes<=0 ){ 18200 18521 sqlite3_free(pOld); /* IMP: R-31593-10574 */ 18201 18522 return 0; ................................................................................ 18210 18531 ** xRoundup. */ 18211 18532 nNew = sqlite3GlobalConfig.m.xRoundup(nBytes); 18212 18533 if( nOld==nNew ){ 18213 18534 pNew = pOld; 18214 18535 }else if( sqlite3GlobalConfig.bMemstat ){ 18215 18536 sqlite3_mutex_enter(mem0.mutex); 18216 18537 sqlite3StatusSet(SQLITE_STATUS_MALLOC_SIZE, nBytes); 18217 - if( sqlite3StatusValue(SQLITE_STATUS_MEMORY_USED)+nNew-nOld >= 18218 - mem0.alarmThreshold ){ 18219 - sqlite3MallocAlarm(nNew-nOld); 18538 + nDiff = nNew - nOld; 18539 + if( sqlite3StatusValue(SQLITE_STATUS_MEMORY_USED) >= 18540 + mem0.alarmThreshold-nDiff ){ 18541 + sqlite3MallocAlarm(nDiff); 18220 18542 } 18221 18543 assert( sqlite3MemdebugHasType(pOld, MEMTYPE_HEAP) ); 18222 18544 assert( sqlite3MemdebugNoType(pOld, ~MEMTYPE_HEAP) ); 18223 18545 pNew = sqlite3GlobalConfig.m.xRealloc(pOld, nNew); 18224 18546 if( pNew==0 && mem0.alarmCallback ){ 18225 18547 sqlite3MallocAlarm(nBytes); 18226 18548 pNew = sqlite3GlobalConfig.m.xRealloc(pOld, nNew); ................................................................................ 19797 20119 while( zIn!=zTerm && (*zIn & 0xc0)==0x80 ){ \ 19798 20120 c = (c<<6) + (0x3f & *(zIn++)); \ 19799 20121 } \ 19800 20122 if( c<0x80 \ 19801 20123 || (c&0xFFFFF800)==0xD800 \ 19802 20124 || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \ 19803 20125 } 19804 -SQLITE_PRIVATE int sqlite3Utf8Read( 20126 +SQLITE_PRIVATE u32 sqlite3Utf8Read( 19805 20127 const unsigned char *zIn, /* First byte of UTF-8 character */ 19806 20128 const unsigned char **pzNext /* Write first byte past UTF-8 char here */ 19807 20129 ){ 19808 20130 unsigned int c; 19809 20131 19810 20132 /* Same as READ_UTF8() above but without the zTerm parameter. 19811 20133 ** For this routine, we assume the UTF8 string is always zero-terminated. ................................................................................ 21178 21500 p[1] = (u8)(v>>16); 21179 21501 p[2] = (u8)(v>>8); 21180 21502 p[3] = (u8)v; 21181 21503 } 21182 21504 21183 21505 21184 21506 21185 -#if !defined(SQLITE_OMIT_BLOB_LITERAL) || defined(SQLITE_HAS_CODEC) 21186 21507 /* 21187 21508 ** Translate a single byte of Hex into an integer. 21188 21509 ** This routine only works if h really is a valid hexadecimal 21189 21510 ** character: 0..9a..fA..F 21190 21511 */ 21191 -static u8 hexToInt(int h){ 21512 +SQLITE_PRIVATE u8 sqlite3HexToInt(int h){ 21192 21513 assert( (h>='0' && h<='9') || (h>='a' && h<='f') || (h>='A' && h<='F') ); 21193 21514 #ifdef SQLITE_ASCII 21194 21515 h += 9*(1&(h>>6)); 21195 21516 #endif 21196 21517 #ifdef SQLITE_EBCDIC 21197 21518 h += 9*(1&~(h>>4)); 21198 21519 #endif 21199 21520 return (u8)(h & 0xf); 21200 21521 } 21201 -#endif /* !SQLITE_OMIT_BLOB_LITERAL || SQLITE_HAS_CODEC */ 21202 21522 21203 21523 #if !defined(SQLITE_OMIT_BLOB_LITERAL) || defined(SQLITE_HAS_CODEC) 21204 21524 /* 21205 21525 ** Convert a BLOB literal of the form "x'hhhhhh'" into its binary 21206 21526 ** value. Return a pointer to its binary value. Space to hold the 21207 21527 ** binary value has been obtained from malloc and must be freed by 21208 21528 ** the calling routine. ................................................................................ 21211 21531 char *zBlob; 21212 21532 int i; 21213 21533 21214 21534 zBlob = (char *)sqlite3DbMallocRaw(db, n/2 + 1); 21215 21535 n--; 21216 21536 if( zBlob ){ 21217 21537 for(i=0; i<n; i+=2){ 21218 - zBlob[i/2] = (hexToInt(z[i])<<4) | hexToInt(z[i+1]); 21538 + zBlob[i/2] = (sqlite3HexToInt(z[i])<<4) | sqlite3HexToInt(z[i+1]); 21219 21539 } 21220 21540 zBlob[i/2] = 0; 21221 21541 } 21222 21542 return zBlob; 21223 21543 } 21224 21544 #endif /* !SQLITE_OMIT_BLOB_LITERAL || SQLITE_HAS_CODEC */ 21225 21545 ................................................................................ 21343 21663 ** if the integer has a value of -2147483648, return +2147483647 21344 21664 */ 21345 21665 SQLITE_PRIVATE int sqlite3AbsInt32(int x){ 21346 21666 if( x>=0 ) return x; 21347 21667 if( x==(int)0x80000000 ) return 0x7fffffff; 21348 21668 return -x; 21349 21669 } 21670 + 21671 +#ifdef SQLITE_ENABLE_8_3_NAMES 21672 +/* 21673 +** If SQLITE_ENABLE_8_3_NAME is set at compile-time and if the database 21674 +** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and 21675 +** if filename in z[] has a suffix (a.k.a. "extension") that is longer than 21676 +** three characters, then shorten the suffix on z[] to be the last three 21677 +** characters of the original suffix. 21678 +** 21679 +** Examples: 21680 +** 21681 +** test.db-journal => test.nal 21682 +** test.db-wal => test.wal 21683 +** test.db-shm => test.shm 21684 +*/ 21685 +SQLITE_PRIVATE void sqlite3FileSuffix3(const char *zBaseFilename, char *z){ 21686 + const char *zOk; 21687 + zOk = sqlite3_uri_parameter(zBaseFilename, "8_3_names"); 21688 + if( zOk && sqlite3GetBoolean(zOk) ){ 21689 + int i, sz; 21690 + sz = sqlite3Strlen30(z); 21691 + for(i=sz-1; i>0 && z[i]!='/' && z[i]!='.'; i--){} 21692 + if( z[i]=='.' && ALWAYS(sz>i+4) ) memcpy(&z[i+1], &z[sz-3], 4); 21693 + } 21694 +} 21695 +#endif 21350 21696 21351 21697 /************** End of util.c ************************************************/ 21352 21698 /************** Begin file hash.c ********************************************/ 21353 21699 /* 21354 21700 ** 2001 September 22 21355 21701 ** 21356 21702 ** The author disclaims copyright to this source code. In place of ................................................................................ 24050 24396 # endif 24051 24397 #endif /* SQLITE_ENABLE_LOCKING_STYLE */ 24052 24398 24053 24399 #if defined(__APPLE__) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS) 24054 24400 # include <sys/mount.h> 24055 24401 #endif 24056 24402 24403 +#ifdef HAVE_UTIME 24404 +# include <utime.h> 24405 +#endif 24406 + 24057 24407 /* 24058 24408 ** Allowed values of unixFile.fsFlags 24059 24409 */ 24060 24410 #define SQLITE_FSFLAGS_IS_MSDOS 0x1 24061 24411 24062 24412 /* 24063 24413 ** If we are to be thread-safe, include the pthreads header and define ................................................................................ 24396 24746 ** testing and debugging only. 24397 24747 */ 24398 24748 #if SQLITE_THREADSAFE 24399 24749 #define threadid pthread_self() 24400 24750 #else 24401 24751 #define threadid 0 24402 24752 #endif 24753 + 24754 +/* 24755 +** Different Unix systems declare open() in different ways. Same use 24756 +** open(const char*,int,mode_t). Others use open(const char*,int,...). 24757 +** The difference is important when using a pointer to the function. 24758 +** 24759 +** The safest way to deal with the problem is to always use this wrapper 24760 +** which always has the same well-defined interface. 24761 +*/ 24762 +static int posixOpen(const char *zFile, int flags, int mode){ 24763 + return open(zFile, flags, mode); 24764 +} 24403 24765 24404 24766 /* 24405 24767 ** Many system calls are accessed through pointer-to-functions so that 24406 24768 ** they may be overridden at runtime to facilitate fault injection during 24407 24769 ** testing and sandboxing. The following array holds the names and pointers 24408 24770 ** to all overrideable system calls. 24409 24771 */ 24410 24772 static struct unix_syscall { 24411 24773 const char *zName; /* Name of the sytem call */ 24412 24774 sqlite3_syscall_ptr pCurrent; /* Current value of the system call */ 24413 24775 sqlite3_syscall_ptr pDefault; /* Default value */ 24414 24776 } aSyscall[] = { 24415 - { "open", (sqlite3_syscall_ptr)open, 0 }, 24416 -#define osOpen ((int(*)(const char*,int,...))aSyscall[0].pCurrent) 24777 + { "open", (sqlite3_syscall_ptr)posixOpen, 0 }, 24778 +#define osOpen ((int(*)(const char*,int,int))aSyscall[0].pCurrent) 24417 24779 24418 24780 { "close", (sqlite3_syscall_ptr)close, 0 }, 24419 24781 #define osClose ((int(*)(int))aSyscall[1].pCurrent) 24420 24782 24421 24783 { "access", (sqlite3_syscall_ptr)access, 0 }, 24422 24784 #define osAccess ((int(*)(const char*,int))aSyscall[2].pCurrent) 24423 24785 ................................................................................ 24446 24808 24447 24809 { "fcntl", (sqlite3_syscall_ptr)fcntl, 0 }, 24448 24810 #define osFcntl ((int(*)(int,int,...))aSyscall[7].pCurrent) 24449 24811 24450 24812 { "read", (sqlite3_syscall_ptr)read, 0 }, 24451 24813 #define osRead ((ssize_t(*)(int,void*,size_t))aSyscall[8].pCurrent) 24452 24814 24453 -#if defined(USE_PREAD) || defined(SQLITE_ENABLE_LOCKING_STYLE) 24815 +#if defined(USE_PREAD) || SQLITE_ENABLE_LOCKING_STYLE 24454 24816 { "pread", (sqlite3_syscall_ptr)pread, 0 }, 24455 24817 #else 24456 24818 { "pread", (sqlite3_syscall_ptr)0, 0 }, 24457 24819 #endif 24458 24820 #define osPread ((ssize_t(*)(int,void*,size_t,off_t))aSyscall[9].pCurrent) 24459 24821 24460 24822 #if defined(USE_PREAD64) ................................................................................ 24463 24825 { "pread64", (sqlite3_syscall_ptr)0, 0 }, 24464 24826 #endif 24465 24827 #define osPread64 ((ssize_t(*)(int,void*,size_t,off_t))aSyscall[10].pCurrent) 24466 24828 24467 24829 { "write", (sqlite3_syscall_ptr)write, 0 }, 24468 24830 #define osWrite ((ssize_t(*)(int,const void*,size_t))aSyscall[11].pCurrent) 24469 24831 24470 -#if defined(USE_PREAD) || defined(SQLITE_ENABLE_LOCKING_STYLE) 24832 +#if defined(USE_PREAD) || SQLITE_ENABLE_LOCKING_STYLE 24471 24833 { "pwrite", (sqlite3_syscall_ptr)pwrite, 0 }, 24472 24834 #else 24473 24835 { "pwrite", (sqlite3_syscall_ptr)0, 0 }, 24474 24836 #endif 24475 24837 #define osPwrite ((ssize_t(*)(int,const void*,size_t,off_t))\ 24476 24838 aSyscall[12].pCurrent) 24477 24839 ................................................................................ 25049 25411 unsigned char bProcessLock; /* An exclusive process lock is held */ 25050 25412 int nRef; /* Number of pointers to this structure */ 25051 25413 unixShmNode *pShmNode; /* Shared memory associated with this inode */ 25052 25414 int nLock; /* Number of outstanding file locks */ 25053 25415 UnixUnusedFd *pUnused; /* Unused file descriptors to close */ 25054 25416 unixInodeInfo *pNext; /* List of all unixInodeInfo objects */ 25055 25417 unixInodeInfo *pPrev; /* .... doubly linked */ 25056 -#if defined(SQLITE_ENABLE_LOCKING_STYLE) 25418 +#if SQLITE_ENABLE_LOCKING_STYLE 25057 25419 unsigned long long sharedByte; /* for AFP simulated shared lock */ 25058 25420 #endif 25059 25421 #if OS_VXWORKS 25060 25422 sem_t *pSem; /* Named POSIX semaphore */ 25061 25423 char aSemName[MAX_PATHNAME+2]; /* Name of that semaphore */ 25062 25424 #endif 25063 25425 }; ................................................................................ 26043 26405 26044 26406 26045 26407 /* If we have any lock, then the lock file already exists. All we have 26046 26408 ** to do is adjust our internal record of the lock level. 26047 26409 */ 26048 26410 if( pFile->eFileLock > NO_LOCK ){ 26049 26411 pFile->eFileLock = eFileLock; 26050 -#if !OS_VXWORKS 26051 26412 /* Always update the timestamp on the old file */ 26413 +#ifdef HAVE_UTIME 26414 + utime(zLockFile, NULL); 26415 +#else 26052 26416 utimes(zLockFile, NULL); 26053 26417 #endif 26054 26418 return SQLITE_OK; 26055 26419 } 26056 26420 26057 26421 /* grab an exclusive lock */ 26058 26422 fd = robust_open(zLockFile,O_RDONLY|O_CREAT|O_EXCL,0600); ................................................................................ 27206 27570 offset += wrote; 27207 27571 pBuf = &((char*)pBuf)[wrote]; 27208 27572 } 27209 27573 SimulateIOError(( wrote=(-1), amt=1 )); 27210 27574 SimulateDiskfullError(( wrote=0, amt=1 )); 27211 27575 27212 27576 if( amt>0 ){ 27213 - if( wrote<0 ){ 27577 + if( wrote<0 && pFile->lastErrno!=ENOSPC ){ 27214 27578 /* lastErrno set by seekAndWrite */ 27215 27579 return SQLITE_IOERR_WRITE; 27216 27580 }else{ 27217 27581 pFile->lastErrno = 0; /* not a system error */ 27218 27582 return SQLITE_FULL; 27219 27583 } 27220 27584 } ................................................................................ 27641 28005 */ 27642 28006 struct unixShmNode { 27643 28007 unixInodeInfo *pInode; /* unixInodeInfo that owns this SHM node */ 27644 28008 sqlite3_mutex *mutex; /* Mutex to access this object */ 27645 28009 char *zFilename; /* Name of the mmapped file */ 27646 28010 int h; /* Open file descriptor */ 27647 28011 int szRegion; /* Size of shared-memory regions */ 27648 - int nRegion; /* Size of array apRegion */ 28012 + u16 nRegion; /* Size of array apRegion */ 28013 + u8 isReadonly; /* True if read-only */ 27649 28014 char **apRegion; /* Array of mapped shared-memory regions */ 27650 28015 int nRef; /* Number of unixShm objects pointing to this */ 27651 28016 unixShm *pFirst; /* All unixShm objects pointing to this */ 27652 28017 #ifdef SQLITE_DEBUG 27653 28018 u8 exclMask; /* Mask of exclusive locks held */ 27654 28019 u8 sharedMask; /* Mask of shared locks held */ 27655 28020 u8 nextShmId; /* Next available unixShm.id value */ ................................................................................ 27873 28238 zShmFilename = pShmNode->zFilename = (char*)&pShmNode[1]; 27874 28239 #ifdef SQLITE_SHM_DIRECTORY 27875 28240 sqlite3_snprintf(nShmFilename, zShmFilename, 27876 28241 SQLITE_SHM_DIRECTORY "/sqlite-shm-%x-%x", 27877 28242 (u32)sStat.st_ino, (u32)sStat.st_dev); 27878 28243 #else 27879 28244 sqlite3_snprintf(nShmFilename, zShmFilename, "%s-shm", pDbFd->zPath); 28245 + sqlite3FileSuffix3(pDbFd->zPath, zShmFilename); 27880 28246 #endif 27881 28247 pShmNode->h = -1; 27882 28248 pDbFd->pInode->pShmNode = pShmNode; 27883 28249 pShmNode->pInode = pDbFd->pInode; 27884 28250 pShmNode->mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST); 27885 28251 if( pShmNode->mutex==0 ){ 27886 28252 rc = SQLITE_NOMEM; ................................................................................ 27887 28253 goto shm_open_err; 27888 28254 } 27889 28255 27890 28256 if( pInode->bProcessLock==0 ){ 27891 28257 pShmNode->h = robust_open(zShmFilename, O_RDWR|O_CREAT, 27892 28258 (sStat.st_mode & 0777)); 27893 28259 if( pShmNode->h<0 ){ 27894 - rc = unixLogError(SQLITE_CANTOPEN_BKPT, "open", zShmFilename); 27895 - goto shm_open_err; 28260 + const char *zRO; 28261 + zRO = sqlite3_uri_parameter(pDbFd->zPath, "readonly_shm"); 28262 + if( zRO && sqlite3GetBoolean(zRO) ){ 28263 + pShmNode->h = robust_open(zShmFilename, O_RDONLY, 28264 + (sStat.st_mode & 0777)); 28265 + pShmNode->isReadonly = 1; 28266 + } 28267 + if( pShmNode->h<0 ){ 28268 + rc = unixLogError(SQLITE_CANTOPEN_BKPT, "open", zShmFilename); 28269 + goto shm_open_err; 28270 + } 27896 28271 } 27897 28272 27898 28273 /* Check to see if another process is holding the dead-man switch. 27899 28274 ** If not, truncate the file to zero length. 27900 28275 */ 27901 28276 rc = SQLITE_OK; 27902 28277 if( unixShmSystemLock(pShmNode, F_WRLCK, UNIX_SHM_DMS, 1)==SQLITE_OK ){ ................................................................................ 28027 28402 rc = SQLITE_IOERR_NOMEM; 28028 28403 goto shmpage_out; 28029 28404 } 28030 28405 pShmNode->apRegion = apNew; 28031 28406 while(pShmNode->nRegion<=iRegion){ 28032 28407 void *pMem; 28033 28408 if( pShmNode->h>=0 ){ 28034 - pMem = mmap(0, szRegion, PROT_READ|PROT_WRITE, 28409 + pMem = mmap(0, szRegion, 28410 + pShmNode->isReadonly ? PROT_READ : PROT_READ|PROT_WRITE, 28035 28411 MAP_SHARED, pShmNode->h, pShmNode->nRegion*szRegion 28036 28412 ); 28037 28413 if( pMem==MAP_FAILED ){ 28038 - rc = SQLITE_IOERR; 28414 + rc = unixLogError(SQLITE_IOERR_SHMMAP, "mmap", pShmNode->zFilename); 28039 28415 goto shmpage_out; 28040 28416 } 28041 28417 }else{ 28042 28418 pMem = sqlite3_malloc(szRegion); 28043 28419 if( pMem==0 ){ 28044 28420 rc = SQLITE_NOMEM; 28045 28421 goto shmpage_out; ................................................................................ 28053 28429 28054 28430 shmpage_out: 28055 28431 if( pShmNode->nRegion>iRegion ){ 28056 28432 *pp = pShmNode->apRegion[iRegion]; 28057 28433 }else{ 28058 28434 *pp = 0; 28059 28435 } 28436 + if( pShmNode->isReadonly && rc==SQLITE_OK ) rc = SQLITE_READONLY; 28060 28437 sqlite3_mutex_leave(pShmNode->mutex); 28061 28438 return rc; 28062 28439 } 28063 28440 28064 28441 /* 28065 28442 ** Change the lock state for a shared-memory segment. 28066 28443 ** ................................................................................ 28906 29283 ** mask SQLITE_DEFAULT_FILE_PERMISSIONS. 28907 29284 ** 28908 29285 ** Finally, if the file being opened is a WAL or regular journal file, then 28909 29286 ** this function queries the file-system for the permissions on the 28910 29287 ** corresponding database file and sets *pMode to this value. Whenever 28911 29288 ** possible, WAL and journal files are created using the same permissions 28912 29289 ** as the associated database file. 29290 +** 29291 +** If the SQLITE_ENABLE_8_3_NAMES option is enabled, then the 29292 +** original filename is unavailable. But 8_3_NAMES is only used for 29293 +** FAT filesystems and permissions do not matter there, so just use 29294 +** the default permissions. 28913 29295 */ 28914 29296 static int findCreateFileMode( 28915 29297 const char *zPath, /* Path of file (possibly) being created */ 28916 29298 int flags, /* Flags passed as 4th argument to xOpen() */ 28917 29299 mode_t *pMode /* OUT: Permissions to open file with */ 28918 29300 ){ 28919 29301 int rc = SQLITE_OK; /* Return Code */ 29302 + *pMode = SQLITE_DEFAULT_FILE_PERMISSIONS; 28920 29303 if( flags & (SQLITE_OPEN_WAL|SQLITE_OPEN_MAIN_JOURNAL) ){ 28921 29304 char zDb[MAX_PATHNAME+1]; /* Database file path */ 28922 29305 int nDb; /* Number of valid bytes in zDb */ 28923 29306 struct stat sStat; /* Output of stat() on database file */ 28924 29307 28925 29308 /* zPath is a path to a WAL or journal file. The following block derives 28926 29309 ** the path to the associated database file from zPath. This block handles 28927 29310 ** the following naming conventions: 28928 29311 ** 28929 29312 ** "<path to db>-journal" 28930 29313 ** "<path to db>-wal" 28931 - ** "<path to db>-journal-NNNN" 28932 - ** "<path to db>-wal-NNNN" 29314 + ** "<path to db>-journalNN" 29315 + ** "<path to db>-walNN" 28933 29316 ** 28934 - ** where NNNN is a 4 digit decimal number. The NNNN naming schemes are 29317 + ** where NN is a 4 digit decimal number. The NN naming schemes are 28935 29318 ** used by the test_multiplex.c module. 28936 29319 */ 28937 29320 nDb = sqlite3Strlen30(zPath) - 1; 28938 - while( nDb>0 && zPath[nDb]!='l' ) nDb--; 28939 - nDb -= ((flags & SQLITE_OPEN_WAL) ? 3 : 7); 29321 + while( nDb>0 && zPath[nDb]!='-' ) nDb--; 29322 + if( nDb==0 ) return SQLITE_OK; 28940 29323 memcpy(zDb, zPath, nDb); 28941 29324 zDb[nDb] = '\0'; 28942 29325 28943 29326 if( 0==stat(zDb, &sStat) ){ 28944 29327 *pMode = sStat.st_mode & 0777; 28945 29328 }else{ 28946 29329 rc = SQLITE_IOERR_FSTAT; 28947 29330 } 28948 29331 }else if( flags & SQLITE_OPEN_DELETEONCLOSE ){ 28949 29332 *pMode = 0600; 28950 - }else{ 28951 - *pMode = SQLITE_DEFAULT_FILE_PERMISSIONS; 28952 29333 } 28953 29334 return rc; 28954 29335 } 28955 29336 28956 29337 /* 28957 29338 ** Open the file zPath. 28958 29339 ** ................................................................................ 31151 31532 WCHAR *zDeleteOnClose; /* Name of file to delete when closing */ 31152 31533 HANDLE hMutex; /* Mutex used to control access to shared lock */ 31153 31534 HANDLE hShared; /* Shared memory segment used for locking */ 31154 31535 winceLock local; /* Locks obtained by this instance of winFile */ 31155 31536 winceLock *shared; /* Global shared lock memory for the file */ 31156 31537 #endif 31157 31538 }; 31539 + 31158 31540 31159 31541 /* 31160 31542 ** Forward prototypes. 31161 31543 */ 31162 31544 static int getSectorSize( 31163 31545 sqlite3_vfs *pVfs, 31164 31546 const char *zRelative /* UTF-8 file name */ ................................................................................ 31319 31701 return zFilenameUtf8; 31320 31702 } 31321 31703 31322 31704 /* 31323 31705 ** Convert UTF-8 to multibyte character string. Space to hold the 31324 31706 ** returned string is obtained from malloc(). 31325 31707 */ 31326 -static char *utf8ToMbcs(const char *zFilename){ 31708 +SQLITE_API char *sqlite3_win32_utf8_to_mbcs(const char *zFilename){ 31327 31709 char *zFilenameMbcs; 31328 31710 WCHAR *zTmpWide; 31329 31711 31330 31712 zTmpWide = utf8ToUnicode(zFilename); 31331 31713 if( zTmpWide==0 ){ 31332 31714 return 0; 31333 31715 } 31334 31716 zFilenameMbcs = unicodeToMbcs(zTmpWide); 31335 31717 free(zTmpWide); 31336 31718 return zFilenameMbcs; 31337 31719 } 31338 31720 31721 + 31722 +/* 31723 +** The return value of getLastErrorMsg 31724 +** is zero if the error message fits in the buffer, or non-zero 31725 +** otherwise (if the message was truncated). 31726 +*/ 31727 +static int getLastErrorMsg(int nBuf, char *zBuf){ 31728 + /* FormatMessage returns 0 on failure. Otherwise it 31729 + ** returns the number of TCHARs written to the output 31730 + ** buffer, excluding the terminating null char. 31731 + */ 31732 + DWORD error = GetLastError(); 31733 + DWORD dwLen = 0; 31734 + char *zOut = 0; 31735 + 31736 + if( isNT() ){ 31737 + WCHAR *zTempWide = NULL; 31738 + dwLen = FormatMessageW(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, 31739 + NULL, 31740 + error, 31741 + 0, 31742 + (LPWSTR) &zTempWide, 31743 + 0, 31744 + 0); 31745 + if( dwLen > 0 ){ 31746 + /* allocate a buffer and convert to UTF8 */ 31747 + zOut = unicodeToUtf8(zTempWide); 31748 + /* free the system buffer allocated by FormatMessage */ 31749 + LocalFree(zTempWide); 31750 + } 31751 +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. 31752 +** Since the ASCII version of these Windows API do not exist for WINCE, 31753 +** it's important to not reference them for WINCE builds. 31754 +*/ 31755 +#if SQLITE_OS_WINCE==0 31756 + }else{ 31757 + char *zTemp = NULL; 31758 + dwLen = FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, 31759 + NULL, 31760 + error, 31761 + 0, 31762 + (LPSTR) &zTemp, 31763 + 0, 31764 + 0); 31765 + if( dwLen > 0 ){ 31766 + /* allocate a buffer and convert to UTF8 */ 31767 + zOut = sqlite3_win32_mbcs_to_utf8(zTemp); 31768 + /* free the system buffer allocated by FormatMessage */ 31769 + LocalFree(zTemp); 31770 + } 31771 +#endif 31772 + } 31773 + if( 0 == dwLen ){ 31774 + sqlite3_snprintf(nBuf, zBuf, "OsError 0x%x (%u)", error, error); 31775 + }else{ 31776 + /* copy a maximum of nBuf chars to output buffer */ 31777 + sqlite3_snprintf(nBuf, zBuf, "%s", zOut); 31778 + /* free the UTF8 buffer */ 31779 + free(zOut); 31780 + } 31781 + return 0; 31782 +} 31783 + 31784 +/* 31785 +** 31786 +** This function - winLogErrorAtLine() - is only ever called via the macro 31787 +** winLogError(). 31788 +** 31789 +** This routine is invoked after an error occurs in an OS function. 31790 +** It logs a message using sqlite3_log() containing the current value of 31791 +** error code and, if possible, the human-readable equivalent from 31792 +** FormatMessage. 31793 +** 31794 +** The first argument passed to the macro should be the error code that 31795 +** will be returned to SQLite (e.g. SQLITE_IOERR_DELETE, SQLITE_CANTOPEN). 31796 +** The two subsequent arguments should be the name of the OS function that 31797 +** failed and the the associated file-system path, if any. 31798 +*/ 31799 +#define winLogError(a,b,c) winLogErrorAtLine(a,b,c,__LINE__) 31800 +static int winLogErrorAtLine( 31801 + int errcode, /* SQLite error code */ 31802 + const char *zFunc, /* Name of OS function that failed */ 31803 + const char *zPath, /* File path associated with error */ 31804 + int iLine /* Source line number where error occurred */ 31805 +){ 31806 + char zMsg[500]; /* Human readable error text */ 31807 + int i; /* Loop counter */ 31808 + DWORD iErrno = GetLastError(); /* Error code */ 31809 + 31810 + zMsg[0] = 0; 31811 + getLastErrorMsg(sizeof(zMsg), zMsg); 31812 + assert( errcode!=SQLITE_OK ); 31813 + if( zPath==0 ) zPath = ""; 31814 + for(i=0; zMsg[i] && zMsg[i]!='\r' && zMsg[i]!='\n'; i++){} 31815 + zMsg[i] = 0; 31816 + sqlite3_log(errcode, 31817 + "os_win.c:%d: (%d) %s(%s) - %s", 31818 + iLine, iErrno, zFunc, zPath, zMsg 31819 + ); 31820 + 31821 + return errcode; 31822 +} 31823 + 31339 31824 #if SQLITE_OS_WINCE 31340 31825 /************************************************************************* 31341 31826 ** This section contains code for WinCE only. 31342 31827 */ 31343 31828 /* 31344 31829 ** WindowsCE does not have a localtime() function. So create a 31345 31830 ** substitute. ................................................................................ 31408 31893 if (*zTok == '\\') *zTok = '_'; 31409 31894 } 31410 31895 31411 31896 /* Create/open the named mutex */ 31412 31897 pFile->hMutex = CreateMutexW(NULL, FALSE, zName); 31413 31898 if (!pFile->hMutex){ 31414 31899 pFile->lastErrno = GetLastError(); 31900 + winLogError(SQLITE_ERROR, "winceCreateLock1", zFilename); 31415 31901 free(zName); 31416 31902 return FALSE; 31417 31903 } 31418 31904 31419 31905 /* Acquire the mutex before continuing */ 31420 31906 winceMutexAcquire(pFile->hMutex); 31421 31907 ................................................................................ 31439 31925 /* If we succeeded in making the shared memory handle, map it. */ 31440 31926 if (pFile->hShared){ 31441 31927 pFile->shared = (winceLock*)MapViewOfFile(pFile->hShared, 31442 31928 FILE_MAP_READ|FILE_MAP_WRITE, 0, 0, sizeof(winceLock)); 31443 31929 /* If mapping failed, close the shared memory handle and erase it */ 31444 31930 if (!pFile->shared){ 31445 31931 pFile->lastErrno = GetLastError(); 31932 + winLogError(SQLITE_ERROR, "winceCreateLock2", zFilename); 31446 31933 CloseHandle(pFile->hShared); 31447 31934 pFile->hShared = NULL; 31448 31935 } 31449 31936 } 31450 31937 31451 31938 /* If shared memory could not be created, then close the mutex and fail */ 31452 31939 if (pFile->hShared == NULL){ ................................................................................ 31684 32171 ** INVALID_SET_FILE_POINTER may also be a valid new offset. So to determine 31685 32172 ** whether an error has actually occured, it is also necessary to call 31686 32173 ** GetLastError(). 31687 32174 */ 31688 32175 dwRet = SetFilePointer(pFile->h, lowerBits, &upperBits, FILE_BEGIN); 31689 32176 if( (dwRet==INVALID_SET_FILE_POINTER && GetLastError()!=NO_ERROR) ){ 31690 32177 pFile->lastErrno = GetLastError(); 32178 + winLogError(SQLITE_IOERR_SEEK, "seekWinFile", pFile->zPath); 31691 32179 return 1; 31692 32180 } 31693 32181 31694 32182 return 0; 31695 32183 } 31696 32184 31697 32185 /* ................................................................................ 31729 32217 Sleep(100); /* Wait a little before trying again */ 31730 32218 } 31731 32219 free(pFile->zDeleteOnClose); 31732 32220 } 31733 32221 #endif 31734 32222 OSTRACE(("CLOSE %d %s\n", pFile->h, rc ? "ok" : "failed")); 31735 32223 OpenCounter(-1); 31736 - return rc ? SQLITE_OK : SQLITE_IOERR; 32224 + return rc ? SQLITE_OK 32225 + : winLogError(SQLITE_IOERR_CLOSE, "winClose", pFile->zPath); 31737 32226 } 31738 32227 31739 32228 /* 31740 32229 ** Read data from a file into a buffer. Return SQLITE_OK if all 31741 32230 ** bytes were read successfully and SQLITE_IOERR if anything goes 31742 32231 ** wrong. 31743 32232 */ ................................................................................ 31755 32244 OSTRACE(("READ %d lock=%d\n", pFile->h, pFile->locktype)); 31756 32245 31757 32246 if( seekWinFile(pFile, offset) ){ 31758 32247 return SQLITE_FULL; 31759 32248 } 31760 32249 if( !ReadFile(pFile->h, pBuf, amt, &nRead, 0) ){ 31761 32250 pFile->lastErrno = GetLastError(); 31762 - return SQLITE_IOERR_READ; 32251 + return winLogError(SQLITE_IOERR_READ, "winRead", pFile->zPath); 31763 32252 } 31764 32253 if( nRead<(DWORD)amt ){ 31765 32254 /* Unread parts of the buffer must be zero-filled */ 31766 32255 memset(&((char*)pBuf)[nRead], 0, amt-nRead); 31767 32256 return SQLITE_IOERR_SHORT_READ; 31768 32257 } 31769 32258 ................................................................................ 31803 32292 if( nRem>0 ){ 31804 32293 pFile->lastErrno = GetLastError(); 31805 32294 rc = 1; 31806 32295 } 31807 32296 } 31808 32297 31809 32298 if( rc ){ 31810 - if( pFile->lastErrno==ERROR_HANDLE_DISK_FULL ){ 32299 + if( ( pFile->lastErrno==ERROR_HANDLE_DISK_FULL ) 32300 + || ( pFile->lastErrno==ERROR_DISK_FULL )){ 31811 32301 return SQLITE_FULL; 31812 32302 } 31813 - return SQLITE_IOERR_WRITE; 32303 + return winLogError(SQLITE_IOERR_WRITE, "winWrite", pFile->zPath); 31814 32304 } 31815 32305 return SQLITE_OK; 31816 32306 } 31817 32307 31818 32308 /* 31819 32309 ** Truncate an open file to a specified size 31820 32310 */ ................................................................................ 31834 32324 */ 31835 32325 if( pFile->szChunk ){ 31836 32326 nByte = ((nByte + pFile->szChunk - 1)/pFile->szChunk) * pFile->szChunk; 31837 32327 } 31838 32328 31839 32329 /* SetEndOfFile() returns non-zero when successful, or zero when it fails. */ 31840 32330 if( seekWinFile(pFile, nByte) ){ 31841 - rc = SQLITE_IOERR_TRUNCATE; 32331 + rc = winLogError(SQLITE_IOERR_TRUNCATE, "winTruncate1", pFile->zPath); 31842 32332 }else if( 0==SetEndOfFile(pFile->h) ){ 31843 32333 pFile->lastErrno = GetLastError(); 31844 - rc = SQLITE_IOERR_TRUNCATE; 32334 + rc = winLogError(SQLITE_IOERR_TRUNCATE, "winTruncate2", pFile->zPath); 31845 32335 } 31846 32336 31847 32337 OSTRACE(("TRUNCATE %d %lld %s\n", pFile->h, nByte, rc ? "failed" : "ok")); 31848 32338 return rc; 31849 32339 } 31850 32340 31851 32341 #ifdef SQLITE_TEST ................................................................................ 31859 32349 31860 32350 /* 31861 32351 ** Make sure all writes to a particular file are committed to disk. 31862 32352 */ 31863 32353 static int winSync(sqlite3_file *id, int flags){ 31864 32354 #if !defined(NDEBUG) || !defined(SQLITE_NO_SYNC) || defined(SQLITE_DEBUG) 31865 32355 winFile *pFile = (winFile*)id; 32356 + BOOL rc; 31866 32357 #else 31867 32358 UNUSED_PARAMETER(id); 31868 32359 #endif 31869 32360 31870 32361 assert( pFile ); 31871 32362 /* Check that one of SQLITE_SYNC_NORMAL or FULL was passed */ 31872 32363 assert((flags&0x0F)==SQLITE_SYNC_NORMAL 31873 32364 || (flags&0x0F)==SQLITE_SYNC_FULL 31874 32365 ); 31875 32366 31876 32367 OSTRACE(("SYNC %d lock=%d\n", pFile->h, pFile->locktype)); 31877 32368 32369 + /* Unix cannot, but some systems may return SQLITE_FULL from here. This 32370 + ** line is to test that doing so does not cause any problems. 32371 + */ 32372 + SimulateDiskfullError( return SQLITE_FULL ); 32373 + 31878 32374 #ifndef SQLITE_TEST 31879 32375 UNUSED_PARAMETER(flags); 31880 32376 #else 31881 - if( flags & SQLITE_SYNC_FULL ){ 32377 + if( (flags&0x0F)==SQLITE_SYNC_FULL ){ 31882 32378 sqlite3_fullsync_count++; 31883 32379 } 31884 32380 sqlite3_sync_count++; 31885 32381 #endif 31886 32382 31887 - /* Unix cannot, but some systems may return SQLITE_FULL from here. This 31888 - ** line is to test that doing so does not cause any problems. 31889 - */ 31890 - SimulateDiskfullError( return SQLITE_FULL ); 31891 - SimulateIOError( return SQLITE_IOERR; ); 31892 - 31893 32383 /* If we compiled with the SQLITE_NO_SYNC flag, then syncing is a 31894 32384 ** no-op 31895 32385 */ 31896 32386 #ifdef SQLITE_NO_SYNC 31897 32387 return SQLITE_OK; 31898 32388 #else 31899 - if( FlushFileBuffers(pFile->h) ){ 32389 + rc = FlushFileBuffers(pFile->h); 32390 + SimulateIOError( rc=FALSE ); 32391 + if( rc ){ 31900 32392 return SQLITE_OK; 31901 32393 }else{ 31902 32394 pFile->lastErrno = GetLastError(); 31903 - return SQLITE_IOERR; 32395 + return winLogError(SQLITE_IOERR_FSYNC, "winSync", pFile->zPath); 31904 32396 } 31905 32397 #endif 31906 32398 } 31907 32399 31908 32400 /* 31909 32401 ** Determine the current size of a file in bytes 31910 32402 */ ................................................................................ 31917 32409 assert( id!=0 ); 31918 32410 SimulateIOError(return SQLITE_IOERR_FSTAT); 31919 32411 lowerBits = GetFileSize(pFile->h, &upperBits); 31920 32412 if( (lowerBits == INVALID_FILE_SIZE) 31921 32413 && ((error = GetLastError()) != NO_ERROR) ) 31922 32414 { 31923 32415 pFile->lastErrno = error; 31924 - return SQLITE_IOERR_FSTAT; 32416 + return winLogError(SQLITE_IOERR_FSTAT, "winFileSize", pFile->zPath); 31925 32417 } 31926 32418 *pSize = (((sqlite3_int64)upperBits)<<32) + lowerBits; 31927 32419 return SQLITE_OK; 31928 32420 } 31929 32421 31930 32422 /* 31931 32423 ** LOCKFILE_FAIL_IMMEDIATELY is undefined on some Windows systems. ................................................................................ 31956 32448 sqlite3_randomness(sizeof(lk), &lk); 31957 32449 pFile->sharedLockByte = (short)((lk & 0x7fffffff)%(SHARED_SIZE - 1)); 31958 32450 res = LockFile(pFile->h, SHARED_FIRST+pFile->sharedLockByte, 0, 1, 0); 31959 32451 #endif 31960 32452 } 31961 32453 if( res == 0 ){ 31962 32454 pFile->lastErrno = GetLastError(); 32455 + /* No need to log a failure to lock */ 31963 32456 } 31964 32457 return res; 31965 32458 } 31966 32459 31967 32460 /* 31968 32461 ** Undo a readlock 31969 32462 */ ................................................................................ 31974 32467 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. 31975 32468 */ 31976 32469 #if SQLITE_OS_WINCE==0 31977 32470 }else{ 31978 32471 res = UnlockFile(pFile->h, SHARED_FIRST + pFile->sharedLockByte, 0, 1, 0); 31979 32472 #endif 31980 32473 } 31981 - if( res == 0 ){ 32474 + if( res==0 && GetLastError()!=ERROR_NOT_LOCKED ){ 31982 32475 pFile->lastErrno = GetLastError(); 32476 + winLogError(SQLITE_IOERR_UNLOCK, "unlockReadLock", pFile->zPath); 31983 32477 } 31984 32478 return res; 31985 32479 } 31986 32480 31987 32481 /* 31988 32482 ** Lock the file with the lock specified by parameter locktype - one 31989 32483 ** of the following: ................................................................................ 32176 32670 pFile->locktype, pFile->sharedLockByte)); 32177 32671 type = pFile->locktype; 32178 32672 if( type>=EXCLUSIVE_LOCK ){ 32179 32673 UnlockFile(pFile->h, SHARED_FIRST, 0, SHARED_SIZE, 0); 32180 32674 if( locktype==SHARED_LOCK && !getReadLock(pFile) ){ 32181 32675 /* This should never happen. We should always be able to 32182 32676 ** reacquire the read lock */ 32183 - rc = SQLITE_IOERR_UNLOCK; 32677 + rc = winLogError(SQLITE_IOERR_UNLOCK, "winUnlock", pFile->zPath); 32184 32678 } 32185 32679 } 32186 32680 if( type>=RESERVED_LOCK ){ 32187 32681 UnlockFile(pFile->h, RESERVED_BYTE, 0, 1, 0); 32188 32682 } 32189 32683 if( locktype==NO_LOCK && type>=SHARED_LOCK ){ 32190 32684 unlockReadLock(pFile); ................................................................................ 32491 32985 if( pNew==0 ){ 32492 32986 sqlite3_free(p); 32493 32987 return SQLITE_NOMEM; 32494 32988 } 32495 32989 memset(pNew, 0, sizeof(*pNew)); 32496 32990 pNew->zFilename = (char*)&pNew[1]; 32497 32991 sqlite3_snprintf(nName+15, pNew->zFilename, "%s-shm", pDbFd->zPath); 32992 + sqlite3FileSuffix3(pDbFd->zPath, pNew->zFilename); 32498 32993 32499 32994 /* Look to see if there is an existing winShmNode that can be used. 32500 32995 ** If no matching winShmNode currently exists, create a new one. 32501 32996 */ 32502 32997 winShmEnterMutex(); 32503 32998 for(pShmNode = winShmNodeList; pShmNode; pShmNode=pShmNode->pNext){ 32504 32999 /* TBD need to come up with better match here. Perhaps ................................................................................ 32533 33028 32534 33029 /* Check to see if another process is holding the dead-man switch. 32535 33030 ** If not, truncate the file to zero length. 32536 33031 */ 32537 33032 if( winShmSystemLock(pShmNode, _SHM_WRLCK, WIN_SHM_DMS, 1)==SQLITE_OK ){ 32538 33033 rc = winTruncate((sqlite3_file *)&pShmNode->hFile, 0); 32539 33034 if( rc!=SQLITE_OK ){ 32540 - rc = SQLITE_IOERR_SHMOPEN; 33035 + rc = winLogError(SQLITE_IOERR_SHMOPEN, "winOpenShm", pDbFd->zPath); 32541 33036 } 32542 33037 } 32543 33038 if( rc==SQLITE_OK ){ 32544 33039 winShmSystemLock(pShmNode, _SHM_UNLCK, WIN_SHM_DMS, 1); 32545 33040 rc = winShmSystemLock(pShmNode, _SHM_RDLCK, WIN_SHM_DMS, 1); 32546 33041 } 32547 33042 if( rc ) goto shm_open_err; ................................................................................ 32792 33287 32793 33288 /* The requested region is not mapped into this processes address space. 32794 33289 ** Check to see if it has been allocated (i.e. if the wal-index file is 32795 33290 ** large enough to contain the requested region). 32796 33291 */ 32797 33292 rc = winFileSize((sqlite3_file *)&pShmNode->hFile, &sz); 32798 33293 if( rc!=SQLITE_OK ){ 32799 - rc = SQLITE_IOERR_SHMSIZE; 33294 + rc = winLogError(SQLITE_IOERR_SHMSIZE, "winShmMap1", pDbFd->zPath); 32800 33295 goto shmpage_out; 32801 33296 } 32802 33297 32803 33298 if( sz<nByte ){ 32804 33299 /* The requested memory region does not exist. If isWrite is set to 32805 33300 ** zero, exit early. *pp will be set to NULL and SQLITE_OK returned. 32806 33301 ** 32807 33302 ** Alternatively, if isWrite is non-zero, use ftruncate() to allocate 32808 33303 ** the requested memory region. 32809 33304 */ 32810 33305 if( !isWrite ) goto shmpage_out; 32811 33306 rc = winTruncate((sqlite3_file *)&pShmNode->hFile, nByte); 32812 33307 if( rc!=SQLITE_OK ){ 32813 - rc = SQLITE_IOERR_SHMSIZE; 33308 + rc = winLogError(SQLITE_IOERR_SHMSIZE, "winShmMap2", pDbFd->zPath); 32814 33309 goto shmpage_out; 32815 33310 } 32816 33311 } 32817 33312 32818 33313 /* Map the requested memory region into this processes address space. */ 32819 33314 apNew = (struct ShmRegion *)sqlite3_realloc( 32820 33315 pShmNode->aRegion, (iRegion+1)*sizeof(apNew[0]) ................................................................................ 32843 33338 ); 32844 33339 OSTRACE(("SHM-MAP pid-%d map region=%d offset=%d size=%d %s\n", 32845 33340 (int)GetCurrentProcessId(), pShmNode->nRegion, iOffset, szRegion, 32846 33341 pMap ? "ok" : "failed")); 32847 33342 } 32848 33343 if( !pMap ){ 32849 33344 pShmNode->lastErrno = GetLastError(); 32850 - rc = SQLITE_IOERR; 33345 + rc = winLogError(SQLITE_IOERR_SHMMAP, "winShmMap3", pDbFd->zPath); 32851 33346 if( hMap ) CloseHandle(hMap); 32852 33347 goto shmpage_out; 32853 33348 } 32854 33349 32855 33350 pShmNode->aRegion[pShmNode->nRegion].pMap = pMap; 32856 33351 pShmNode->aRegion[pShmNode->nRegion].hMap = hMap; 32857 33352 pShmNode->nRegion++; ................................................................................ 32925 33420 void *zConverted = 0; 32926 33421 if( isNT() ){ 32927 33422 zConverted = utf8ToUnicode(zFilename); 32928 33423 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. 32929 33424 */ 32930 33425 #if SQLITE_OS_WINCE==0 32931 33426 }else{ 32932 - zConverted = utf8ToMbcs(zFilename); 33427 + zConverted = sqlite3_win32_utf8_to_mbcs(zFilename); 32933 33428 #endif 32934 33429 } 32935 33430 /* caller will handle out of memory */ 32936 33431 return zConverted; 32937 33432 } 32938 33433 32939 33434 /* ................................................................................ 33005 33500 } 33006 33501 zBuf[j] = 0; 33007 33502 33008 33503 OSTRACE(("TEMP FILENAME: %s\n", zBuf)); 33009 33504 return SQLITE_OK; 33010 33505 } 33011 33506 33012 -/* 33013 -** The return value of getLastErrorMsg 33014 -** is zero if the error message fits in the buffer, or non-zero 33015 -** otherwise (if the message was truncated). 33016 -*/ 33017 -static int getLastErrorMsg(int nBuf, char *zBuf){ 33018 - /* FormatMessage returns 0 on failure. Otherwise it 33019 - ** returns the number of TCHARs written to the output 33020 - ** buffer, excluding the terminating null char. 33021 - */ 33022 - DWORD error = GetLastError(); 33023 - DWORD dwLen = 0; 33024 - char *zOut = 0; 33025 - 33026 - if( isNT() ){ 33027 - WCHAR *zTempWide = NULL; 33028 - dwLen = FormatMessageW(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, 33029 - NULL, 33030 - error, 33031 - 0, 33032 - (LPWSTR) &zTempWide, 33033 - 0, 33034 - 0); 33035 - if( dwLen > 0 ){ 33036 - /* allocate a buffer and convert to UTF8 */ 33037 - zOut = unicodeToUtf8(zTempWide); 33038 - /* free the system buffer allocated by FormatMessage */ 33039 - LocalFree(zTempWide); 33040 - } 33041 -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. 33042 -** Since the ASCII version of these Windows API do not exist for WINCE, 33043 -** it's important to not reference them for WINCE builds. 33044 -*/ 33045 -#if SQLITE_OS_WINCE==0 33046 - }else{ 33047 - char *zTemp = NULL; 33048 - dwLen = FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, 33049 - NULL, 33050 - error, 33051 - 0, 33052 - (LPSTR) &zTemp, 33053 - 0, 33054 - 0); 33055 - if( dwLen > 0 ){ 33056 - /* allocate a buffer and convert to UTF8 */ 33057 - zOut = sqlite3_win32_mbcs_to_utf8(zTemp); 33058 - /* free the system buffer allocated by FormatMessage */ 33059 - LocalFree(zTemp); 33060 - } 33061 -#endif 33062 - } 33063 - if( 0 == dwLen ){ 33064 - sqlite3_snprintf(nBuf, zBuf, "OsError 0x%x (%u)", error, error); 33065 - }else{ 33066 - /* copy a maximum of nBuf chars to output buffer */ 33067 - sqlite3_snprintf(nBuf, zBuf, "%s", zOut); 33068 - /* free the UTF8 buffer */ 33069 - free(zOut); 33070 - } 33071 - return 0; 33072 -} 33073 - 33074 33507 /* 33075 33508 ** Open a file. 33076 33509 */ 33077 33510 static int winOpen( 33078 33511 sqlite3_vfs *pVfs, /* Not used */ 33079 33512 const char *zName, /* Name of the file (UTF-8) */ 33080 33513 sqlite3_file *id, /* Write the SQLite file handle here */ ................................................................................ 33238 33671 33239 33672 OSTRACE(("OPEN %d %s 0x%lx %s\n", 33240 33673 h, zName, dwDesiredAccess, 33241 33674 h==INVALID_HANDLE_VALUE ? "failed" : "ok")); 33242 33675 33243 33676 if( h==INVALID_HANDLE_VALUE ){ 33244 33677 pFile->lastErrno = GetLastError(); 33678 + winLogError(SQLITE_CANTOPEN, "winOpen", zUtf8Name); 33245 33679 free(zConverted); 33246 33680 if( isReadWrite ){ 33247 33681 return winOpen(pVfs, zName, id, 33248 33682 ((flags|SQLITE_OPEN_READONLY)&~(SQLITE_OPEN_CREATE|SQLITE_OPEN_READWRITE)), pOutFlags); 33249 33683 }else{ 33250 33684 return SQLITE_CANTOPEN_BKPT; 33251 33685 } ................................................................................ 33341 33775 } 33342 33776 free(zConverted); 33343 33777 OSTRACE(("DELETE \"%s\" %s\n", zFilename, 33344 33778 ( (rc==INVALID_FILE_ATTRIBUTES) && (error==ERROR_FILE_NOT_FOUND)) ? 33345 33779 "ok" : "failed" )); 33346 33780 33347 33781 return ( (rc == INVALID_FILE_ATTRIBUTES) 33348 - && (error == ERROR_FILE_NOT_FOUND)) ? SQLITE_OK : SQLITE_IOERR_DELETE; 33782 + && (error == ERROR_FILE_NOT_FOUND)) ? SQLITE_OK : 33783 + winLogError(SQLITE_IOERR_DELETE, "winDelete", zFilename); 33349 33784 } 33350 33785 33351 33786 /* 33352 33787 ** Check the existance and status of a file. 33353 33788 */ 33354 33789 static int winAccess( 33355 33790 sqlite3_vfs *pVfs, /* Not used on win32 */ ................................................................................ 33381 33816 && sAttrData.nFileSizeLow==0 ){ 33382 33817 attr = INVALID_FILE_ATTRIBUTES; 33383 33818 }else{ 33384 33819 attr = sAttrData.dwFileAttributes; 33385 33820 } 33386 33821 }else{ 33387 33822 if( GetLastError()!=ERROR_FILE_NOT_FOUND ){ 33823 + winLogError(SQLITE_IOERR_ACCESS, "winAccess", zFilename); 33388 33824 free(zConverted); 33389 33825 return SQLITE_IOERR_ACCESS; 33390 33826 }else{ 33391 33827 attr = INVALID_FILE_ATTRIBUTES; 33392 33828 } 33393 33829 } 33394 33830 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ................................................................................ 33444 33880 return SQLITE_OK; 33445 33881 #endif 33446 33882 33447 33883 #if !SQLITE_OS_WINCE && !defined(__CYGWIN__) 33448 33884 int nByte; 33449 33885 void *zConverted; 33450 33886 char *zOut; 33887 + 33888 + /* If this path name begins with "/X:", where "X" is any alphabetic 33889 + ** character, discard the initial "/" from the pathname. 33890 + */ 33891 + if( zRelative[0]=='/' && sqlite3Isalpha(zRelative[1]) && zRelative[2]==':' ){ 33892 + zRelative++; 33893 + } 33451 33894 33452 33895 /* It's odd to simulate an io-error here, but really this is just 33453 33896 ** using the io-error infrastructure to test that SQLite handles this 33454 33897 ** function failing. This function could fail if, for example, the 33455 33898 ** current working directory has been unlinked. 33456 33899 */ 33457 33900 SimulateIOError( return SQLITE_ERROR ); ................................................................................ 34485 34928 ); 34486 34929 pCache->pSynced = pPg; 34487 34930 if( !pPg ){ 34488 34931 for(pPg=pCache->pDirtyTail; pPg && pPg->nRef; pPg=pPg->pDirtyPrev); 34489 34932 } 34490 34933 if( pPg ){ 34491 34934 int rc; 34935 +#ifdef SQLITE_LOG_CACHE_SPILL 34936 + sqlite3_log(SQLITE_FULL, 34937 + "spill page %d making room for %d - cache used: %d/%d", 34938 + pPg->pgno, pgno, 34939 + sqlite3GlobalConfig.pcache.xPagecount(pCache->pCache), 34940 + pCache->nMax); 34941 +#endif 34492 34942 rc = pCache->xStress(pCache->pStress, pPg); 34493 34943 if( rc!=SQLITE_OK && rc!=SQLITE_BUSY ){ 34494 34944 return rc; 34495 34945 } 34496 34946 } 34497 34947 34498 34948 pPage = sqlite3GlobalConfig.pcache.xFetch(pCache->pCache, pgno, 2); ................................................................................ 35395 35845 pCache = (PCache1 *)sqlite3_malloc(sz); 35396 35846 if( pCache ){ 35397 35847 memset(pCache, 0, sz); 35398 35848 if( separateCache ){ 35399 35849 pGroup = (PGroup*)&pCache[1]; 35400 35850 pGroup->mxPinned = 10; 35401 35851 }else{ 35402 - pGroup = &pcache1_g.grp; 35852 + pGroup = &pcache1.grp; 35403 35853 } 35404 35854 pCache->pGroup = pGroup; 35405 35855 pCache->szPage = szPage; 35406 35856 pCache->bPurgeable = (bPurgeable ? 1 : 0); 35407 35857 if( bPurgeable ){ 35408 35858 pCache->nMin = 10; 35409 35859 pcache1EnterMutex(pGroup); ................................................................................ 36256 36706 36257 36707 #ifndef _WAL_H_ 36258 36708 #define _WAL_H_ 36259 36709 36260 36710 36261 36711 #ifdef SQLITE_OMIT_WAL 36262 36712 # define sqlite3WalOpen(x,y,z) 0 36713 +# define sqlite3WalLimit(x,y) 36263 36714 # define sqlite3WalClose(w,x,y,z) 0 36264 36715 # define sqlite3WalBeginReadTransaction(y,z) 0 36265 36716 # define sqlite3WalEndReadTransaction(z) 36266 36717 # define sqlite3WalRead(v,w,x,y,z) 0 36267 36718 # define sqlite3WalDbsize(y) 0 36268 36719 # define sqlite3WalBeginWriteTransaction(y) 0 36269 36720 # define sqlite3WalEndWriteTransaction(x) 0 ................................................................................ 36281 36732 36282 36733 /* Connection to a write-ahead log (WAL) file. 36283 36734 ** There is one object of this type for each pager. 36284 36735 */ 36285 36736 typedef struct Wal Wal; 36286 36737 36287 36738 /* Open and close a connection to a write-ahead log. */ 36288 -SQLITE_PRIVATE int sqlite3WalOpen(sqlite3_vfs*, sqlite3_file*, const char *zName, int, Wal**); 36739 +SQLITE_PRIVATE int sqlite3WalOpen(sqlite3_vfs*, sqlite3_file*, const char *, int, i64, Wal**); 36289 36740 SQLITE_PRIVATE int sqlite3WalClose(Wal *pWal, int sync_flags, int, u8 *); 36290 36741 36742 +/* Set the limiting size of a WAL file. */ 36743 +SQLITE_PRIVATE void sqlite3WalLimit(Wal*, i64); 36744 + 36291 36745 /* Used by readers to open (lock) and close (unlock) a snapshot. A 36292 36746 ** snapshot is like a read-transaction. It is the state of the database 36293 36747 ** at an instant in time. sqlite3WalOpenSnapshot gets a read lock and 36294 36748 ** preserves the current state even if the other threads or processes 36295 36749 ** write to or checkpoint the WAL. sqlite3WalCloseSnapshot() closes the 36296 36750 ** transaction and releases the lock. 36297 36751 */ ................................................................................ 40632 41086 int journalFileSize; /* Bytes to allocate for each journal fd */ 40633 41087 char *zPathname = 0; /* Full path to database file */ 40634 41088 int nPathname = 0; /* Number of bytes in zPathname */ 40635 41089 int useJournal = (flags & PAGER_OMIT_JOURNAL)==0; /* False to omit journal */ 40636 41090 int noReadlock = (flags & PAGER_NO_READLOCK)!=0; /* True to omit read-lock */ 40637 41091 int pcacheSize = sqlite3PcacheSize(); /* Bytes to allocate for PCache */ 40638 41092 u32 szPageDflt = SQLITE_DEFAULT_PAGE_SIZE; /* Default page size */ 41093 + const char *zUri = 0; /* URI args to copy */ 41094 + int nUri = 0; /* Number of bytes of URI args at *zUri */ 40639 41095 40640 41096 /* Figure out how much space is required for each journal file-handle 40641 41097 ** (there are two of them, the main journal and the sub-journal). This 40642 41098 ** is the maximum space required for an in-memory journal file handle 40643 41099 ** and a regular journal file-handle. Note that a "regular journal-handle" 40644 41100 ** may be a wrapper capable of caching the first portion of the journal 40645 41101 ** file in memory to implement the atomic-write optimization (see ................................................................................ 40662 41118 #endif 40663 41119 40664 41120 /* Compute and store the full pathname in an allocated buffer pointed 40665 41121 ** to by zPathname, length nPathname. Or, if this is a temporary file, 40666 41122 ** leave both nPathname and zPathname set to 0. 40667 41123 */ 40668 41124 if( zFilename && zFilename[0] ){ 41125 + const char *z; 40669 41126 nPathname = pVfs->mxPathname+1; 40670 41127 zPathname = sqlite3Malloc(nPathname*2); 40671 41128 if( zPathname==0 ){ 40672 41129 return SQLITE_NOMEM; 40673 41130 } 40674 41131 zPathname[0] = 0; /* Make sure initialized even if FullPathname() fails */ 40675 41132 rc = sqlite3OsFullPathname(pVfs, zFilename, nPathname, zPathname); 40676 41133 nPathname = sqlite3Strlen30(zPathname); 41134 + z = zUri = &zFilename[sqlite3Strlen30(zFilename)+1]; 41135 + while( *z ){ 41136 + z += sqlite3Strlen30(z)+1; 41137 + z += sqlite3Strlen30(z)+1; 41138 + } 41139 + nUri = &z[1] - zUri; 40677 41140 if( rc==SQLITE_OK && nPathname+8>pVfs->mxPathname ){ 40678 41141 /* This branch is taken when the journal path required by 40679 41142 ** the database being opened will be more than pVfs->mxPathname 40680 41143 ** bytes in length. This means the database cannot be opened, 40681 41144 ** as it will not be possible to open the journal file or even 40682 41145 ** check for a hot-journal before reading. 40683 41146 */ ................................................................................ 40702 41165 ** Journal file name (nPathname+8+1 bytes) 40703 41166 */ 40704 41167 pPtr = (u8 *)sqlite3MallocZero( 40705 41168 ROUND8(sizeof(*pPager)) + /* Pager structure */ 40706 41169 ROUND8(pcacheSize) + /* PCache object */ 40707 41170 ROUND8(pVfs->szOsFile) + /* The main db file */ 40708 41171 journalFileSize * 2 + /* The two journal files */ 40709 - nPathname + 1 + /* zFilename */ 41172 + nPathname + 1 + nUri + /* zFilename */ 40710 41173 nPathname + 8 + 1 /* zJournal */ 40711 41174 #ifndef SQLITE_OMIT_WAL 40712 41175 + nPathname + 4 + 1 /* zWal */ 40713 41176 #endif 40714 41177 ); 40715 41178 assert( EIGHT_BYTE_ALIGNMENT(SQLITE_INT_TO_PTR(journalFileSize)) ); 40716 41179 if( !pPtr ){ ................................................................................ 40724 41187 pPager->jfd = (sqlite3_file*)(pPtr += journalFileSize); 40725 41188 pPager->zFilename = (char*)(pPtr += journalFileSize); 40726 41189 assert( EIGHT_BYTE_ALIGNMENT(pPager->jfd) ); 40727 41190 40728 41191 /* Fill in the Pager.zFilename and Pager.zJournal buffers, if required. */ 40729 41192 if( zPathname ){ 40730 41193 assert( nPathname>0 ); 40731 - pPager->zJournal = (char*)(pPtr += nPathname + 1); 41194 + pPager->zJournal = (char*)(pPtr += nPathname + 1 + nUri); 40732 41195 memcpy(pPager->zFilename, zPathname, nPathname); 41196 + memcpy(&pPager->zFilename[nPathname+1], zUri, nUri); 40733 41197 memcpy(pPager->zJournal, zPathname, nPathname); 40734 41198 memcpy(&pPager->zJournal[nPathname], "-journal", 8); 41199 + sqlite3FileSuffix3(pPager->zFilename, pPager->zJournal); 40735 41200 #ifndef SQLITE_OMIT_WAL 40736 41201 pPager->zWal = &pPager->zJournal[nPathname+8+1]; 40737 41202 memcpy(pPager->zWal, zPathname, nPathname); 40738 41203 memcpy(&pPager->zWal[nPathname], "-wal", 4); 41204 + sqlite3FileSuffix3(pPager->zFilename, pPager->zWal); 40739 41205 #endif 40740 41206 sqlite3_free(zPathname); 40741 41207 } 40742 41208 pPager->pVfs = pVfs; 40743 41209 pPager->vfsFlags = vfsFlags; 40744 41210 40745 41211 /* Open the pager file. ................................................................................ 42076 42542 if( pList==0 ){ 42077 42543 /* Must have at least one page for the WAL commit flag. 42078 42544 ** Ticket [2d1a5c67dfc2363e44f29d9bbd57f] 2011-05-18 */ 42079 42545 rc = sqlite3PagerGet(pPager, 1, &pPageOne); 42080 42546 pList = pPageOne; 42081 42547 pList->pDirty = 0; 42082 42548 } 42083 - assert( pList!=0 || rc!=SQLITE_OK ); 42084 - if( pList ){ 42549 + assert( rc==SQLITE_OK ); 42550 + if( ALWAYS(pList) ){ 42085 42551 rc = pagerWalFrames(pPager, pList, pPager->dbSize, 1, 42086 42552 (pPager->fullSync ? pPager->syncFlags : 0) 42087 42553 ); 42088 42554 } 42089 42555 sqlite3PagerUnref(pPageOne); 42090 42556 if( rc==SQLITE_OK ){ 42091 42557 sqlite3PcacheCleanAll(pPager->pPCache); ................................................................................ 42940 43406 ** 42941 43407 ** Setting the size limit to -1 means no limit is enforced. 42942 43408 ** An attempt to set a limit smaller than -1 is a no-op. 42943 43409 */ 42944 43410 SQLITE_PRIVATE i64 sqlite3PagerJournalSizeLimit(Pager *pPager, i64 iLimit){ 42945 43411 if( iLimit>=-1 ){ 42946 43412 pPager->journalSizeLimit = iLimit; 43413 + sqlite3WalLimit(pPager->pWal, iLimit); 42947 43414 } 42948 43415 return pPager->journalSizeLimit; 42949 43416 } 42950 43417 42951 43418 /* 42952 43419 ** Return a pointer to the pPager->pBackup variable. The backup module 42953 43420 ** in backup.c maintains the content of this variable. This module ................................................................................ 43031 43498 } 43032 43499 43033 43500 /* Open the connection to the log file. If this operation fails, 43034 43501 ** (e.g. due to malloc() failure), return an error code. 43035 43502 */ 43036 43503 if( rc==SQLITE_OK ){ 43037 43504 rc = sqlite3WalOpen(pPager->pVfs, 43038 - pPager->fd, pPager->zWal, pPager->exclusiveMode, &pPager->pWal 43505 + pPager->fd, pPager->zWal, pPager->exclusiveMode, 43506 + pPager->journalSizeLimit, &pPager->pWal 43039 43507 ); 43040 43508 } 43041 43509 43042 43510 return rc; 43043 43511 } 43044 43512 43045 43513 ................................................................................ 43563 44031 ** following object. 43564 44032 */ 43565 44033 struct Wal { 43566 44034 sqlite3_vfs *pVfs; /* The VFS used to create pDbFd */ 43567 44035 sqlite3_file *pDbFd; /* File handle for the database file */ 43568 44036 sqlite3_file *pWalFd; /* File handle for WAL file */ 43569 44037 u32 iCallback; /* Value to pass to log callback (or 0) */ 44038 + i64 mxWalSize; /* Truncate WAL to this size upon reset */ 43570 44039 int nWiData; /* Size of array apWiData */ 43571 44040 volatile u32 **apWiData; /* Pointer to wal-index content in memory */ 43572 44041 u32 szPage; /* Database page size */ 43573 44042 i16 readLock; /* Which read lock is being held. -1 for none */ 43574 44043 u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */ 43575 44044 u8 writeLock; /* True if in a write transaction */ 43576 44045 u8 ckptLock; /* True if holding a checkpoint lock */ 43577 - u8 readOnly; /* True if the WAL file is open read-only */ 44046 + u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */ 43578 44047 WalIndexHdr hdr; /* Wal-index header for current transaction */ 43579 44048 const char *zWalName; /* Name of WAL file */ 43580 44049 u32 nCkpt; /* Checkpoint sequence counter in the wal-header */ 43581 44050 #ifdef SQLITE_DEBUG 43582 44051 u8 lockError; /* True if a locking error has occurred */ 43583 44052 #endif 43584 44053 }; ................................................................................ 43586 44055 /* 43587 44056 ** Candidate values for Wal.exclusiveMode. 43588 44057 */ 43589 44058 #define WAL_NORMAL_MODE 0 43590 44059 #define WAL_EXCLUSIVE_MODE 1 43591 44060 #define WAL_HEAPMEMORY_MODE 2 43592 44061 44062 +/* 44063 +** Possible values for WAL.readOnly 44064 +*/ 44065 +#define WAL_RDWR 0 /* Normal read/write connection */ 44066 +#define WAL_RDONLY 1 /* The WAL file is readonly */ 44067 +#define WAL_SHM_RDONLY 2 /* The SHM file is readonly */ 44068 + 43593 44069 /* 43594 44070 ** Each page of the wal-index mapping contains a hash-table made up of 43595 44071 ** an array of HASHTABLE_NSLOT elements of the following type. 43596 44072 */ 43597 44073 typedef u16 ht_slot; 43598 44074 43599 44075 /* ................................................................................ 43679 44155 if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){ 43680 44156 pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ); 43681 44157 if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM; 43682 44158 }else{ 43683 44159 rc = sqlite3OsShmMap(pWal->pDbFd, iPage, WALINDEX_PGSZ, 43684 44160 pWal->writeLock, (void volatile **)&pWal->apWiData[iPage] 43685 44161 ); 44162 + if( rc==SQLITE_READONLY ){ 44163 + pWal->readOnly |= WAL_SHM_RDONLY; 44164 + rc = SQLITE_OK; 44165 + } 43686 44166 } 43687 44167 } 43688 44168 43689 44169 *ppPage = pWal->apWiData[iPage]; 43690 44170 assert( iPage==0 || *ppPage || rc!=SQLITE_OK ); 43691 44171 return rc; 43692 44172 } ................................................................................ 44385 44865 ** an SQLite error code is returned and *ppWal is left unmodified. 44386 44866 */ 44387 44867 SQLITE_PRIVATE int sqlite3WalOpen( 44388 44868 sqlite3_vfs *pVfs, /* vfs module to open wal and wal-index */ 44389 44869 sqlite3_file *pDbFd, /* The open database file */ 44390 44870 const char *zWalName, /* Name of the WAL file */ 44391 44871 int bNoShm, /* True to run in heap-memory mode */ 44872 + i64 mxWalSize, /* Truncate WAL to this size on reset */ 44392 44873 Wal **ppWal /* OUT: Allocated Wal handle */ 44393 44874 ){ 44394 44875 int rc; /* Return Code */ 44395 44876 Wal *pRet; /* Object to allocate and return */ 44396 44877 int flags; /* Flags passed to OsOpen() */ 44397 44878 44398 44879 assert( zWalName && zWalName[0] ); ................................................................................ 44417 44898 return SQLITE_NOMEM; 44418 44899 } 44419 44900 44420 44901 pRet->pVfs = pVfs; 44421 44902 pRet->pWalFd = (sqlite3_file *)&pRet[1]; 44422 44903 pRet->pDbFd = pDbFd; 44423 44904 pRet->readLock = -1; 44905 + pRet->mxWalSize = mxWalSize; 44424 44906 pRet->zWalName = zWalName; 44425 44907 pRet->exclusiveMode = (bNoShm ? WAL_HEAPMEMORY_MODE: WAL_NORMAL_MODE); 44426 44908 44427 44909 /* Open file handle on the write-ahead log file. */ 44428 44910 flags = (SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|SQLITE_OPEN_WAL); 44429 44911 rc = sqlite3OsOpen(pVfs, zWalName, pRet->pWalFd, flags, &flags); 44430 44912 if( rc==SQLITE_OK && flags&SQLITE_OPEN_READONLY ){ 44431 - pRet->readOnly = 1; 44913 + pRet->readOnly = WAL_RDONLY; 44432 44914 } 44433 44915 44434 44916 if( rc!=SQLITE_OK ){ 44435 44917 walIndexClose(pRet, 0); 44436 44918 sqlite3OsClose(pRet->pWalFd); 44437 44919 sqlite3_free(pRet); 44438 44920 }else{ 44439 44921 *ppWal = pRet; 44440 44922 WALTRACE(("WAL%d: opened\n", pRet)); 44441 44923 } 44442 44924 return rc; 44443 44925 } 44926 + 44927 +/* 44928 +** Change the size to which the WAL file is trucated on each reset. 44929 +*/ 44930 +SQLITE_PRIVATE void sqlite3WalLimit(Wal *pWal, i64 iLimit){ 44931 + if( pWal ) pWal->mxWalSize = iLimit; 44932 +} 44444 44933 44445 44934 /* 44446 44935 ** Find the smallest page number out of all pages held in the WAL that 44447 44936 ** has not been returned by any prior invocation of this method on the 44448 44937 ** same WalIterator object. Write into *piFrame the frame index where 44449 44938 ** that page was last written into the WAL. Write into *piPage the page 44450 44939 ** number. ................................................................................ 45058 45547 */ 45059 45548 badHdr = (page0 ? walIndexTryHdr(pWal, pChanged) : 1); 45060 45549 45061 45550 /* If the first attempt failed, it might have been due to a race 45062 45551 ** with a writer. So get a WRITE lock and try again. 45063 45552 */ 45064 45553 assert( badHdr==0 || pWal->writeLock==0 ); 45065 - if( badHdr && SQLITE_OK==(rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1)) ){ 45066 - pWal->writeLock = 1; 45067 - if( SQLITE_OK==(rc = walIndexPage(pWal, 0, &page0)) ){ 45068 - badHdr = walIndexTryHdr(pWal, pChanged); 45069 - if( badHdr ){ 45070 - /* If the wal-index header is still malformed even while holding 45071 - ** a WRITE lock, it can only mean that the header is corrupted and 45072 - ** needs to be reconstructed. So run recovery to do exactly that. 45073 - */ 45074 - rc = walIndexRecover(pWal); 45075 - *pChanged = 1; 45554 + if( badHdr ){ 45555 + if( pWal->readOnly & WAL_SHM_RDONLY ){ 45556 + if( SQLITE_OK==(rc = walLockShared(pWal, WAL_WRITE_LOCK)) ){ 45557 + walUnlockShared(pWal, WAL_WRITE_LOCK); 45558 + rc = SQLITE_READONLY_RECOVERY; 45076 45559 } 45560 + }else if( SQLITE_OK==(rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1)) ){ 45561 + pWal->writeLock = 1; 45562 + if( SQLITE_OK==(rc = walIndexPage(pWal, 0, &page0)) ){ 45563 + badHdr = walIndexTryHdr(pWal, pChanged); 45564 + if( badHdr ){ 45565 + /* If the wal-index header is still malformed even while holding 45566 + ** a WRITE lock, it can only mean that the header is corrupted and 45567 + ** needs to be reconstructed. So run recovery to do exactly that. 45568 + */ 45569 + rc = walIndexRecover(pWal); 45570 + *pChanged = 1; 45571 + } 45572 + } 45573 + pWal->writeLock = 0; 45574 + walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1); 45077 45575 } 45078 - pWal->writeLock = 0; 45079 - walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1); 45080 45576 } 45081 45577 45082 45578 /* If the header is read successfully, check the version number to make 45083 45579 ** sure the wal-index was not constructed with some future format that 45084 45580 ** this version of SQLite cannot understand. 45085 45581 */ 45086 45582 if( badHdr==0 && pWal->hdr.iVersion!=WALINDEX_MAX_VERSION ){ ................................................................................ 45259 45755 assert( thisMark!=READMARK_NOT_USED ); 45260 45756 mxReadMark = thisMark; 45261 45757 mxI = i; 45262 45758 } 45263 45759 } 45264 45760 /* There was once an "if" here. The extra "{" is to preserve indentation. */ 45265 45761 { 45266 - if( mxReadMark < pWal->hdr.mxFrame || mxI==0 ){ 45762 + if( (pWal->readOnly & WAL_SHM_RDONLY)==0 45763 + && (mxReadMark<pWal->hdr.mxFrame || mxI==0) 45764 + ){ 45267 45765 for(i=1; i<WAL_NREADER; i++){ 45268 45766 rc = walLockExclusive(pWal, WAL_READ_LOCK(i), 1); 45269 45767 if( rc==SQLITE_OK ){ 45270 45768 mxReadMark = pInfo->aReadMark[i] = pWal->hdr.mxFrame; 45271 45769 mxI = i; 45272 45770 walUnlockExclusive(pWal, WAL_READ_LOCK(i), 1); 45273 45771 break; 45274 45772 }else if( rc!=SQLITE_BUSY ){ 45275 45773 return rc; 45276 45774 } 45277 45775 } 45278 45776 } 45279 45777 if( mxI==0 ){ 45280 - assert( rc==SQLITE_BUSY ); 45281 - return WAL_RETRY; 45778 + assert( rc==SQLITE_BUSY || (pWal->readOnly & WAL_SHM_RDONLY)!=0 ); 45779 + return rc==SQLITE_BUSY ? WAL_RETRY : SQLITE_READONLY_CANTLOCK; 45282 45780 } 45283 45781 45284 45782 rc = walLockShared(pWal, WAL_READ_LOCK(mxI)); 45285 45783 if( rc ){ 45286 45784 return rc==SQLITE_BUSY ? WAL_RETRY : rc; 45287 45785 } 45288 45786 /* Now that the read-lock has been obtained, check that neither the ................................................................................ 45673 46171 ** In theory it would be Ok to update the cache of the header only 45674 46172 ** at this point. But updating the actual wal-index header is also 45675 46173 ** safe and means there is no special case for sqlite3WalUndo() 45676 46174 ** to handle if this transaction is rolled back. 45677 46175 */ 45678 46176 int i; /* Loop counter */ 45679 46177 u32 *aSalt = pWal->hdr.aSalt; /* Big-endian salt values */ 46178 + 46179 + /* Limit the size of WAL file if the journal_size_limit PRAGMA is 46180 + ** set to a non-negative value. Log errors encountered 46181 + ** during the truncation attempt. */ 46182 + if( pWal->mxWalSize>=0 ){ 46183 + i64 sz; 46184 + int rx; 46185 + sqlite3BeginBenignMalloc(); 46186 + rx = sqlite3OsFileSize(pWal->pWalFd, &sz); 46187 + if( rx==SQLITE_OK && (sz > pWal->mxWalSize) ){ 46188 + rx = sqlite3OsTruncate(pWal->pWalFd, pWal->mxWalSize); 46189 + } 46190 + sqlite3EndBenignMalloc(); 46191 + if( rx ){ 46192 + sqlite3_log(rx, "cannot limit WAL size: %s", pWal->zWalName); 46193 + } 46194 + } 46195 + 45680 46196 pWal->nCkpt++; 45681 46197 pWal->hdr.mxFrame = 0; 45682 46198 sqlite3Put4byte((u8*)&aSalt[0], 1 + sqlite3Get4byte((u8*)&aSalt[0])); 45683 46199 aSalt[1] = salt1; 45684 46200 walIndexWriteHdr(pWal); 45685 46201 pInfo->nBackfill = 0; 45686 46202 for(i=1; i<WAL_NREADER; i++) pInfo->aReadMark[i] = READMARK_NOT_USED; ................................................................................ 45898 46414 int rc; /* Return code */ 45899 46415 int isChanged = 0; /* True if a new wal-index header is loaded */ 45900 46416 int eMode2 = eMode; /* Mode to pass to walCheckpoint() */ 45901 46417 45902 46418 assert( pWal->ckptLock==0 ); 45903 46419 assert( pWal->writeLock==0 ); 45904 46420 46421 + if( pWal->readOnly ) return SQLITE_READONLY; 45905 46422 WALTRACE(("WAL%p: checkpoint begins\n", pWal)); 45906 46423 rc = walLockExclusive(pWal, WAL_CKPT_LOCK, 1); 45907 46424 if( rc ){ 45908 46425 /* Usually this is SQLITE_BUSY meaning that another thread or process 45909 46426 ** is already running a checkpoint, or maybe a recovery. But it might 45910 46427 ** also be SQLITE_IOERR. */ 45911 46428 return rc; ................................................................................ 47778 48295 return; 47779 48296 } 47780 48297 offset = PTRMAP_PTROFFSET(iPtrmap, key); 47781 48298 if( offset<0 ){ 47782 48299 *pRC = SQLITE_CORRUPT_BKPT; 47783 48300 goto ptrmap_exit; 47784 48301 } 48302 + assert( offset <= (int)pBt->usableSize-5 ); 47785 48303 pPtrmap = (u8 *)sqlite3PagerGetData(pDbPage); 47786 48304 47787 48305 if( eType!=pPtrmap[offset] || get4byte(&pPtrmap[offset+1])!=parent ){ 47788 48306 TRACE(("PTRMAP_UPDATE: %d->(%d,%d)\n", key, eType, parent)); 47789 48307 *pRC= rc = sqlite3PagerWrite(pDbPage); 47790 48308 if( rc==SQLITE_OK ){ 47791 48309 pPtrmap[offset] = eType; ................................................................................ 47817 48335 rc = sqlite3PagerGet(pBt->pPager, iPtrmap, &pDbPage); 47818 48336 if( rc!=0 ){ 47819 48337 return rc; 47820 48338 } 47821 48339 pPtrmap = (u8 *)sqlite3PagerGetData(pDbPage); 47822 48340 47823 48341 offset = PTRMAP_PTROFFSET(iPtrmap, key); 48342 + if( offset<0 ){ 48343 + sqlite3PagerUnref(pDbPage); 48344 + return SQLITE_CORRUPT_BKPT; 48345 + } 48346 + assert( offset <= (int)pBt->usableSize-5 ); 47824 48347 assert( pEType!=0 ); 47825 48348 *pEType = pPtrmap[offset]; 47826 48349 if( pPgno ) *pPgno = get4byte(&pPtrmap[offset+1]); 47827 48350 47828 48351 sqlite3PagerUnref(pDbPage); 47829 48352 if( *pEType<1 || *pEType>5 ) return SQLITE_CORRUPT_BKPT; 47830 48353 return SQLITE_OK; ................................................................................ 47841 48364 ** the page, 1 means the second cell, and so forth) return a pointer 47842 48365 ** to the cell content. 47843 48366 ** 47844 48367 ** This routine works only for pages that do not contain overflow cells. 47845 48368 */ 47846 48369 #define findCell(P,I) \ 47847 48370 ((P)->aData + ((P)->maskPage & get2byte(&(P)->aData[(P)->cellOffset+2*(I)]))) 48371 +#define findCellv2(D,M,O,I) (D+(M&get2byte(D+(O+2*(I))))) 48372 + 47848 48373 47849 48374 /* 47850 48375 ** This a more complex version of findCell() that works for 47851 48376 ** pages that do contain overflow cells. 47852 48377 */ 47853 48378 static u8 *findOverflowCell(MemPage *pPage, int iCell){ 47854 48379 int i; ................................................................................ 48678 49203 ** If the database is already opened in the same database connection 48679 49204 ** and we are in shared cache mode, then the open will fail with an 48680 49205 ** SQLITE_CONSTRAINT error. We cannot allow two or more BtShared 48681 49206 ** objects in the same database connection since doing so will lead 48682 49207 ** to problems with locking. 48683 49208 */ 48684 49209 SQLITE_PRIVATE int sqlite3BtreeOpen( 49210 + sqlite3_vfs *pVfs, /* VFS to use for this b-tree */ 48685 49211 const char *zFilename, /* Name of the file containing the BTree database */ 48686 49212 sqlite3 *db, /* Associated database handle */ 48687 49213 Btree **ppBtree, /* Pointer to new Btree object written here */ 48688 49214 int flags, /* Options */ 48689 49215 int vfsFlags /* Flags passed through to sqlite3_vfs.xOpen() */ 48690 49216 ){ 48691 - sqlite3_vfs *pVfs; /* The VFS to use for this btree */ 48692 49217 BtShared *pBt = 0; /* Shared part of btree structure */ 48693 49218 Btree *p; /* Handle to return */ 48694 49219 sqlite3_mutex *mutexOpen = 0; /* Prevents a race condition. Ticket #3537 */ 48695 49220 int rc = SQLITE_OK; /* Result code from this function */ 48696 49221 u8 nReserve; /* Byte of unused space on each page */ 48697 49222 unsigned char zDbHeader[100]; /* Database header content */ 48698 49223 ................................................................................ 48706 49231 const int isMemdb = 0; 48707 49232 #else 48708 49233 const int isMemdb = (zFilename && strcmp(zFilename, ":memory:")==0) 48709 49234 || (isTempDb && sqlite3TempInMemory(db)); 48710 49235 #endif 48711 49236 48712 49237 assert( db!=0 ); 49238 + assert( pVfs!=0 ); 48713 49239 assert( sqlite3_mutex_held(db->mutex) ); 48714 49240 assert( (flags&0xff)==flags ); /* flags fit in 8 bits */ 48715 49241 48716 49242 /* Only a BTREE_SINGLE database can be BTREE_UNORDERED */ 48717 49243 assert( (flags & BTREE_UNORDERED)==0 || (flags & BTREE_SINGLE)!=0 ); 48718 49244 48719 49245 /* A BTREE_SINGLE database is always a temporary and/or ephemeral */ ................................................................................ 48724 49250 } 48725 49251 if( isMemdb ){ 48726 49252 flags |= BTREE_MEMORY; 48727 49253 } 48728 49254 if( (vfsFlags & SQLITE_OPEN_MAIN_DB)!=0 && (isMemdb || isTempDb) ){ 48729 49255 vfsFlags = (vfsFlags & ~SQLITE_OPEN_MAIN_DB) | SQLITE_OPEN_TEMP_DB; 48730 49256 } 48731 - pVfs = db->pVfs; 48732 49257 p = sqlite3MallocZero(sizeof(Btree)); 48733 49258 if( !p ){ 48734 49259 return SQLITE_NOMEM; 48735 49260 } 48736 49261 p->inTrans = TRANS_NONE; 48737 49262 p->db = db; 48738 49263 #ifndef SQLITE_OMIT_SHARED_CACHE ................................................................................ 51435 51960 if( pCur->eState==CURSOR_INVALID ){ 51436 51961 *pRes = -1; 51437 51962 assert( pCur->apPage[pCur->iPage]->nCell==0 ); 51438 51963 return SQLITE_OK; 51439 51964 } 51440 51965 assert( pCur->apPage[0]->intKey || pIdxKey ); 51441 51966 for(;;){ 51442 - int lwr, upr; 51967 + int lwr, upr, idx; 51443 51968 Pgno chldPg; 51444 51969 MemPage *pPage = pCur->apPage[pCur->iPage]; 51445 51970 int c; 51446 51971 51447 51972 /* pPage->nCell must be greater than zero. If this is the root-page 51448 51973 ** the cursor would have been INVALID above and this for(;;) loop 51449 51974 ** not run. If this is not the root-page, then the moveToChild() routine ................................................................................ 51451 51976 ** be the right kind (index or table) of b-tree page. Otherwise 51452 51977 ** a moveToChild() or moveToRoot() call would have detected corruption. */ 51453 51978 assert( pPage->nCell>0 ); 51454 51979 assert( pPage->intKey==(pIdxKey==0) ); 51455 51980 lwr = 0; 51456 51981 upr = pPage->nCell-1; 51457 51982 if( biasRight ){ 51458 - pCur->aiIdx[pCur->iPage] = (u16)upr; 51983 + pCur->aiIdx[pCur->iPage] = (u16)(idx = upr); 51459 51984 }else{ 51460 - pCur->aiIdx[pCur->iPage] = (u16)((upr+lwr)/2); 51985 + pCur->aiIdx[pCur->iPage] = (u16)(idx = (upr+lwr)/2); 51461 51986 } 51462 51987 for(;;){ 51463 - int idx = pCur->aiIdx[pCur->iPage]; /* Index of current cell in pPage */ 51464 51988 u8 *pCell; /* Pointer to current cell in pPage */ 51465 51989 51990 + assert( idx==pCur->aiIdx[pCur->iPage] ); 51466 51991 pCur->info.nSize = 0; 51467 51992 pCell = findCell(pPage, idx) + pPage->childPtrSize; 51468 51993 if( pPage->intKey ){ 51469 51994 i64 nCellKey; 51470 51995 if( pPage->hasData ){ 51471 51996 u32 dummy; 51472 51997 pCell += getVarint32(pCell, dummy); ................................................................................ 51541 52066 lwr = idx+1; 51542 52067 }else{ 51543 52068 upr = idx-1; 51544 52069 } 51545 52070 if( lwr>upr ){ 51546 52071 break; 51547 52072 } 51548 - pCur->aiIdx[pCur->iPage] = (u16)((lwr+upr)/2); 52073 + pCur->aiIdx[pCur->iPage] = (u16)(idx = (lwr+upr)/2); 51549 52074 } 51550 52075 assert( lwr==upr+1 ); 51551 52076 assert( pPage->isInit ); 51552 52077 if( pPage->leaf ){ 51553 52078 chldPg = 0; 51554 52079 }else if( lwr>=pPage->nCell ){ 51555 52080 chldPg = get4byte(&pPage->aData[pPage->hdrOffset+8]); ................................................................................ 52374 52899 ** The cell content is not freed or deallocated. It is assumed that 52375 52900 ** the cell content has been copied someplace else. This routine just 52376 52901 ** removes the reference to the cell from pPage. 52377 52902 ** 52378 52903 ** "sz" must be the number of bytes in the cell. 52379 52904 */ 52380 52905 static void dropCell(MemPage *pPage, int idx, int sz, int *pRC){ 52381 - int i; /* Loop counter */ 52382 52906 u32 pc; /* Offset to cell content of cell being deleted */ 52383 52907 u8 *data; /* pPage->aData */ 52384 52908 u8 *ptr; /* Used to move bytes around within data[] */ 52909 + u8 *endPtr; /* End of loop */ 52385 52910 int rc; /* The return code */ 52386 52911 int hdr; /* Beginning of the header. 0 most pages. 100 page 1 */ 52387 52912 52388 52913 if( *pRC ) return; 52389 52914 52390 52915 assert( idx>=0 && idx<pPage->nCell ); 52391 52916 assert( sz==cellSize(pPage, idx) ); ................................................................................ 52402 52927 return; 52403 52928 } 52404 52929 rc = freeSpace(pPage, pc, sz); 52405 52930 if( rc ){ 52406 52931 *pRC = rc; 52407 52932 return; 52408 52933 } 52409 - for(i=idx+1; i<pPage->nCell; i++, ptr+=2){ 52410 - ptr[0] = ptr[2]; 52411 - ptr[1] = ptr[3]; 52934 + endPtr = &data[pPage->cellOffset + 2*pPage->nCell - 2]; 52935 + assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */ 52936 + while( ptr<endPtr ){ 52937 + *(u16*)ptr = *(u16*)&ptr[2]; 52938 + ptr += 2; 52412 52939 } 52413 52940 pPage->nCell--; 52414 52941 put2byte(&data[hdr+3], pPage->nCell); 52415 52942 pPage->nFree += 2; 52416 52943 } 52417 52944 52418 52945 /* ................................................................................ 52444 52971 int idx = 0; /* Where to write new cell content in data[] */ 52445 52972 int j; /* Loop counter */ 52446 52973 int end; /* First byte past the last cell pointer in data[] */ 52447 52974 int ins; /* Index in data[] where new cell pointer is inserted */ 52448 52975 int cellOffset; /* Address of first cell pointer in data[] */ 52449 52976 u8 *data; /* The content of the whole page */ 52450 52977 u8 *ptr; /* Used for moving information around in data[] */ 52978 + u8 *endPtr; /* End of the loop */ 52451 52979 52452 52980 int nSkip = (iChild ? 4 : 0); 52453 52981 52454 52982 if( *pRC ) return; 52455 52983 52456 52984 assert( i>=0 && i<=pPage->nCell+pPage->nOverflow ); 52457 52985 assert( pPage->nCell<=MX_CELL(pPage->pBt) && MX_CELL(pPage->pBt)<=10921 ); ................................................................................ 52494 53022 assert( idx+sz <= (int)pPage->pBt->usableSize ); 52495 53023 pPage->nCell++; 52496 53024 pPage->nFree -= (u16)(2 + sz); 52497 53025 memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip); 52498 53026 if( iChild ){ 52499 53027 put4byte(&data[idx], iChild); 52500 53028 } 52501 - for(j=end, ptr=&data[j]; j>ins; j-=2, ptr-=2){ 52502 - ptr[0] = ptr[-2]; 52503 - ptr[1] = ptr[-1]; 53029 + ptr = &data[end]; 53030 + endPtr = &data[ins]; 53031 + assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */ 53032 + while( ptr>endPtr ){ 53033 + *(u16*)ptr = *(u16*)&ptr[-2]; 53034 + ptr -= 2; 52504 53035 } 52505 53036 put2byte(&data[ins], idx); 52506 53037 put2byte(&data[pPage->hdrOffset+3], pPage->nCell); 52507 53038 #ifndef SQLITE_OMIT_AUTOVACUUM 52508 53039 if( pPage->pBt->autoVacuum ){ 52509 53040 /* The cell may contain a pointer to an overflow page. If so, write 52510 53041 ** the entry for the overflow page into the pointer map. ................................................................................ 52541 53072 /* Check that the page has just been zeroed by zeroPage() */ 52542 53073 assert( pPage->nCell==0 ); 52543 53074 assert( get2byteNotZero(&data[hdr+5])==nUsable ); 52544 53075 52545 53076 pCellptr = &data[pPage->cellOffset + nCell*2]; 52546 53077 cellbody = nUsable; 52547 53078 for(i=nCell-1; i>=0; i--){ 53079 + u16 sz = aSize[i]; 52548 53080 pCellptr -= 2; 52549 - cellbody -= aSize[i]; 53081 + cellbody -= sz; 52550 53082 put2byte(pCellptr, cellbody); 52551 - memcpy(&data[cellbody], apCell[i], aSize[i]); 53083 + memcpy(&data[cellbody], apCell[i], sz); 52552 53084 } 52553 53085 put2byte(&data[hdr+3], nCell); 52554 53086 put2byte(&data[hdr+5], cellbody); 52555 53087 pPage->nFree -= (nCell*2 + nUsable - cellbody); 52556 53088 pPage->nCell = (u16)nCell; 52557 53089 } 52558 53090 ................................................................................ 52998 53530 ** process of being overwritten. */ 52999 53531 MemPage *pOld = apCopy[i] = (MemPage*)&aSpace1[pBt->pageSize + k*i]; 53000 53532 memcpy(pOld, apOld[i], sizeof(MemPage)); 53001 53533 pOld->aData = (void*)&pOld[1]; 53002 53534 memcpy(pOld->aData, apOld[i]->aData, pBt->pageSize); 53003 53535 53004 53536 limit = pOld->nCell+pOld->nOverflow; 53005 - for(j=0; j<limit; j++){ 53006 - assert( nCell<nMaxCells ); 53007 - apCell[nCell] = findOverflowCell(pOld, j); 53008 - szCell[nCell] = cellSizePtr(pOld, apCell[nCell]); 53009 - nCell++; 53010 - } 53537 + if( pOld->nOverflow>0 ){ 53538 + for(j=0; j<limit; j++){ 53539 + assert( nCell<nMaxCells ); 53540 + apCell[nCell] = findOverflowCell(pOld, j); 53541 + szCell[nCell] = cellSizePtr(pOld, apCell[nCell]); 53542 + nCell++; 53543 + } 53544 + }else{ 53545 + u8 *aData = pOld->aData; 53546 + u16 maskPage = pOld->maskPage; 53547 + u16 cellOffset = pOld->cellOffset; 53548 + for(j=0; j<limit; j++){ 53549 + assert( nCell<nMaxCells ); 53550 + apCell[nCell] = findCellv2(aData, maskPage, cellOffset, j); 53551 + szCell[nCell] = cellSizePtr(pOld, apCell[nCell]); 53552 + nCell++; 53553 + } 53554 + } 53011 53555 if( i<nOld-1 && !leafData){ 53012 53556 u16 sz = (u16)szNew[i]; 53013 53557 u8 *pTemp; 53014 53558 assert( nCell<nMaxCells ); 53015 53559 szCell[nCell] = sz; 53016 53560 pTemp = &aSpace1[iSpace1]; 53017 53561 iSpace1 += sz; ................................................................................ 57152 57696 pOp->opcode = (u8)op; 57153 57697 pOp->p5 = 0; 57154 57698 pOp->p1 = p1; 57155 57699 pOp->p2 = p2; 57156 57700 pOp->p3 = p3; 57157 57701 pOp->p4.p = 0; 57158 57702 pOp->p4type = P4_NOTUSED; 57159 - p->expired = 0; 57160 - if( op==OP_ParseSchema ){ 57161 - /* Any program that uses the OP_ParseSchema opcode needs to lock 57162 - ** all btrees. */ 57163 - int j; 57164 - for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j); 57165 - } 57166 57703 #ifdef SQLITE_DEBUG 57167 57704 pOp->zComment = 0; 57168 57705 if( sqlite3VdbeAddopTrace ) sqlite3VdbePrintOp(0, i, &p->aOp[i]); 57169 57706 #endif 57170 57707 #ifdef VDBE_PROFILE 57171 57708 pOp->cycles = 0; 57172 57709 pOp->cnt = 0; ................................................................................ 57196 57733 const char *zP4, /* The P4 operand */ 57197 57734 int p4type /* P4 operand type */ 57198 57735 ){ 57199 57736 int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3); 57200 57737 sqlite3VdbeChangeP4(p, addr, zP4, p4type); 57201 57738 return addr; 57202 57739 } 57740 + 57741 +/* 57742 +** Add an OP_ParseSchema opcode. This routine is broken out from 57743 +** sqlite3VdbeAddOp4() since it needs to also local all btrees. 57744 +** 57745 +** The zWhere string must have been obtained from sqlite3_malloc(). 57746 +** This routine will take ownership of the allocated memory. 57747 +*/ 57748 +SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe *p, int iDb, char *zWhere){ 57749 + int j; 57750 + int addr = sqlite3VdbeAddOp3(p, OP_ParseSchema, iDb, 0, 0); 57751 + sqlite3VdbeChangeP4(p, addr, zWhere, P4_DYNAMIC); 57752 + for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j); 57753 +} 57203 57754 57204 57755 /* 57205 57756 ** Add an opcode that includes the p4 value as an integer. 57206 57757 */ 57207 57758 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int( 57208 57759 Vdbe *p, /* Add the opcode to this VM */ 57209 57760 int op, /* The new opcode */ ................................................................................ 58387 58938 }else{ 58388 58939 *pnByte += nByte; 58389 58940 } 58390 58941 return pBuf; 58391 58942 } 58392 58943 58393 58944 /* 58394 -** Prepare a virtual machine for execution. This involves things such 58395 -** as allocating stack space and initializing the program counter. 58396 -** After the VDBE has be prepped, it can be executed by one or more 58397 -** calls to sqlite3VdbeExec(). 58398 -** 58399 -** This is the only way to move a VDBE from VDBE_MAGIC_INIT to 58400 -** VDBE_MAGIC_RUN. 58401 -** 58402 -** This function may be called more than once on a single virtual machine. 58403 -** The first call is made while compiling the SQL statement. Subsequent 58404 -** calls are made as part of the process of resetting a statement to be 58405 -** re-executed (from a call to sqlite3_reset()). The nVar, nMem, nCursor 58406 -** and isExplain parameters are only passed correct values the first time 58407 -** the function is called. On subsequent calls, from sqlite3_reset(), nVar 58408 -** is passed -1 and nMem, nCursor and isExplain are all passed zero. 58945 +** Rewind the VDBE back to the beginning in preparation for 58946 +** running it. 58409 58947 */ 58410 -SQLITE_PRIVATE void sqlite3VdbeMakeReady( 58411 - Vdbe *p, /* The VDBE */ 58412 - int nVar, /* Number of '?' see in the SQL statement */ 58413 - int nMem, /* Number of memory cells to allocate */ 58414 - int nCursor, /* Number of cursors to allocate */ 58415 - int nArg, /* Maximum number of args in SubPrograms */ 58416 - int isExplain, /* True if the EXPLAIN keywords is present */ 58417 - int usesStmtJournal /* True to set Vdbe.usesStmtJournal */ 58418 -){ 58419 - int n; 58420 - sqlite3 *db = p->db; 58421 - 58948 +SQLITE_PRIVATE void sqlite3VdbeRewind(Vdbe *p){ 58949 +#if defined(SQLITE_DEBUG) || defined(VDBE_PROFILE) 58950 + int i; 58951 +#endif 58422 58952 assert( p!=0 ); 58423 58953 assert( p->magic==VDBE_MAGIC_INIT ); 58424 58954 58425 58955 /* There should be at least one opcode. 58426 58956 */ 58427 58957 assert( p->nOp>0 ); 58428 58958 58429 58959 /* Set the magic to VDBE_MAGIC_RUN sooner rather than later. */ 58430 58960 p->magic = VDBE_MAGIC_RUN; 58431 58961 58962 +#ifdef SQLITE_DEBUG 58963 + for(i=1; i<p->nMem; i++){ 58964 + assert( p->aMem[i].db==p->db ); 58965 + } 58966 +#endif 58967 + p->pc = -1; 58968 + p->rc = SQLITE_OK; 58969 + p->errorAction = OE_Abort; 58970 + p->magic = VDBE_MAGIC_RUN; 58971 + p->nChange = 0; 58972 + p->cacheCtr = 1; 58973 + p->minWriteFileFormat = 255; 58974 + p->iStatement = 0; 58975 + p->nFkConstraint = 0; 58976 +#ifdef VDBE_PROFILE 58977 + for(i=0; i<p->nOp; i++){ 58978 + p->aOp[i].cnt = 0; 58979 + p->aOp[i].cycles = 0; 58980 + } 58981 +#endif 58982 +} 58983 + 58984 +/* 58985 +** Prepare a virtual machine for execution for the first time after 58986 +** creating the virtual machine. This involves things such 58987 +** as allocating stack space and initializing the program counter. 58988 +** After the VDBE has be prepped, it can be executed by one or more 58989 +** calls to sqlite3VdbeExec(). 58990 +** 58991 +** This function may be called exact once on a each virtual machine. 58992 +** After this routine is called the VM has been "packaged" and is ready 58993 +** to run. After this routine is called, futher calls to 58994 +** sqlite3VdbeAddOp() functions are prohibited. This routine disconnects 58995 +** the Vdbe from the Parse object that helped generate it so that the 58996 +** the Vdbe becomes an independent entity and the Parse object can be 58997 +** destroyed. 58998 +** 58999 +** Use the sqlite3VdbeRewind() procedure to restore a virtual machine back 59000 +** to its initial state after it has been run. 59001 +*/ 59002 +SQLITE_PRIVATE void sqlite3VdbeMakeReady( 59003 + Vdbe *p, /* The VDBE */ 59004 + Parse *pParse /* Parsing context */ 59005 +){ 59006 + sqlite3 *db; /* The database connection */ 59007 + int nVar; /* Number of parameters */ 59008 + int nMem; /* Number of VM memory registers */ 59009 + int nCursor; /* Number of cursors required */ 59010 + int nArg; /* Number of arguments in subprograms */ 59011 + int n; /* Loop counter */ 59012 + u8 *zCsr; /* Memory available for allocation */ 59013 + u8 *zEnd; /* First byte past allocated memory */ 59014 + int nByte; /* How much extra memory is needed */ 59015 + 59016 + assert( p!=0 ); 59017 + assert( p->nOp>0 ); 59018 + assert( pParse!=0 ); 59019 + assert( p->magic==VDBE_MAGIC_INIT ); 59020 + db = p->db; 59021 + assert( db->mallocFailed==0 ); 59022 + nVar = pParse->nVar; 59023 + nMem = pParse->nMem; 59024 + nCursor = pParse->nTab; 59025 + nArg = pParse->nMaxArg; 59026 + 58432 59027 /* For each cursor required, also allocate a memory cell. Memory 58433 59028 ** cells (nMem+1-nCursor)..nMem, inclusive, will never be used by 58434 59029 ** the vdbe program. Instead they are used to allocate space for 58435 59030 ** VdbeCursor/BtCursor structures. The blob of memory associated with 58436 59031 ** cursor 0 is stored in memory cell nMem. Memory cell (nMem-1) 58437 59032 ** stores the blob of memory associated with cursor 1, etc. 58438 59033 ** 58439 59034 ** See also: allocateCursor(). 58440 59035 */ 58441 59036 nMem += nCursor; 58442 59037 58443 59038 /* Allocate space for memory registers, SQL variables, VDBE cursors and 58444 - ** an array to marshal SQL function arguments in. This is only done the 58445 - ** first time this function is called for a given VDBE, not when it is 58446 - ** being called from sqlite3_reset() to reset the virtual machine. 58447 - */ 58448 - if( nVar>=0 && ALWAYS(db->mallocFailed==0) ){ 58449 - u8 *zCsr = (u8 *)&p->aOp[p->nOp]; /* Memory avaliable for alloation */ 58450 - u8 *zEnd = (u8 *)&p->aOp[p->nOpAlloc]; /* First byte past available mem */ 58451 - int nByte; /* How much extra memory needed */ 58452 - 58453 - resolveP2Values(p, &nArg); 58454 - p->usesStmtJournal = (u8)usesStmtJournal; 58455 - if( isExplain && nMem<10 ){ 58456 - nMem = 10; 58457 - } 58458 - memset(zCsr, 0, zEnd-zCsr); 58459 - zCsr += (zCsr - (u8*)0)&7; 58460 - assert( EIGHT_BYTE_ALIGNMENT(zCsr) ); 58461 - 58462 - /* Memory for registers, parameters, cursor, etc, is allocated in two 58463 - ** passes. On the first pass, we try to reuse unused space at the 58464 - ** end of the opcode array. If we are unable to satisfy all memory 58465 - ** requirements by reusing the opcode array tail, then the second 58466 - ** pass will fill in the rest using a fresh allocation. 58467 - ** 58468 - ** This two-pass approach that reuses as much memory as possible from 58469 - ** the leftover space at the end of the opcode array can significantly 58470 - ** reduce the amount of memory held by a prepared statement. 58471 - */ 58472 - do { 58473 - nByte = 0; 58474 - p->aMem = allocSpace(p->aMem, nMem*sizeof(Mem), &zCsr, zEnd, &nByte); 58475 - p->aVar = allocSpace(p->aVar, nVar*sizeof(Mem), &zCsr, zEnd, &nByte); 58476 - p->apArg = allocSpace(p->apArg, nArg*sizeof(Mem*), &zCsr, zEnd, &nByte); 58477 - p->azVar = allocSpace(p->azVar, nVar*sizeof(char*), &zCsr, zEnd, &nByte); 58478 - p->apCsr = allocSpace(p->apCsr, nCursor*sizeof(VdbeCursor*), 58479 - &zCsr, zEnd, &nByte); 58480 - if( nByte ){ 58481 - p->pFree = sqlite3DbMallocZero(db, nByte); 58482 - } 58483 - zCsr = p->pFree; 58484 - zEnd = &zCsr[nByte]; 58485 - }while( nByte && !db->mallocFailed ); 58486 - 58487 - p->nCursor = (u16)nCursor; 58488 - if( p->aVar ){ 58489 - p->nVar = (ynVar)nVar; 58490 - for(n=0; n<nVar; n++){ 58491 - p->aVar[n].flags = MEM_Null; 58492 - p->aVar[n].db = db; 58493 - } 58494 - } 58495 - if( p->aMem ){ 58496 - p->aMem--; /* aMem[] goes from 1..nMem */ 58497 - p->nMem = nMem; /* not from 0..nMem-1 */ 58498 - for(n=1; n<=nMem; n++){ 58499 - p->aMem[n].flags = MEM_Null; 58500 - p->aMem[n].db = db; 58501 - } 58502 - } 58503 - } 58504 -#ifdef SQLITE_DEBUG 58505 - for(n=1; n<p->nMem; n++){ 58506 - assert( p->aMem[n].db==db ); 58507 - } 58508 -#endif 58509 - 58510 - p->pc = -1; 58511 - p->rc = SQLITE_OK; 58512 - p->errorAction = OE_Abort; 58513 - p->explain |= isExplain; 58514 - p->magic = VDBE_MAGIC_RUN; 58515 - p->nChange = 0; 58516 - p->cacheCtr = 1; 58517 - p->minWriteFileFormat = 255; 58518 - p->iStatement = 0; 58519 - p->nFkConstraint = 0; 58520 -#ifdef VDBE_PROFILE 58521 - { 58522 - int i; 58523 - for(i=0; i<p->nOp; i++){ 58524 - p->aOp[i].cnt = 0; 58525 - p->aOp[i].cycles = 0; 58526 - } 58527 - } 58528 -#endif 59039 + ** an array to marshal SQL function arguments in. 59040 + */ 59041 + zCsr = (u8*)&p->aOp[p->nOp]; /* Memory avaliable for allocation */ 59042 + zEnd = (u8*)&p->aOp[p->nOpAlloc]; /* First byte past end of zCsr[] */ 59043 + 59044 + resolveP2Values(p, &nArg); 59045 + p->usesStmtJournal = (u8)(pParse->isMultiWrite && pParse->mayAbort); 59046 + if( pParse->explain && nMem<10 ){ 59047 + nMem = 10; 59048 + } 59049 + memset(zCsr, 0, zEnd-zCsr); 59050 + zCsr += (zCsr - (u8*)0)&7; 59051 + assert( EIGHT_BYTE_ALIGNMENT(zCsr) ); 59052 + p->expired = 0; 59053 + 59054 + /* Memory for registers, parameters, cursor, etc, is allocated in two 59055 + ** passes. On the first pass, we try to reuse unused space at the 59056 + ** end of the opcode array. If we are unable to satisfy all memory 59057 + ** requirements by reusing the opcode array tail, then the second 59058 + ** pass will fill in the rest using a fresh allocation. 59059 + ** 59060 + ** This two-pass approach that reuses as much memory as possible from 59061 + ** the leftover space at the end of the opcode array can significantly 59062 + ** reduce the amount of memory held by a prepared statement. 59063 + */ 59064 + do { 59065 + nByte = 0; 59066 + p->aMem = allocSpace(p->aMem, nMem*sizeof(Mem), &zCsr, zEnd, &nByte); 59067 + p->aVar = allocSpace(p->aVar, nVar*sizeof(Mem), &zCsr, zEnd, &nByte); 59068 + p->apArg = allocSpace(p->apArg, nArg*sizeof(Mem*), &zCsr, zEnd, &nByte); 59069 + p->azVar = allocSpace(p->azVar, nVar*sizeof(char*), &zCsr, zEnd, &nByte); 59070 + p->apCsr = allocSpace(p->apCsr, nCursor*sizeof(VdbeCursor*), 59071 + &zCsr, zEnd, &nByte); 59072 + if( nByte ){ 59073 + p->pFree = sqlite3DbMallocZero(db, nByte); 59074 + } 59075 + zCsr = p->pFree; 59076 + zEnd = &zCsr[nByte]; 59077 + }while( nByte && !db->mallocFailed ); 59078 + 59079 + p->nCursor = (u16)nCursor; 59080 + if( p->aVar ){ 59081 + p->nVar = (ynVar)nVar; 59082 + for(n=0; n<nVar; n++){ 59083 + p->aVar[n].flags = MEM_Null; 59084 + p->aVar[n].db = db; 59085 + } 59086 + } 59087 + if( p->azVar ){ 59088 + p->nzVar = pParse->nzVar; 59089 + memcpy(p->azVar, pParse->azVar, p->nzVar*sizeof(p->azVar[0])); 59090 + memset(pParse->azVar, 0, pParse->nzVar*sizeof(pParse->azVar[0])); 59091 + } 59092 + if( p->aMem ){ 59093 + p->aMem--; /* aMem[] goes from 1..nMem */ 59094 + p->nMem = nMem; /* not from 0..nMem-1 */ 59095 + for(n=1; n<=nMem; n++){ 59096 + p->aMem[n].flags = MEM_Null; 59097 + p->aMem[n].db = db; 59098 + } 59099 + } 59100 + p->explain = pParse->explain; 59101 + sqlite3VdbeRewind(p); 58529 59102 } 58530 59103 58531 59104 /* 58532 59105 ** Close a VDBE cursor and release all the resources that cursor 58533 59106 ** happens to hold. 58534 59107 */ 58535 59108 SQLITE_PRIVATE void sqlite3VdbeFreeCursor(Vdbe *p, VdbeCursor *pCx){ ................................................................................ 58795 59368 u32 iRandom; 58796 59369 sqlite3DbFree(db, zMaster); 58797 59370 sqlite3_randomness(sizeof(iRandom), &iRandom); 58798 59371 zMaster = sqlite3MPrintf(db, "%s-mj%08X", zMainFile, iRandom&0x7fffffff); 58799 59372 if( !zMaster ){ 58800 59373 return SQLITE_NOMEM; 58801 59374 } 59375 + sqlite3FileSuffix3(zMainFile, zMaster); 58802 59376 rc = sqlite3OsAccess(pVfs, zMaster, SQLITE_ACCESS_EXISTS, &res); 58803 59377 }while( rc==SQLITE_OK && res ); 58804 59378 if( rc==SQLITE_OK ){ 58805 59379 /* Open the master journal. */ 58806 59380 rc = sqlite3OsOpenMalloc(pVfs, zMaster, &pMaster, 58807 59381 SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE| 58808 59382 SQLITE_OPEN_EXCLUSIVE|SQLITE_OPEN_MASTER_JOURNAL, 0 ................................................................................ 59008 59582 if( rc==SQLITE_OK ){ 59009 59583 rc = rc2; 59010 59584 } 59011 59585 } 59012 59586 } 59013 59587 db->nStatement--; 59014 59588 p->iStatement = 0; 59589 + 59590 + if( rc==SQLITE_OK ){ 59591 + if( eOp==SAVEPOINT_ROLLBACK ){ 59592 + rc = sqlite3VtabSavepoint(db, SAVEPOINT_ROLLBACK, iSavepoint); 59593 + } 59594 + if( rc==SQLITE_OK ){ 59595 + rc = sqlite3VtabSavepoint(db, SAVEPOINT_RELEASE, iSavepoint); 59596 + } 59597 + } 59015 59598 59016 59599 /* If the statement transaction is being rolled back, also restore the 59017 59600 ** database handles deferred constraint counter to the value it had when 59018 59601 ** the statement transaction was opened. */ 59019 59602 if( eOp==SAVEPOINT_ROLLBACK ){ 59020 59603 db->nDeferredCons = p->nStmtDefCons; 59021 59604 } ................................................................................ 59188 59771 } 59189 59772 59190 59773 /* If eStatementOp is non-zero, then a statement transaction needs to 59191 59774 ** be committed or rolled back. Call sqlite3VdbeCloseStatement() to 59192 59775 ** do so. If this operation returns an error, and the current statement 59193 59776 ** error code is SQLITE_OK or SQLITE_CONSTRAINT, then promote the 59194 59777 ** current statement error code. 59195 - ** 59196 - ** Note that sqlite3VdbeCloseStatement() can only fail if eStatementOp 59197 - ** is SAVEPOINT_ROLLBACK. But if p->rc==SQLITE_OK then eStatementOp 59198 - ** must be SAVEPOINT_RELEASE. Hence the NEVER(p->rc==SQLITE_OK) in 59199 - ** the following code. 59200 59778 */ 59201 59779 if( eStatementOp ){ 59202 59780 rc = sqlite3VdbeCloseStatement(p, eStatementOp); 59203 59781 if( rc ){ 59204 - assert( eStatementOp==SAVEPOINT_ROLLBACK ); 59205 - if( NEVER(p->rc==SQLITE_OK) || p->rc==SQLITE_CONSTRAINT ){ 59782 + if( p->rc==SQLITE_OK || p->rc==SQLITE_CONSTRAINT ){ 59206 59783 p->rc = rc; 59207 59784 sqlite3DbFree(db, p->zErrMsg); 59208 59785 p->zErrMsg = 0; 59209 59786 } 59210 59787 invalidateCursorsOnModifiedBtrees(db); 59211 59788 sqlite3RollbackAll(db); 59212 59789 sqlite3CloseSavepoints(db); ................................................................................ 59391 59968 ** Free all memory associated with the Vdbe passed as the second argument. 59392 59969 ** The difference between this function and sqlite3VdbeDelete() is that 59393 59970 ** VdbeDelete() also unlinks the Vdbe from the list of VMs associated with 59394 59971 ** the database connection. 59395 59972 */ 59396 59973 SQLITE_PRIVATE void sqlite3VdbeDeleteObject(sqlite3 *db, Vdbe *p){ 59397 59974 SubProgram *pSub, *pNext; 59975 + int i; 59398 59976 assert( p->db==0 || p->db==db ); 59399 59977 releaseMemArray(p->aVar, p->nVar); 59400 59978 releaseMemArray(p->aColName, p->nResColumn*COLNAME_N); 59401 59979 for(pSub=p->pProgram; pSub; pSub=pNext){ 59402 59980 pNext = pSub->pNext; 59403 59981 vdbeFreeOpArray(db, pSub->aOp, pSub->nOp); 59404 59982 sqlite3DbFree(db, pSub); 59405 59983 } 59984 + for(i=p->nzVar-1; i>=0; i--) sqlite3DbFree(db, p->azVar[i]); 59406 59985 vdbeFreeOpArray(db, p->aOp, p->nOp); 59407 59986 sqlite3DbFree(db, p->aLabel); 59408 59987 sqlite3DbFree(db, p->aColName); 59409 59988 sqlite3DbFree(db, p->zSql); 59410 59989 sqlite3DbFree(db, p->pFree); 59411 59990 sqlite3DbFree(db, p); 59412 59991 } ................................................................................ 59844 60423 u = 0; 59845 60424 while( idx<szHdr && u<p->nField && d<=nKey ){ 59846 60425 u32 serial_type; 59847 60426 59848 60427 idx += getVarint32(&aKey[idx], serial_type); 59849 60428 pMem->enc = pKeyInfo->enc; 59850 60429 pMem->db = pKeyInfo->db; 59851 - pMem->flags = 0; 60430 + /* pMem->flags = 0; // sqlite3VdbeSerialGet() will set this for us */ 59852 60431 pMem->zMalloc = 0; 59853 60432 d += sqlite3VdbeSerialGet(&aKey[d], serial_type, pMem); 59854 60433 pMem++; 59855 60434 u++; 59856 60435 } 59857 60436 assert( u<=pKeyInfo->nField + 1 ); 59858 60437 p->nField = u; ................................................................................ 59859 60438 return (void*)p; 59860 60439 } 59861 60440 59862 60441 /* 59863 60442 ** This routine destroys a UnpackedRecord object. 59864 60443 */ 59865 60444 SQLITE_PRIVATE void sqlite3VdbeDeleteUnpackedRecord(UnpackedRecord *p){ 60445 +#ifdef SQLITE_DEBUG 59866 60446 int i; 59867 60447 Mem *pMem; 59868 60448 59869 60449 assert( p!=0 ); 59870 60450 assert( p->flags & UNPACKED_NEED_DESTROY ); 59871 60451 for(i=0, pMem=p->aMem; i<p->nField; i++, pMem++){ 59872 60452 /* The unpacked record is always constructed by the 59873 60453 ** sqlite3VdbeUnpackRecord() function above, which makes all 59874 60454 ** strings and blobs static. And none of the elements are 59875 60455 ** ever transformed, so there is never anything to delete. 59876 60456 */ 59877 60457 if( NEVER(pMem->zMalloc) ) sqlite3VdbeMemRelease(pMem); 59878 60458 } 60459 +#endif 59879 60460 if( p->flags & UNPACKED_NEED_FREE ){ 59880 60461 sqlite3DbFree(p->pKeyInfo->db, p); 59881 60462 } 59882 60463 } 59883 60464 59884 60465 /* 59885 60466 ** This function compares the two table rows or index records ................................................................................ 59925 60506 mem1.enc = pKeyInfo->enc; 59926 60507 mem1.db = pKeyInfo->db; 59927 60508 /* mem1.flags = 0; // Will be initialized by sqlite3VdbeSerialGet() */ 59928 60509 VVA_ONLY( mem1.zMalloc = 0; ) /* Only needed by assert() statements */ 59929 60510 59930 60511 /* Compilers may complain that mem1.u.i is potentially uninitialized. 59931 60512 ** We could initialize it, as shown here, to silence those complaints. 59932 - ** But in fact, mem1.u.i will never actually be used initialized, and doing 60513 + ** But in fact, mem1.u.i will never actually be used uninitialized, and doing 59933 60514 ** the unnecessary initialization has a measurable negative performance 59934 60515 ** impact, since this routine is a very high runner. And so, we choose 59935 60516 ** to ignore the compiler warnings and leave this variable uninitialized. 59936 60517 */ 59937 60518 /* mem1.u.i = 0; // not needed, here to silence compiler warning */ 59938 60519 59939 60520 idx1 = getVarint32(aKey1, szHdr1); ................................................................................ 60307 60888 int rc; 60308 60889 if( pStmt==0 ){ 60309 60890 rc = SQLITE_OK; 60310 60891 }else{ 60311 60892 Vdbe *v = (Vdbe*)pStmt; 60312 60893 sqlite3_mutex_enter(v->db->mutex); 60313 60894 rc = sqlite3VdbeReset(v); 60314 - sqlite3VdbeMakeReady(v, -1, 0, 0, 0, 0, 0); 60895 + sqlite3VdbeRewind(v); 60315 60896 assert( (rc & (v->db->errMask))==rc ); 60316 60897 rc = sqlite3ApiExit(v->db, rc); 60317 60898 sqlite3_mutex_leave(v->db->mutex); 60318 60899 } 60319 60900 return rc; 60320 60901 } 60321 60902 ................................................................................ 60664 61245 ** caller. Set the error code in the database handle to the same value. 60665 61246 */ 60666 61247 rc = db->errCode = p->rc; 60667 61248 } 60668 61249 return (rc&db->errMask); 60669 61250 } 60670 61251 61252 +/* 61253 +** The maximum number of times that a statement will try to reparse 61254 +** itself before giving up and returning SQLITE_SCHEMA. 61255 +*/ 61256 +#ifndef SQLITE_MAX_SCHEMA_RETRY 61257 +# define SQLITE_MAX_SCHEMA_RETRY 5 61258 +#endif 61259 + 60671 61260 /* 60672 61261 ** This is the top-level implementation of sqlite3_step(). Call 60673 61262 ** sqlite3Step() to do most of the work. If a schema error occurs, 60674 61263 ** call sqlite3Reprepare() and try again. 60675 61264 */ 60676 61265 SQLITE_API int sqlite3_step(sqlite3_stmt *pStmt){ 60677 61266 int rc = SQLITE_OK; /* Result from sqlite3Step() */ ................................................................................ 60682 61271 60683 61272 if( vdbeSafetyNotNull(v) ){ 60684 61273 return SQLITE_MISUSE_BKPT; 60685 61274 } 60686 61275 db = v->db; 60687 61276 sqlite3_mutex_enter(db->mutex); 60688 61277 while( (rc = sqlite3Step(v))==SQLITE_SCHEMA 60689 - && cnt++ < 5 61278 + && cnt++ < SQLITE_MAX_SCHEMA_RETRY 60690 61279 && (rc2 = rc = sqlite3Reprepare(v))==SQLITE_OK ){ 60691 61280 sqlite3_reset(pStmt); 60692 61281 v->expired = 0; 60693 61282 } 60694 61283 if( rc2!=SQLITE_OK && ALWAYS(v->isPrepareV2) && ALWAYS(db->pErr) ){ 60695 61284 /* This case occurs after failing to recompile an sql statement. 60696 61285 ** The error message from the SQL compiler has already been loaded ................................................................................ 61372 61961 ** This routine is added to support DBD::SQLite. 61373 61962 */ 61374 61963 SQLITE_API int sqlite3_bind_parameter_count(sqlite3_stmt *pStmt){ 61375 61964 Vdbe *p = (Vdbe*)pStmt; 61376 61965 return p ? p->nVar : 0; 61377 61966 } 61378 61967 61379 -/* 61380 -** Create a mapping from variable numbers to variable names 61381 -** in the Vdbe.azVar[] array, if such a mapping does not already 61382 -** exist. 61383 -*/ 61384 -static void createVarMap(Vdbe *p){ 61385 - if( !p->okVar ){ 61386 - int j; 61387 - Op *pOp; 61388 - sqlite3_mutex_enter(p->db->mutex); 61389 - /* The race condition here is harmless. If two threads call this 61390 - ** routine on the same Vdbe at the same time, they both might end 61391 - ** up initializing the Vdbe.azVar[] array. That is a little extra 61392 - ** work but it results in the same answer. 61393 - */ 61394 - for(j=0, pOp=p->aOp; j<p->nOp; j++, pOp++){ 61395 - if( pOp->opcode==OP_Variable ){ 61396 - assert( pOp->p1>0 && pOp->p1<=p->nVar ); 61397 - p->azVar[pOp->p1-1] = pOp->p4.z; 61398 - } 61399 - } 61400 - p->okVar = 1; 61401 - sqlite3_mutex_leave(p->db->mutex); 61402 - } 61403 -} 61404 - 61405 61968 /* 61406 61969 ** Return the name of a wildcard parameter. Return NULL if the index 61407 61970 ** is out of range or if the wildcard is unnamed. 61408 61971 ** 61409 61972 ** The result is always UTF-8. 61410 61973 */ 61411 61974 SQLITE_API const char *sqlite3_bind_parameter_name(sqlite3_stmt *pStmt, int i){ 61412 61975 Vdbe *p = (Vdbe*)pStmt; 61413 - if( p==0 || i<1 || i>p->nVar ){ 61976 + if( p==0 || i<1 || i>p->nzVar ){ 61414 61977 return 0; 61415 61978 } 61416 - createVarMap(p); 61417 61979 return p->azVar[i-1]; 61418 61980 } 61419 61981 61420 61982 /* 61421 61983 ** Given a wildcard parameter name, return the index of the variable 61422 61984 ** with that name. If there is no variable with the given name, 61423 61985 ** return 0. 61424 61986 */ 61425 61987 SQLITE_PRIVATE int sqlite3VdbeParameterIndex(Vdbe *p, const char *zName, int nName){ 61426 61988 int i; 61427 61989 if( p==0 ){ 61428 61990 return 0; 61429 61991 } 61430 - createVarMap(p); 61431 61992 if( zName ){ 61432 - for(i=0; i<p->nVar; i++){ 61993 + for(i=0; i<p->nzVar; i++){ 61433 61994 const char *z = p->azVar[i]; 61434 61995 if( z && memcmp(z,zName,nName)==0 && z[nName]==0 ){ 61435 61996 return i+1; 61436 61997 } 61437 61998 } 61438 61999 } 61439 62000 return 0; ................................................................................ 62340 62901 Mem *aMem = p->aMem; /* Copy of p->aMem */ 62341 62902 Mem *pIn1 = 0; /* 1st input operand */ 62342 62903 Mem *pIn2 = 0; /* 2nd input operand */ 62343 62904 Mem *pIn3 = 0; /* 3rd input operand */ 62344 62905 Mem *pOut = 0; /* Output operand */ 62345 62906 int iCompare = 0; /* Result of last OP_Compare operation */ 62346 62907 int *aPermute = 0; /* Permutation of columns for OP_Compare */ 62908 + i64 lastRowid = db->lastRowid; /* Saved value of the last insert ROWID */ 62347 62909 #ifdef VDBE_PROFILE 62348 62910 u64 start; /* CPU clock count at start of opcode */ 62349 62911 int origPc; /* Program counter at start of opcode */ 62350 62912 #endif 62351 62913 /******************************************************************** 62352 62914 ** Automatically generated code 62353 62915 ** ................................................................................ 62748 63310 int i; 62749 63311 sqlite_int64 rowid; 62750 63312 Mem **apArg; 62751 63313 Mem *pX; 62752 63314 } cm; 62753 63315 struct OP_Trace_stack_vars { 62754 63316 char *zTrace; 63317 + char *z; 62755 63318 } cn; 62756 63319 } u; 62757 63320 /* End automatically generated code 62758 63321 ********************************************************************/ 62759 63322 62760 63323 assert( p->magic==VDBE_MAGIC_RUN ); /* sqlite3_step() verifies this */ 62761 63324 sqlite3VdbeEnter(p); ................................................................................ 62981 63544 REGISTER_TRACE(pOp->p1, pIn1); 62982 63545 pc = u.aa.pcDest; 62983 63546 break; 62984 63547 } 62985 63548 62986 63549 /* Opcode: HaltIfNull P1 P2 P3 P4 * 62987 63550 ** 62988 -** Check the value in register P3. If is is NULL then Halt using 63551 +** Check the value in register P3. If it is NULL then Halt using 62989 63552 ** parameter P1, P2, and P4 as if this were a Halt instruction. If the 62990 63553 ** value in register P3 is not NULL, then this routine is a no-op. 62991 63554 */ 62992 63555 case OP_HaltIfNull: { /* in3 */ 62993 63556 pIn3 = &aMem[pOp->p3]; 62994 63557 if( (pIn3->flags & MEM_Null)==0 ) break; 62995 63558 /* Fall through into OP_Halt */ ................................................................................ 63018 63581 if( pOp->p1==SQLITE_OK && p->pFrame ){ 63019 63582 /* Halt the sub-program. Return control to the parent frame. */ 63020 63583 VdbeFrame *pFrame = p->pFrame; 63021 63584 p->pFrame = pFrame->pParent; 63022 63585 p->nFrame--; 63023 63586 sqlite3VdbeSetChanges(db, p->nChange); 63024 63587 pc = sqlite3VdbeFrameRestore(pFrame); 63588 + lastRowid = db->lastRowid; 63025 63589 if( pOp->p2==OE_Ignore ){ 63026 63590 /* Instruction pc is the OP_Program that invoked the sub-program 63027 63591 ** currently being halted. If the p2 instruction of this OP_Halt 63028 63592 ** instruction is set to OE_Ignore, then the sub-program is throwing 63029 63593 ** an IGNORE exception. In this case jump to the address specified 63030 63594 ** as the p2 of the calling OP_Program. */ 63031 63595 pc = p->aOp[pc].p2-1; ................................................................................ 63173 63737 */ 63174 63738 case OP_Variable: { /* out2-prerelease */ 63175 63739 #if 0 /* local variables moved into u.ab */ 63176 63740 Mem *pVar; /* Value being transferred */ 63177 63741 #endif /* local variables moved into u.ab */ 63178 63742 63179 63743 assert( pOp->p1>0 && pOp->p1<=p->nVar ); 63744 + assert( pOp->p4.z==0 || pOp->p4.z==p->azVar[pOp->p1-1] ); 63180 63745 u.ab.pVar = &p->aVar[pOp->p1 - 1]; 63181 63746 if( sqlite3VdbeMemTooBig(u.ab.pVar) ){ 63182 63747 goto too_big; 63183 63748 } 63184 63749 sqlite3VdbeMemShallowCopy(pOut, u.ab.pVar, MEM_Static); 63185 63750 UPDATE_MAX_BLOBSIZE(pOut); 63186 63751 break; ................................................................................ 63590 64155 u.ag.ctx.isError = 0; 63591 64156 if( u.ag.ctx.pFunc->flags & SQLITE_FUNC_NEEDCOLL ){ 63592 64157 assert( pOp>aOp ); 63593 64158 assert( pOp[-1].p4type==P4_COLLSEQ ); 63594 64159 assert( pOp[-1].opcode==OP_CollSeq ); 63595 64160 u.ag.ctx.pColl = pOp[-1].p4.pColl; 63596 64161 } 64162 + db->lastRowid = lastRowid; 63597 64163 (*u.ag.ctx.pFunc->xFunc)(&u.ag.ctx, u.ag.n, u.ag.apVal); /* IMP: R-24505-23230 */ 63598 - if( db->mallocFailed ){ 63599 - /* Even though a malloc() has failed, the implementation of the 63600 - ** user function may have called an sqlite3_result_XXX() function 63601 - ** to return a value. The following call releases any resources 63602 - ** associated with such a value. 63603 - */ 63604 - sqlite3VdbeMemRelease(&u.ag.ctx.s); 63605 - goto no_mem; 63606 - } 64164 + lastRowid = db->lastRowid; 63607 64165 63608 64166 /* If any auxiliary data functions have been called by this user function, 63609 64167 ** immediately call the destructor for any non-static values. 63610 64168 */ 63611 64169 if( u.ag.ctx.pVdbeFunc ){ 63612 64170 sqlite3VdbeDeleteAuxData(u.ag.ctx.pVdbeFunc, pOp->p1); 63613 64171 pOp->p4.pVdbeFunc = u.ag.ctx.pVdbeFunc; 63614 64172 pOp->p4type = P4_VDBEFUNC; 63615 64173 } 64174 + 64175 + if( db->mallocFailed ){ 64176 + /* Even though a malloc() has failed, the implementation of the 64177 + ** user function may have called an sqlite3_result_XXX() function 64178 + ** to return a value. The following call releases any resources 64179 + ** associated with such a value. 64180 + */ 64181 + sqlite3VdbeMemRelease(&u.ag.ctx.s); 64182 + goto no_mem; 64183 + } 63616 64184 63617 64185 /* If the function returned an error, throw an exception */ 63618 64186 if( u.ag.ctx.isError ){ 63619 64187 sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.ag.ctx.s)); 63620 64188 rc = u.ag.ctx.isError; 63621 64189 } 63622 64190 ................................................................................ 63913 64481 ** This works just like the Lt opcode except that the jump is taken if 63914 64482 ** the operands in registers P1 and P3 are not equal. See the Lt opcode for 63915 64483 ** additional information. 63916 64484 ** 63917 64485 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either 63918 64486 ** true or false and is never NULL. If both operands are NULL then the result 63919 64487 ** of comparison is false. If either operand is NULL then the result is true. 63920 -** If neither operand is NULL the the result is the same as it would be if 64488 +** If neither operand is NULL the result is the same as it would be if 63921 64489 ** the SQLITE_NULLEQ flag were omitted from P5. 63922 64490 */ 63923 64491 /* Opcode: Eq P1 P2 P3 P4 P5 63924 64492 ** 63925 64493 ** This works just like the Lt opcode except that the jump is taken if 63926 64494 ** the operands in registers P1 and P3 are equal. 63927 64495 ** See the Lt opcode for additional information. 63928 64496 ** 63929 64497 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either 63930 64498 ** true or false and is never NULL. If both operands are NULL then the result 63931 64499 ** of comparison is true. If either operand is NULL then the result is false. 63932 -** If neither operand is NULL the the result is the same as it would be if 64500 +** If neither operand is NULL the result is the same as it would be if 63933 64501 ** the SQLITE_NULLEQ flag were omitted from P5. 63934 64502 */ 63935 64503 /* Opcode: Le P1 P2 P3 P4 P5 63936 64504 ** 63937 64505 ** This works just like the Lt opcode except that the jump is taken if 63938 64506 ** the content of register P3 is less than or equal to the content of 63939 64507 ** register P1. See the Lt opcode for additional information. ................................................................................ 63963 64531 u16 flags3; /* Copy of initial value of pIn3->flags */ 63964 64532 #endif /* local variables moved into u.ai */ 63965 64533 63966 64534 pIn1 = &aMem[pOp->p1]; 63967 64535 pIn3 = &aMem[pOp->p3]; 63968 64536 u.ai.flags1 = pIn1->flags; 63969 64537 u.ai.flags3 = pIn3->flags; 63970 - if( (pIn1->flags | pIn3->flags)&MEM_Null ){ 64538 + if( (u.ai.flags1 | u.ai.flags3)&MEM_Null ){ 63971 64539 /* One or both operands are NULL */ 63972 64540 if( pOp->p5 & SQLITE_NULLEQ ){ 63973 64541 /* If SQLITE_NULLEQ is set (which will only happen if the operator is 63974 64542 ** OP_Eq or OP_Ne) then take the jump or not depending on whether 63975 64543 ** or not both operands are null. 63976 64544 */ 63977 64545 assert( pOp->opcode==OP_Eq || pOp->opcode==OP_Ne ); 63978 - u.ai.res = (pIn1->flags & pIn3->flags & MEM_Null)==0; 64546 + u.ai.res = (u.ai.flags1 & u.ai.flags3 & MEM_Null)==0; 63979 64547 }else{ 63980 64548 /* SQLITE_NULLEQ is clear and at least one operand is NULL, 63981 64549 ** then the result is always NULL. 63982 64550 ** The jump is taken if the SQLITE_JUMPIFNULL bit is set. 63983 64551 */ 63984 64552 if( pOp->p5 & SQLITE_STOREP2 ){ 63985 64553 pOut = &aMem[pOp->p2]; ................................................................................ 64210 64778 sqlite3VdbeMemSetInt64(pOut, ~sqlite3VdbeIntValue(pIn1)); 64211 64779 } 64212 64780 break; 64213 64781 } 64214 64782 64215 64783 /* Opcode: If P1 P2 P3 * * 64216 64784 ** 64217 -** Jump to P2 if the value in register P1 is true. The value is 64785 +** Jump to P2 if the value in register P1 is true. The value 64218 64786 ** is considered true if it is numeric and non-zero. If the value 64219 64787 ** in P1 is NULL then take the jump if P3 is true. 64220 64788 */ 64221 64789 /* Opcode: IfNot P1 P2 P3 * * 64222 64790 ** 64223 -** Jump to P2 if the value in register P1 is False. The value is 64791 +** Jump to P2 if the value in register P1 is False. The value 64224 64792 ** is considered true if it has a numeric value of zero. If the value 64225 64793 ** in P1 is NULL then take the jump if P3 is true. 64226 64794 */ 64227 64795 case OP_If: /* jump, in1 */ 64228 64796 case OP_IfNot: { /* jump, in1 */ 64229 64797 #if 0 /* local variables moved into u.al */ 64230 64798 int c; ................................................................................ 64796 65364 ** statements (i.e. open read/write incremental blob handles). 64797 65365 */ 64798 65366 sqlite3SetString(&p->zErrMsg, db, "cannot open savepoint - " 64799 65367 "SQL statements in progress"); 64800 65368 rc = SQLITE_BUSY; 64801 65369 }else{ 64802 65370 u.aq.nName = sqlite3Strlen30(u.aq.zName); 65371 + 65372 +#ifndef SQLITE_OMIT_VIRTUALTABLE 65373 + /* This call is Ok even if this savepoint is actually a transaction 65374 + ** savepoint (and therefore should not prompt xSavepoint()) callbacks. 65375 + ** If this is a transaction savepoint being opened, it is guaranteed 65376 + ** that the db->aVTrans[] array is empty. */ 65377 + assert( db->autoCommit==0 || db->nVTrans==0 ); 65378 + rc = sqlite3VtabSavepoint(db, SAVEPOINT_BEGIN, 65379 + db->nStatement+db->nSavepoint); 65380 + if( rc!=SQLITE_OK ) goto abort_due_to_error; 65381 +#endif 64803 65382 64804 65383 /* Create a new savepoint structure. */ 64805 65384 u.aq.pNew = sqlite3DbMallocRaw(db, sizeof(Savepoint)+u.aq.nName+1); 64806 65385 if( u.aq.pNew ){ 64807 65386 u.aq.pNew->zName = (char *)&u.aq.pNew[1]; 64808 65387 memcpy(u.aq.pNew->zName, u.aq.zName, u.aq.nName+1); 64809 65388 ................................................................................ 64903 65482 sqlite3DbFree(db, u.aq.pSavepoint); 64904 65483 if( !isTransaction ){ 64905 65484 db->nSavepoint--; 64906 65485 } 64907 65486 }else{ 64908 65487 db->nDeferredCons = u.aq.pSavepoint->nDeferredCons; 64909 65488 } 65489 + 65490 + if( !isTransaction ){ 65491 + rc = sqlite3VtabSavepoint(db, u.aq.p1, u.aq.iSavepoint); 65492 + if( rc!=SQLITE_OK ) goto abort_due_to_error; 65493 + } 64910 65494 } 64911 65495 } 64912 65496 64913 65497 break; 64914 65498 } 64915 65499 64916 65500 /* Opcode: AutoCommit P1 P2 * * * ................................................................................ 65042 65626 ){ 65043 65627 assert( sqlite3BtreeIsInTrans(u.as.pBt) ); 65044 65628 if( p->iStatement==0 ){ 65045 65629 assert( db->nStatement>=0 && db->nSavepoint>=0 ); 65046 65630 db->nStatement++; 65047 65631 p->iStatement = db->nSavepoint + db->nStatement; 65048 65632 } 65049 - rc = sqlite3BtreeBeginStmt(u.as.pBt, p->iStatement); 65633 + 65634 + rc = sqlite3VtabSavepoint(db, SAVEPOINT_BEGIN, p->iStatement-1); 65635 + if( rc==SQLITE_OK ){ 65636 + rc = sqlite3BtreeBeginStmt(u.as.pBt, p->iStatement); 65637 + } 65050 65638 65051 65639 /* Store the current value of the database handles deferred constraint 65052 65640 ** counter. If the statement transaction needs to be rolled back, 65053 65641 ** the value of this counter needs to be restored too. */ 65054 65642 p->nStmtDefCons = db->nDeferredCons; 65055 65643 } 65056 65644 } ................................................................................ 65363 65951 SQLITE_OPEN_DELETEONCLOSE | 65364 65952 SQLITE_OPEN_TRANSIENT_DB; 65365 65953 65366 65954 assert( pOp->p1>=0 ); 65367 65955 u.ax.pCx = allocateCursor(p, pOp->p1, pOp->p2, -1, 1); 65368 65956 if( u.ax.pCx==0 ) goto no_mem; 65369 65957 u.ax.pCx->nullRow = 1; 65370 - rc = sqlite3BtreeOpen(0, db, &u.ax.pCx->pBt, 65958 + rc = sqlite3BtreeOpen(db->pVfs, 0, db, &u.ax.pCx->pBt, 65371 65959 BTREE_OMIT_JOURNAL | BTREE_SINGLE | pOp->p5, vfsFlags); 65372 65960 if( rc==SQLITE_OK ){ 65373 65961 rc = sqlite3BtreeBeginTrans(u.ax.pCx->pBt, 1); 65374 65962 } 65375 65963 if( rc==SQLITE_OK ){ 65376 65964 /* If a transient index is required, create it by calling 65377 65965 ** sqlite3BtreeCreateTable() with the BTREE_BLOBKEY flag before ................................................................................ 65850 66438 } 65851 66439 } 65852 66440 break; 65853 66441 } 65854 66442 65855 66443 /* Opcode: NotExists P1 P2 P3 * * 65856 66444 ** 65857 -** Use the content of register P3 as a integer key. If a record 66445 +** Use the content of register P3 as an integer key. If a record 65858 66446 ** with that key does not exist in table of P1, then jump to P2. 65859 66447 ** If the record does exist, then fall through. The cursor is left 65860 66448 ** pointing to the record if it exists. 65861 66449 ** 65862 66450 ** The difference between this operation and NotFound is that this 65863 66451 ** operation assumes the key is an integer and that P1 is a table whereas 65864 66452 ** NotFound assumes key is a blob constructed from MakeRecord and ................................................................................ 65928 66516 ** The record number is not previously used as a key in the database 65929 66517 ** table that cursor P1 points to. The new record number is written 65930 66518 ** written to register P2. 65931 66519 ** 65932 66520 ** If P3>0 then P3 is a register in the root frame of this VDBE that holds 65933 66521 ** the largest previously generated record number. No new record numbers are 65934 66522 ** allowed to be less than this value. When this value reaches its maximum, 65935 -** a SQLITE_FULL error is generated. The P3 register is updated with the ' 66523 +** an SQLITE_FULL error is generated. The P3 register is updated with the ' 65936 66524 ** generated record number. This P3 mechanism is used to help implement the 65937 66525 ** AUTOINCREMENT feature. 65938 66526 */ 65939 66527 case OP_NewRowid: { /* out2-prerelease */ 65940 66528 #if 0 /* local variables moved into u.be */ 65941 66529 i64 v; /* The new rowid */ 65942 66530 VdbeCursor *pC; /* Cursor of table to get the new rowid */ ................................................................................ 66037 66625 /* IMPLEMENTATION-OF: R-07677-41881 If the largest ROWID is equal to the 66038 66626 ** largest possible integer (9223372036854775807) then the database 66039 66627 ** engine starts picking positive candidate ROWIDs at random until 66040 66628 ** it finds one that is not previously used. */ 66041 66629 assert( pOp->p3==0 ); /* We cannot be in random rowid mode if this is 66042 66630 ** an AUTOINCREMENT table. */ 66043 66631 /* on the first attempt, simply do one more than previous */ 66044 - u.be.v = db->lastRowid; 66632 + u.be.v = lastRowid; 66045 66633 u.be.v &= (MAX_ROWID>>1); /* ensure doesn't go negative */ 66046 66634 u.be.v++; /* ensure non-zero */ 66047 66635 u.be.cnt = 0; 66048 66636 while( ((rc = sqlite3BtreeMovetoUnpacked(u.be.pC->pCursor, 0, (u64)u.be.v, 66049 66637 0, &u.be.res))==SQLITE_OK) 66050 66638 && (u.be.res==0) 66051 66639 && (++u.be.cnt<100)){ ................................................................................ 66149 66737 u.bf.iKey = u.bf.pKey->u.i; 66150 66738 }else{ 66151 66739 assert( pOp->opcode==OP_InsertInt ); 66152 66740 u.bf.iKey = pOp->p3; 66153 66741 } 66154 66742 66155 66743 if( pOp->p5 & OPFLAG_NCHANGE ) p->nChange++; 66156 - if( pOp->p5 & OPFLAG_LASTROWID ) db->lastRowid = u.bf.iKey; 66744 + if( pOp->p5 & OPFLAG_LASTROWID ) db->lastRowid = lastRowid = u.bf.iKey; 66157 66745 if( u.bf.pData->flags & MEM_Null ){ 66158 66746 u.bf.pData->z = 0; 66159 66747 u.bf.pData->n = 0; 66160 66748 }else{ 66161 66749 assert( u.bf.pData->flags & (MEM_Blob|MEM_Str) ); 66162 66750 } 66163 66751 u.bf.seekResult = ((pOp->p5 & OPFLAG_USESEEKRESULT) ? u.bf.pC->seekResult : 0); ................................................................................ 66570 67158 } 66571 67159 u.bm.pC->rowidIsValid = 0; 66572 67160 break; 66573 67161 } 66574 67162 66575 67163 /* Opcode: IdxInsert P1 P2 P3 * P5 66576 67164 ** 66577 -** Register P2 holds a SQL index key made using the 67165 +** Register P2 holds an SQL index key made using the 66578 67166 ** MakeRecord instructions. This opcode writes that key 66579 67167 ** into the index P1. Data for the entry is nil. 66580 67168 ** 66581 67169 ** P3 is a flag that provides a hint to the b-tree layer that this 66582 67170 ** insert is likely to be an append. 66583 67171 ** 66584 67172 ** This instruction only works for indices. The equivalent instruction ................................................................................ 67275 67863 assert( u.by.pProgram->nMem+u.by.pProgram->nCsr==u.by.pFrame->nChildMem ); 67276 67864 assert( u.by.pProgram->nCsr==u.by.pFrame->nChildCsr ); 67277 67865 assert( pc==u.by.pFrame->pc ); 67278 67866 } 67279 67867 67280 67868 p->nFrame++; 67281 67869 u.by.pFrame->pParent = p->pFrame; 67282 - u.by.pFrame->lastRowid = db->lastRowid; 67870 + u.by.pFrame->lastRowid = lastRowid; 67283 67871 u.by.pFrame->nChange = p->nChange; 67284 67872 p->nChange = 0; 67285 67873 p->pFrame = u.by.pFrame; 67286 67874 p->aMem = aMem = &VdbeFrameMem(u.by.pFrame)[-1]; 67287 67875 p->nMem = u.by.pFrame->nChildMem; 67288 67876 p->nCursor = (u16)u.by.pFrame->nChildCsr; 67289 67877 p->apCsr = (VdbeCursor **)&aMem[p->nMem+1]; ................................................................................ 68086 68674 int nArg; 68087 68675 int i; 68088 68676 sqlite_int64 rowid; 68089 68677 Mem **apArg; 68090 68678 Mem *pX; 68091 68679 #endif /* local variables moved into u.cm */ 68092 68680 68681 + assert( pOp->p2==1 || pOp->p5==OE_Fail || pOp->p5==OE_Rollback 68682 + || pOp->p5==OE_Abort || pOp->p5==OE_Ignore || pOp->p5==OE_Replace 68683 + ); 68093 68684 u.cm.pVtab = pOp->p4.pVtab->pVtab; 68094 68685 u.cm.pModule = (sqlite3_module *)u.cm.pVtab->pModule; 68095 68686 u.cm.nArg = pOp->p2; 68096 68687 assert( pOp->p4type==P4_VTAB ); 68097 68688 if( ALWAYS(u.cm.pModule->xUpdate) ){ 68689 + u8 vtabOnConflict = db->vtabOnConflict; 68098 68690 u.cm.apArg = p->apArg; 68099 68691 u.cm.pX = &aMem[pOp->p3]; 68100 68692 for(u.cm.i=0; u.cm.i<u.cm.nArg; u.cm.i++){ 68101 68693 assert( memIsValid(u.cm.pX) ); 68102 68694 memAboutToChange(p, u.cm.pX); 68103 68695 sqlite3VdbeMemStoreType(u.cm.pX); 68104 68696 u.cm.apArg[u.cm.i] = u.cm.pX; 68105 68697 u.cm.pX++; 68106 68698 } 68699 + db->vtabOnConflict = pOp->p5; 68107 68700 rc = u.cm.pModule->xUpdate(u.cm.pVtab, u.cm.nArg, u.cm.apArg, &u.cm.rowid); 68701 + db->vtabOnConflict = vtabOnConflict; 68108 68702 importVtabErrMsg(p, u.cm.pVtab); 68109 68703 if( rc==SQLITE_OK && pOp->p1 ){ 68110 68704 assert( u.cm.nArg>1 && u.cm.apArg[0] && (u.cm.apArg[0]->flags&MEM_Null) ); 68111 - db->lastRowid = u.cm.rowid; 68705 + db->lastRowid = lastRowid = u.cm.rowid; 68112 68706 } 68113 - p->nChange++; 68707 + if( rc==SQLITE_CONSTRAINT && pOp->p4.pVtab->bConstraint ){ 68708 + if( pOp->p5==OE_Ignore ){ 68709 + rc = SQLITE_OK; 68710 + }else{ 68711 + p->errorAction = ((pOp->p5==OE_Replace) ? OE_Abort : pOp->p5); 68712 + } 68713 + }else{ 68714 + p->nChange++; 68715 + } 68114 68716 } 68115 68717 break; 68116 68718 } 68117 68719 #endif /* SQLITE_OMIT_VIRTUALTABLE */ 68118 68720 68119 68721 #ifndef SQLITE_OMIT_PAGER_PRAGMAS 68120 68722 /* Opcode: Pagecount P1 P2 * * * ................................................................................ 68158 68760 ** 68159 68761 ** If tracing is enabled (by the sqlite3_trace()) interface, then 68160 68762 ** the UTF-8 string contained in P4 is emitted on the trace callback. 68161 68763 */ 68162 68764 case OP_Trace: { 68163 68765 #if 0 /* local variables moved into u.cn */ 68164 68766 char *zTrace; 68767 + char *z; 68165 68768 #endif /* local variables moved into u.cn */ 68166 68769 68167 - u.cn.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql); 68168 - if( u.cn.zTrace ){ 68169 - if( db->xTrace ){ 68170 - char *z = sqlite3VdbeExpandSql(p, u.cn.zTrace); 68171 - db->xTrace(db->pTraceArg, z); 68172 - sqlite3DbFree(db, z); 68173 - } 68770 + if( db->xTrace && (u.cn.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql))!=0 ){ 68771 + u.cn.z = sqlite3VdbeExpandSql(p, u.cn.zTrace); 68772 + db->xTrace(db->pTraceArg, u.cn.z); 68773 + sqlite3DbFree(db, u.cn.z); 68774 + } 68174 68775 #ifdef SQLITE_DEBUG 68175 - if( (db->flags & SQLITE_SqlTrace)!=0 ){ 68176 - sqlite3DebugPrintf("SQL-trace: %s\n", u.cn.zTrace); 68177 - } 68776 + if( (db->flags & SQLITE_SqlTrace)!=0 68777 + && (u.cn.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql))!=0 68778 + ){ 68779 + sqlite3DebugPrintf("SQL-trace: %s\n", u.cn.zTrace); 68780 + } 68178 68781 #endif /* SQLITE_DEBUG */ 68179 - } 68180 68782 break; 68181 68783 } 68182 68784 #endif 68183 68785 68184 68786 68185 68787 /* Opcode: Noop * * * * * 68186 68788 ** ................................................................................ 68256 68858 sqlite3ResetInternalSchema(db, resetSchemaOnFault-1); 68257 68859 } 68258 68860 68259 68861 /* This is the only way out of this procedure. We have to 68260 68862 ** release the mutexes on btrees that were acquired at the 68261 68863 ** top. */ 68262 68864 vdbe_return: 68865 + db->lastRowid = lastRowid; 68263 68866 sqlite3VdbeLeave(p); 68264 68867 return rc; 68265 68868 68266 68869 /* Jump to here if a string or blob larger than SQLITE_MAX_LENGTH 68267 68870 ** is encountered. 68268 68871 */ 68269 68872 too_big: ................................................................................ 68596 69199 ** always return an SQL NULL. This is useful because it means 68597 69200 ** we can invoke OP_Column to fill in the vdbe cursors type 68598 69201 ** and offset cache without causing any IO. 68599 69202 */ 68600 69203 sqlite3VdbeChangeP4(v, 3+flags, SQLITE_INT_TO_PTR(pTab->nCol+1),P4_INT32); 68601 69204 sqlite3VdbeChangeP2(v, 7, pTab->nCol); 68602 69205 if( !db->mallocFailed ){ 68603 - sqlite3VdbeMakeReady(v, 1, 1, 1, 0, 0, 0); 69206 + pParse->nVar = 1; 69207 + pParse->nMem = 1; 69208 + pParse->nTab = 1; 69209 + sqlite3VdbeMakeReady(v, pParse); 68604 69210 } 68605 69211 } 68606 69212 68607 69213 pBlob->flags = flags; 68608 69214 pBlob->iCol = iCol; 68609 69215 pBlob->db = db; 68610 69216 sqlite3BtreeLeaveAll(db); ................................................................................ 71163 71769 z = pExpr->u.zToken; 71164 71770 assert( z!=0 ); 71165 71771 assert( z[0]!=0 ); 71166 71772 if( z[1]==0 ){ 71167 71773 /* Wildcard of the form "?". Assign the next variable number */ 71168 71774 assert( z[0]=='?' ); 71169 71775 pExpr->iColumn = (ynVar)(++pParse->nVar); 71170 - }else if( z[0]=='?' ){ 71171 - /* Wildcard of the form "?nnn". Convert "nnn" to an integer and 71172 - ** use it as the variable number */ 71173 - i64 i; 71174 - int bOk = 0==sqlite3Atoi64(&z[1], &i, sqlite3Strlen30(&z[1]), SQLITE_UTF8); 71175 - pExpr->iColumn = (ynVar)i; 71176 - testcase( i==0 ); 71177 - testcase( i==1 ); 71178 - testcase( i==db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER]-1 ); 71179 - testcase( i==db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ); 71180 - if( bOk==0 || i<1 || i>db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ){ 71181 - sqlite3ErrorMsg(pParse, "variable number must be between ?1 and ?%d", 71182 - db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER]); 71183 - } 71184 - if( i>pParse->nVar ){ 71185 - pParse->nVar = (int)i; 71186 - } 71187 - }else{ 71188 - /* Wildcards like ":aaa", "$aaa" or "@aaa". Reuse the same variable 71189 - ** number as the prior appearance of the same name, or if the name 71190 - ** has never appeared before, reuse the same variable number 71191 - */ 71192 - int i; 71193 - u32 n; 71194 - n = sqlite3Strlen30(z); 71195 - for(i=0; i<pParse->nVarExpr; i++){ 71196 - Expr *pE = pParse->apVarExpr[i]; 71197 - assert( pE!=0 ); 71198 - if( memcmp(pE->u.zToken, z, n)==0 && pE->u.zToken[n]==0 ){ 71199 - pExpr->iColumn = pE->iColumn; 71200 - break; 71201 - } 71202 - } 71203 - if( i>=pParse->nVarExpr ){ 71204 - pExpr->iColumn = (ynVar)(++pParse->nVar); 71205 - if( pParse->nVarExpr>=pParse->nVarExprAlloc-1 ){ 71206 - pParse->nVarExprAlloc += pParse->nVarExprAlloc + 10; 71207 - pParse->apVarExpr = 71208 - sqlite3DbReallocOrFree( 71209 - db, 71210 - pParse->apVarExpr, 71211 - pParse->nVarExprAlloc*sizeof(pParse->apVarExpr[0]) 71212 - ); 71213 - } 71214 - if( !db->mallocFailed ){ 71215 - assert( pParse->apVarExpr!=0 ); 71216 - pParse->apVarExpr[pParse->nVarExpr++] = pExpr; 71776 + }else{ 71777 + ynVar x = 0; 71778 + u32 n = sqlite3Strlen30(z); 71779 + if( z[0]=='?' ){ 71780 + /* Wildcard of the form "?nnn". Convert "nnn" to an integer and 71781 + ** use it as the variable number */ 71782 + i64 i; 71783 + int bOk = 0==sqlite3Atoi64(&z[1], &i, n-1, SQLITE_UTF8); 71784 + pExpr->iColumn = x = (ynVar)i; 71785 + testcase( i==0 ); 71786 + testcase( i==1 ); 71787 + testcase( i==db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER]-1 ); 71788 + testcase( i==db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ); 71789 + if( bOk==0 || i<1 || i>db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ){ 71790 + sqlite3ErrorMsg(pParse, "variable number must be between ?1 and ?%d", 71791 + db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER]); 71792 + x = 0; 71793 + } 71794 + if( i>pParse->nVar ){ 71795 + pParse->nVar = (int)i; 71796 + } 71797 + }else{ 71798 + /* Wildcards like ":aaa", "$aaa" or "@aaa". Reuse the same variable 71799 + ** number as the prior appearance of the same name, or if the name 71800 + ** has never appeared before, reuse the same variable number 71801 + */ 71802 + ynVar i; 71803 + for(i=0; i<pParse->nzVar; i++){ 71804 + if( pParse->azVar[i] && memcmp(pParse->azVar[i],z,n+1)==0 ){ 71805 + pExpr->iColumn = x = (ynVar)i+1; 71806 + break; 71807 + } 71808 + } 71809 + if( x==0 ) x = pExpr->iColumn = (ynVar)(++pParse->nVar); 71810 + } 71811 + if( x>0 ){ 71812 + if( x>pParse->nzVar ){ 71813 + char **a; 71814 + a = sqlite3DbRealloc(db, pParse->azVar, x*sizeof(a[0])); 71815 + if( a==0 ) return; /* Error reported through db->mallocFailed */ 71816 + pParse->azVar = a; 71817 + memset(&a[pParse->nzVar], 0, (x-pParse->nzVar)*sizeof(a[0])); 71818 + pParse->nzVar = x; 71819 + } 71820 + if( z[0]!='?' || pParse->azVar[x-1]==0 ){ 71821 + sqlite3DbFree(db, pParse->azVar[x-1]); 71822 + pParse->azVar[x-1] = sqlite3DbStrNDup(db, z, n); 71217 71823 } 71218 71824 } 71219 71825 } 71220 71826 if( !pParse->nErr && pParse->nVar>db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ){ 71221 71827 sqlite3ErrorMsg(pParse, "too many SQL variables"); 71222 71828 } 71223 71829 } ................................................................................ 72953 73559 #endif 72954 73560 case TK_VARIABLE: { 72955 73561 assert( !ExprHasProperty(pExpr, EP_IntValue) ); 72956 73562 assert( pExpr->u.zToken!=0 ); 72957 73563 assert( pExpr->u.zToken[0]!=0 ); 72958 73564 sqlite3VdbeAddOp2(v, OP_Variable, pExpr->iColumn, target); 72959 73565 if( pExpr->u.zToken[1]!=0 ){ 72960 - sqlite3VdbeChangeP4(v, -1, pExpr->u.zToken, P4_TRANSIENT); 73566 + assert( pExpr->u.zToken[0]=='?' 73567 + || strcmp(pExpr->u.zToken, pParse->azVar[pExpr->iColumn-1])==0 ); 73568 + sqlite3VdbeChangeP4(v, -1, pParse->azVar[pExpr->iColumn-1], P4_STATIC); 72961 73569 } 72962 73570 break; 72963 73571 } 72964 73572 case TK_REGISTER: { 72965 73573 inReg = pExpr->iTable; 72966 73574 break; 72967 73575 } ................................................................................ 74723 75331 74724 75332 /* Drop the table and index from the internal schema. */ 74725 75333 sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0); 74726 75334 74727 75335 /* Reload the table, index and permanent trigger schemas. */ 74728 75336 zWhere = sqlite3MPrintf(pParse->db, "tbl_name=%Q", zName); 74729 75337 if( !zWhere ) return; 74730 - sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC); 75338 + sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere); 74731 75339 74732 75340 #ifndef SQLITE_OMIT_TRIGGER 74733 75341 /* Now, if the table is not stored in the temp database, reload any temp 74734 75342 ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined. 74735 75343 */ 74736 75344 if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){ 74737 - sqlite3VdbeAddOp4(v, OP_ParseSchema, 1, 0, 0, zWhere, P4_DYNAMIC); 75345 + sqlite3VdbeAddParseSchemaOp(v, 1, zWhere); 74738 75346 } 74739 75347 #endif 74740 75348 } 74741 75349 74742 75350 /* 74743 75351 ** Parameter zName is the name of a table that is about to be altered 74744 75352 ** (either with ALTER TABLE ... RENAME TO or ALTER TABLE ... ADD COLUMN). ................................................................................ 75984 76592 sqlite3_value **argv 75985 76593 ){ 75986 76594 int i; 75987 76595 int rc = 0; 75988 76596 sqlite3 *db = sqlite3_context_db_handle(context); 75989 76597 const char *zName; 75990 76598 const char *zFile; 76599 + char *zPath = 0; 76600 + char *zErr = 0; 76601 + unsigned int flags; 75991 76602 Db *aNew; 75992 76603 char *zErrDyn = 0; 76604 + sqlite3_vfs *pVfs; 75993 76605 75994 76606 UNUSED_PARAMETER(NotUsed); 75995 76607 75996 76608 zFile = (const char *)sqlite3_value_text(argv[0]); 75997 76609 zName = (const char *)sqlite3_value_text(argv[1]); 75998 76610 if( zFile==0 ) zFile = ""; 75999 76611 if( zName==0 ) zName = ""; ................................................................................ 76038 76650 aNew = &db->aDb[db->nDb]; 76039 76651 memset(aNew, 0, sizeof(*aNew)); 76040 76652 76041 76653 /* Open the database file. If the btree is successfully opened, use 76042 76654 ** it to obtain the database schema. At this point the schema may 76043 76655 ** or may not be initialised. 76044 76656 */ 76045 - rc = sqlite3BtreeOpen(zFile, db, &aNew->pBt, 0, 76046 - db->openFlags | SQLITE_OPEN_MAIN_DB); 76657 + flags = db->openFlags; 76658 + rc = sqlite3ParseUri(db->pVfs->zName, zFile, &flags, &pVfs, &zPath, &zErr); 76659 + if( rc!=SQLITE_OK ){ 76660 + if( rc==SQLITE_NOMEM ) db->mallocFailed = 1; 76661 + sqlite3_result_error(context, zErr, -1); 76662 + sqlite3_free(zErr); 76663 + return; 76664 + } 76665 + assert( pVfs ); 76666 + flags |= SQLITE_OPEN_MAIN_DB; 76667 + rc = sqlite3BtreeOpen(pVfs, zPath, db, &aNew->pBt, 0, flags); 76668 + sqlite3_free( zPath ); 76047 76669 db->nDb++; 76048 76670 if( rc==SQLITE_CONSTRAINT ){ 76049 76671 rc = SQLITE_ERROR; 76050 76672 zErrDyn = sqlite3MPrintf(db, "database is already attached"); 76051 76673 }else if( rc==SQLITE_OK ){ 76052 76674 Pager *pPager; 76053 76675 aNew->pSchema = sqlite3SchemaGet(db, aNew->pBt); ................................................................................ 76910 77532 FILE *trace = (db->flags & SQLITE_VdbeTrace)!=0 ? stdout : 0; 76911 77533 sqlite3VdbeTrace(v, trace); 76912 77534 #endif 76913 77535 assert( pParse->iCacheLevel==0 ); /* Disables and re-enables match */ 76914 77536 /* A minimum of one cursor is required if autoincrement is used 76915 77537 * See ticket [a696379c1f08866] */ 76916 77538 if( pParse->pAinc!=0 && pParse->nTab==0 ) pParse->nTab = 1; 76917 - sqlite3VdbeMakeReady(v, pParse->nVar, pParse->nMem, 76918 - pParse->nTab, pParse->nMaxArg, pParse->explain, 76919 - pParse->isMultiWrite && pParse->mayAbort); 77539 + sqlite3VdbeMakeReady(v, pParse); 76920 77540 pParse->rc = SQLITE_DONE; 76921 77541 pParse->colNamesSet = 0; 76922 77542 }else{ 76923 77543 pParse->rc = SQLITE_ERROR; 76924 77544 } 76925 77545 pParse->nTab = 0; 76926 77546 pParse->nMem = 0; ................................................................................ 78331 78951 pDb->zName 78332 78952 ); 78333 78953 } 78334 78954 } 78335 78955 #endif 78336 78956 78337 78957 /* Reparse everything to update our internal data structures */ 78338 - sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, 78339 - sqlite3MPrintf(db, "tbl_name='%q'",p->zName), P4_DYNAMIC); 78958 + sqlite3VdbeAddParseSchemaOp(v, iDb, 78959 + sqlite3MPrintf(db, "tbl_name='%q'", p->zName)); 78340 78960 } 78341 78961 78342 78962 78343 78963 /* Add the table to the in-memory representation of the database. 78344 78964 */ 78345 78965 if( db->init.busy ){ 78346 78966 Table *pOld; ................................................................................ 79529 80149 79530 80150 /* Fill the index with data and reparse the schema. Code an OP_Expire 79531 80151 ** to invalidate all pre-compiled statements. 79532 80152 */ 79533 80153 if( pTblName ){ 79534 80154 sqlite3RefillIndex(pParse, pIndex, iMem); 79535 80155 sqlite3ChangeCookie(pParse, iDb); 79536 - sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, 79537 - sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName), 79538 - P4_DYNAMIC); 80156 + sqlite3VdbeAddParseSchemaOp(v, iDb, 80157 + sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName)); 79539 80158 sqlite3VdbeAddOp1(v, OP_Expire, 0); 79540 80159 } 79541 80160 } 79542 80161 79543 80162 /* When adding an index to the list of indices for a table, make 79544 80163 ** sure all indices labeled OE_Replace come after all those labeled 79545 80164 ** OE_Ignore. This is necessary for the correct constraint check ................................................................................ 80153 80772 static const int flags = 80154 80773 SQLITE_OPEN_READWRITE | 80155 80774 SQLITE_OPEN_CREATE | 80156 80775 SQLITE_OPEN_EXCLUSIVE | 80157 80776 SQLITE_OPEN_DELETEONCLOSE | 80158 80777 SQLITE_OPEN_TEMP_DB; 80159 80778 80160 - rc = sqlite3BtreeOpen(0, db, &pBt, 0, flags); 80779 + rc = sqlite3BtreeOpen(db->pVfs, 0, db, &pBt, 0, flags); 80161 80780 if( rc!=SQLITE_OK ){ 80162 80781 sqlite3ErrorMsg(pParse, "unable to open a temporary database " 80163 80782 "file for storing temporary tables"); 80164 80783 pParse->rc = rc; 80165 80784 return 1; 80166 80785 } 80167 80786 db->aDb[1].pBt = pBt; ................................................................................ 81329 81948 81330 81949 /* Delete the row */ 81331 81950 #ifndef SQLITE_OMIT_VIRTUALTABLE 81332 81951 if( IsVirtual(pTab) ){ 81333 81952 const char *pVTab = (const char *)sqlite3GetVTable(db, pTab); 81334 81953 sqlite3VtabMakeWritable(pParse, pTab); 81335 81954 sqlite3VdbeAddOp4(v, OP_VUpdate, 0, 1, iRowid, pVTab, P4_VTAB); 81955 + sqlite3VdbeChangeP5(v, OE_Abort); 81336 81956 sqlite3MayAbort(pParse); 81337 81957 }else 81338 81958 #endif 81339 81959 { 81340 81960 int count = (pParse->nested==0); /* True to count changes */ 81341 81961 sqlite3GenerateRowDelete(pParse, pTab, iCur, iRowid, count, pTrigger, OE_Default); 81342 81962 } ................................................................................ 81563 82183 sqlite3VdbeAddOp2(v, OP_SCopy, regBase+nCol, regBase+j); 81564 82184 }else{ 81565 82185 sqlite3VdbeAddOp3(v, OP_Column, iCur, idx, regBase+j); 81566 82186 sqlite3ColumnDefault(v, pTab, idx, -1); 81567 82187 } 81568 82188 } 81569 82189 if( doMakeRec ){ 82190 + const char *zAff; 82191 + if( pTab->pSelect || (pParse->db->flags & SQLITE_IdxRealAsInt)!=0 ){ 82192 + zAff = 0; 82193 + }else{ 82194 + zAff = sqlite3IndexAffinityStr(v, pIdx); 82195 + } 81570 82196 sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol+1, regOut); 81571 - sqlite3VdbeChangeP4(v, -1, sqlite3IndexAffinityStr(v, pIdx), P4_TRANSIENT); 82197 + sqlite3VdbeChangeP4(v, -1, zAff, P4_TRANSIENT); 81572 82198 } 81573 82199 sqlite3ReleaseTempRange(pParse, regBase, nCol+1); 81574 82200 return regBase; 81575 82201 } 81576 82202 81577 82203 /************** End of delete.c **********************************************/ 81578 82204 /************** Begin file func.c ********************************************/ ................................................................................ 82076 82702 /* 82077 82703 ** For LIKE and GLOB matching on EBCDIC machines, assume that every 82078 82704 ** character is exactly one byte in size. Also, all characters are 82079 82705 ** able to participate in upper-case-to-lower-case mappings in EBCDIC 82080 82706 ** whereas only characters less than 0x80 do in ASCII. 82081 82707 */ 82082 82708 #if defined(SQLITE_EBCDIC) 82083 -# define sqlite3Utf8Read(A,C) (*(A++)) 82084 -# define GlogUpperToLower(A) A = sqlite3UpperToLower[A] 82709 +# define sqlite3Utf8Read(A,C) (*(A++)) 82710 +# define GlogUpperToLower(A) A = sqlite3UpperToLower[A] 82085 82711 #else 82086 -# define GlogUpperToLower(A) if( A<0x80 ){ A = sqlite3UpperToLower[A]; } 82712 +# define GlogUpperToLower(A) if( !((A)&~0x7f) ){ A = sqlite3UpperToLower[A]; } 82087 82713 #endif 82088 82714 82089 82715 static const struct compareInfo globInfo = { '*', '?', '[', 0 }; 82090 82716 /* The correct SQL-92 behavior is for the LIKE operator to ignore 82091 82717 ** case. Thus 'a' LIKE 'A' would be true. */ 82092 82718 static const struct compareInfo likeInfoNorm = { '%', '_', 0, 1 }; 82093 82719 /* If SQLITE_CASE_SENSITIVE_LIKE is defined, then the LIKE operator ................................................................................ 82122 82748 ** 82123 82749 ** abc[*]xyz Matches "abc*xyz" only 82124 82750 */ 82125 82751 static int patternCompare( 82126 82752 const u8 *zPattern, /* The glob pattern */ 82127 82753 const u8 *zString, /* The string to compare against the glob */ 82128 82754 const struct compareInfo *pInfo, /* Information about how to do the compare */ 82129 - const int esc /* The escape character */ 82755 + u32 esc /* The escape character */ 82130 82756 ){ 82131 - int c, c2; 82757 + u32 c, c2; 82132 82758 int invert; 82133 82759 int seen; 82134 82760 u8 matchOne = pInfo->matchOne; 82135 82761 u8 matchAll = pInfo->matchAll; 82136 82762 u8 matchSet = pInfo->matchSet; 82137 82763 u8 noCase = pInfo->noCase; 82138 82764 int prevEscape = 0; /* True if the previous character was 'escape' */ ................................................................................ 82178 82804 } 82179 82805 return 0; 82180 82806 }else if( !prevEscape && c==matchOne ){ 82181 82807 if( sqlite3Utf8Read(zString, &zString)==0 ){ 82182 82808 return 0; 82183 82809 } 82184 82810 }else if( c==matchSet ){ 82185 - int prior_c = 0; 82811 + u32 prior_c = 0; 82186 82812 assert( esc==0 ); /* This only occurs for GLOB, not LIKE */ 82187 82813 seen = 0; 82188 82814 invert = 0; 82189 82815 c = sqlite3Utf8Read(zString, &zString); 82190 82816 if( c==0 ) return 0; 82191 82817 c2 = sqlite3Utf8Read(zPattern, &zPattern); 82192 82818 if( c2=='^' ){ ................................................................................ 82254 82880 */ 82255 82881 static void likeFunc( 82256 82882 sqlite3_context *context, 82257 82883 int argc, 82258 82884 sqlite3_value **argv 82259 82885 ){ 82260 82886 const unsigned char *zA, *zB; 82261 - int escape = 0; 82887 + u32 escape = 0; 82262 82888 int nPat; 82263 82889 sqlite3 *db = sqlite3_context_db_handle(context); 82264 82890 82265 82891 zB = sqlite3_value_text(argv[0]); 82266 82892 zA = sqlite3_value_text(argv[1]); 82267 82893 82268 82894 /* Limit the length of the LIKE or GLOB pattern to avoid problems ................................................................................ 82343 82969 sqlite3_value **NotUsed2 82344 82970 ){ 82345 82971 UNUSED_PARAMETER2(NotUsed, NotUsed2); 82346 82972 /* IMP: R-24470-31136 This function is an SQL wrapper around the 82347 82973 ** sqlite3_sourceid() C interface. */ 82348 82974 sqlite3_result_text(context, sqlite3_sourceid(), -1, SQLITE_STATIC); 82349 82975 } 82976 + 82977 +/* 82978 +** Implementation of the sqlite_log() function. This is a wrapper around 82979 +** sqlite3_log(). The return value is NULL. The function exists purely for 82980 +** its side-effects. 82981 +*/ 82982 +static void errlogFunc( 82983 + sqlite3_context *context, 82984 + int argc, 82985 + sqlite3_value **argv 82986 +){ 82987 + UNUSED_PARAMETER(argc); 82988 + UNUSED_PARAMETER(context); 82989 + sqlite3_log(sqlite3_value_int(argv[0]), "%s", sqlite3_value_text(argv[1])); 82990 +} 82350 82991 82351 82992 /* 82352 82993 ** Implementation of the sqlite_compileoption_used() function. 82353 82994 ** The result is an integer that identifies if the compiler option 82354 82995 ** was used to build SQLite. 82355 82996 */ 82356 82997 #ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS ................................................................................ 83111 83752 /* FUNCTION(ifnull, 2, 0, 0, ifnullFunc ), */ 83112 83753 {2,SQLITE_UTF8,SQLITE_FUNC_COALESCE,0,0,ifnullFunc,0,0,"ifnull",0,0}, 83113 83754 FUNCTION(random, 0, 0, 0, randomFunc ), 83114 83755 FUNCTION(randomblob, 1, 0, 0, randomBlob ), 83115 83756 FUNCTION(nullif, 2, 0, 1, nullifFunc ), 83116 83757 FUNCTION(sqlite_version, 0, 0, 0, versionFunc ), 83117 83758 FUNCTION(sqlite_source_id, 0, 0, 0, sourceidFunc ), 83759 + FUNCTION(sqlite_log, 2, 0, 0, errlogFunc ), 83118 83760 #ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS 83119 83761 FUNCTION(sqlite_compileoption_used,1, 0, 0, compileoptionusedFunc ), 83120 83762 FUNCTION(sqlite_compileoption_get, 1, 0, 0, compileoptiongetFunc ), 83121 83763 #endif /* SQLITE_OMIT_COMPILEOPTION_DIAGS */ 83122 83764 FUNCTION(quote, 1, 0, 0, quoteFunc ), 83123 83765 FUNCTION(last_insert_rowid, 0, 0, 0, last_insert_rowid), 83124 83766 FUNCTION(changes, 0, 0, 0, changes ), ................................................................................ 83549 84191 for(i=0; i<nCol; i++){ 83550 84192 sqlite3VdbeAddOp2(v, OP_Copy, aiCol[i]+1+regData, regTemp+i); 83551 84193 } 83552 84194 83553 84195 /* If the parent table is the same as the child table, and we are about 83554 84196 ** to increment the constraint-counter (i.e. this is an INSERT operation), 83555 84197 ** then check if the row being inserted matches itself. If so, do not 83556 - ** increment the constraint-counter. */ 84198 + ** increment the constraint-counter. 84199 + ** 84200 + ** If any of the parent-key values are NULL, then the row cannot match 84201 + ** itself. So set JUMPIFNULL to make sure we do the OP_Found if any 84202 + ** of the parent-key values are NULL (at this point it is known that 84203 + ** none of the child key values are). 84204 + */ 83557 84205 if( pTab==pFKey->pFrom && nIncr==1 ){ 83558 84206 int iJump = sqlite3VdbeCurrentAddr(v) + nCol + 1; 83559 84207 for(i=0; i<nCol; i++){ 83560 84208 int iChild = aiCol[i]+1+regData; 83561 84209 int iParent = pIdx->aiColumn[i]+1+regData; 84210 + assert( aiCol[i]!=pTab->iPKey ); 84211 + if( pIdx->aiColumn[i]==pTab->iPKey ){ 84212 + /* The parent key is a composite key that includes the IPK column */ 84213 + iParent = regData; 84214 + } 83562 84215 sqlite3VdbeAddOp3(v, OP_Ne, iChild, iJump, iParent); 84216 + sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL); 83563 84217 } 83564 84218 sqlite3VdbeAddOp2(v, OP_Goto, 0, iOk); 83565 84219 } 83566 84220 83567 84221 sqlite3VdbeAddOp3(v, OP_MakeRecord, regTemp, nCol, regRec); 83568 84222 sqlite3VdbeChangeP4(v, -1, sqlite3IndexAffinityStr(v,pIdx), P4_TRANSIENT); 83569 84223 sqlite3VdbeAddOp4Int(v, OP_Found, iCur, iOk, regRec, 0); ................................................................................ 85323 85977 ** do the insertion. 85324 85978 */ 85325 85979 #ifndef SQLITE_OMIT_VIRTUALTABLE 85326 85980 if( IsVirtual(pTab) ){ 85327 85981 const char *pVTab = (const char *)sqlite3GetVTable(db, pTab); 85328 85982 sqlite3VtabMakeWritable(pParse, pTab); 85329 85983 sqlite3VdbeAddOp4(v, OP_VUpdate, 1, pTab->nCol+2, regIns, pVTab, P4_VTAB); 85984 + sqlite3VdbeChangeP5(v, onError==OE_Default ? OE_Abort : onError); 85330 85985 sqlite3MayAbort(pParse); 85331 85986 }else 85332 85987 #endif 85333 85988 { 85334 85989 int isReplace; /* Set to true if constraints may cause a replace */ 85335 85990 sqlite3GenerateConstraintChecks(pParse, pTab, baseCur, regIns, aRegIdx, 85336 85991 keyColumn>=0, 0, onError, endOfLoop, &isReplace ................................................................................ 86087 86742 return 0; /* pDestIdx has no corresponding index in pSrc */ 86088 86743 } 86089 86744 } 86090 86745 #ifndef SQLITE_OMIT_CHECK 86091 86746 if( pDest->pCheck && sqlite3ExprCompare(pSrc->pCheck, pDest->pCheck) ){ 86092 86747 return 0; /* Tables have different CHECK constraints. Ticket #2252 */ 86093 86748 } 86749 +#endif 86750 +#ifndef SQLITE_OMIT_FOREIGN_KEY 86751 + /* Disallow the transfer optimization if the destination table constains 86752 + ** any foreign key constraints. This is more restrictive than necessary. 86753 + ** But the main beneficiary of the transfer optimization is the VACUUM 86754 + ** command, and the VACUUM command disables foreign key constraints. So 86755 + ** the extra complication to make this rule less restrictive is probably 86756 + ** not worth the effort. Ticket [6284df89debdfa61db8073e062908af0c9b6118e] 86757 + */ 86758 + if( (pParse->db->flags & SQLITE_ForeignKeys)!=0 && pDest->pFKey!=0 ){ 86759 + return 0; 86760 + } 86094 86761 #endif 86095 86762 86096 86763 /* If we get this far, it means either: 86097 86764 ** 86098 86765 ** * We can always do the transfer if the table contains an 86099 86766 ** an integer primary key 86100 86767 ** ................................................................................ 87420 88087 ** May you find forgiveness for yourself and forgive others. 87421 88088 ** May you share freely, never taking more than you give. 87422 88089 ** 87423 88090 ************************************************************************* 87424 88091 ** This file contains code used to implement the PRAGMA command. 87425 88092 */ 87426 88093 87427 -/* Ignore this whole file if pragmas are disabled 87428 -*/ 87429 -#if !defined(SQLITE_OMIT_PRAGMA) 87430 - 87431 88094 /* 87432 88095 ** Interpret the given string as a safety level. Return 0 for OFF, 87433 88096 ** 1 for ON or NORMAL and 2 for FULL. Return 1 for an empty or 87434 88097 ** unrecognized string argument. 87435 88098 ** 87436 88099 ** Note that the values returned are one less that the values that 87437 88100 ** should be passed into sqlite3BtreeSetSafetyLevel(). The is done ................................................................................ 87456 88119 } 87457 88120 return 1; 87458 88121 } 87459 88122 87460 88123 /* 87461 88124 ** Interpret the given string as a boolean value. 87462 88125 */ 87463 -static u8 getBoolean(const char *z){ 88126 +SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z){ 87464 88127 return getSafetyLevel(z)&1; 87465 88128 } 88129 + 88130 +/* The sqlite3GetBoolean() function is used by other modules but the 88131 +** remainder of this file is specific to PRAGMA processing. So omit 88132 +** the rest of the file if PRAGMAs are omitted from the build. 88133 +*/ 88134 +#if !defined(SQLITE_OMIT_PRAGMA) 87466 88135 87467 88136 /* 87468 88137 ** Interpret the given string as a locking mode value. 87469 88138 */ 87470 88139 static int getLockingMode(const char *z){ 87471 88140 if( z ){ 87472 88141 if( 0==sqlite3StrICmp(z, "exclusive") ) return PAGER_LOCKINGMODE_EXCLUSIVE; ................................................................................ 87626 88295 int mask = p->mask; /* Mask of bits to set or clear. */ 87627 88296 if( db->autoCommit==0 ){ 87628 88297 /* Foreign key support may not be enabled or disabled while not 87629 88298 ** in auto-commit mode. */ 87630 88299 mask &= ~(SQLITE_ForeignKeys); 87631 88300 } 87632 88301 87633 - if( getBoolean(zRight) ){ 88302 + if( sqlite3GetBoolean(zRight) ){ 87634 88303 db->flags |= mask; 87635 88304 }else{ 87636 88305 db->flags &= ~mask; 87637 88306 } 87638 88307 87639 88308 /* Many of the flag-pragmas modify the code generated by the SQL 87640 88309 ** compiler (eg. count_changes). So add an opcode to expire all ................................................................................ 87840 88509 ** flag setting and reports thenew value. 87841 88510 */ 87842 88511 if( sqlite3StrICmp(zLeft,"secure_delete")==0 ){ 87843 88512 Btree *pBt = pDb->pBt; 87844 88513 int b = -1; 87845 88514 assert( pBt!=0 ); 87846 88515 if( zRight ){ 87847 - b = getBoolean(zRight); 88516 + b = sqlite3GetBoolean(zRight); 87848 88517 } 87849 88518 if( pId2->n==0 && b>=0 ){ 87850 88519 int ii; 87851 88520 for(ii=0; ii<db->nDb; ii++){ 87852 88521 sqlite3BtreeSecureDelete(db->aDb[ii].pBt, b); 87853 88522 } 87854 88523 } ................................................................................ 88440 89109 } 88441 89110 }else 88442 89111 #endif /* !defined(SQLITE_OMIT_FOREIGN_KEY) */ 88443 89112 88444 89113 #ifndef NDEBUG 88445 89114 if( sqlite3StrICmp(zLeft, "parser_trace")==0 ){ 88446 89115 if( zRight ){ 88447 - if( getBoolean(zRight) ){ 89116 + if( sqlite3GetBoolean(zRight) ){ 88448 89117 sqlite3ParserTrace(stderr, "parser: "); 88449 89118 }else{ 88450 89119 sqlite3ParserTrace(0, 0); 88451 89120 } 88452 89121 } 88453 89122 }else 88454 89123 #endif 88455 89124 88456 89125 /* Reinstall the LIKE and GLOB functions. The variant of LIKE 88457 89126 ** used will be case sensitive or not depending on the RHS. 88458 89127 */ 88459 89128 if( sqlite3StrICmp(zLeft, "case_sensitive_like")==0 ){ 88460 89129 if( zRight ){ 88461 - sqlite3RegisterLikeFunctions(db, getBoolean(zRight)); 89130 + sqlite3RegisterLikeFunctions(db, sqlite3GetBoolean(zRight)); 88462 89131 } 88463 89132 }else 88464 89133 88465 89134 #ifndef SQLITE_INTEGRITY_CHECK_ERROR_MAX 88466 89135 # define SQLITE_INTEGRITY_CHECK_ERROR_MAX 100 88467 89136 #endif 88468 89137 ................................................................................ 94031 94700 /* Search for the index that has the least amount of columns. If 94032 94701 ** there is such an index, and it has less columns than the table 94033 94702 ** does, then we can assume that it consumes less space on disk and 94034 94703 ** will therefore be cheaper to scan to determine the query result. 94035 94704 ** In this case set iRoot to the root page number of the index b-tree 94036 94705 ** and pKeyInfo to the KeyInfo structure required to navigate the 94037 94706 ** index. 94707 + ** 94708 + ** (2011-04-15) Do not do a full scan of an unordered index. 94038 94709 ** 94039 94710 ** In practice the KeyInfo structure will not be used. It is only 94040 94711 ** passed to keep OP_OpenRead happy. 94041 94712 */ 94042 94713 for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){ 94043 - if( !pBest || pIdx->nColumn<pBest->nColumn ){ 94714 + if( pIdx->bUnordered==0 && (!pBest || pIdx->nColumn<pBest->nColumn) ){ 94044 94715 pBest = pIdx; 94045 94716 } 94046 94717 } 94047 94718 if( pBest && pBest->nColumn<pTab->nCol ){ 94048 94719 iRoot = pBest->tnum; 94049 94720 pKeyInfo = sqlite3IndexKeyinfo(pParse, pBest); 94050 94721 } ................................................................................ 94758 95429 z = sqlite3DbStrNDup(db, (char*)pAll->z, pAll->n); 94759 95430 sqlite3NestedParse(pParse, 94760 95431 "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')", 94761 95432 db->aDb[iDb].zName, SCHEMA_TABLE(iDb), zName, 94762 95433 pTrig->table, z); 94763 95434 sqlite3DbFree(db, z); 94764 95435 sqlite3ChangeCookie(pParse, iDb); 94765 - sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, sqlite3MPrintf( 94766 - db, "type='trigger' AND name='%q'", zName), P4_DYNAMIC 94767 - ); 95436 + sqlite3VdbeAddParseSchemaOp(v, iDb, 95437 + sqlite3MPrintf(db, "type='trigger' AND name='%q'", zName)); 94768 95438 } 94769 95439 94770 95440 if( db->init.busy ){ 94771 95441 Trigger *pLink = pTrig; 94772 95442 Hash *pHash = &db->aDb[iDb].pSchema->trigHash; 94773 95443 assert( sqlite3SchemaMutexHeld(db, iDb, 0) ); 94774 95444 pTrig = sqlite3HashInsert(pHash, zName, sqlite3Strlen30(zName), pTrig); ................................................................................ 95593 96263 static void updateVirtualTable( 95594 96264 Parse *pParse, /* The parsing context */ 95595 96265 SrcList *pSrc, /* The virtual table to be modified */ 95596 96266 Table *pTab, /* The virtual table */ 95597 96267 ExprList *pChanges, /* The columns to change in the UPDATE statement */ 95598 96268 Expr *pRowidExpr, /* Expression used to recompute the rowid */ 95599 96269 int *aXRef, /* Mapping from columns of pTab to entries in pChanges */ 95600 - Expr *pWhere /* WHERE clause of the UPDATE statement */ 96270 + Expr *pWhere, /* WHERE clause of the UPDATE statement */ 96271 + int onError /* ON CONFLICT strategy */ 95601 96272 ); 95602 96273 #endif /* SQLITE_OMIT_VIRTUALTABLE */ 95603 96274 95604 96275 /* 95605 96276 ** The most recently coded instruction was an OP_Column to retrieve the 95606 96277 ** i-th column of table pTab. This routine sets the P4 parameter of the 95607 96278 ** OP_Column to the default value, if any. ................................................................................ 95813 96484 for(nIdx=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, nIdx++){} 95814 96485 if( nIdx>0 ){ 95815 96486 aRegIdx = sqlite3DbMallocRaw(db, sizeof(Index*) * nIdx ); 95816 96487 if( aRegIdx==0 ) goto update_cleanup; 95817 96488 } 95818 96489 for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){ 95819 96490 int reg; 95820 - if( chngRowid ){ 96491 + if( hasFK || chngRowid ){ 95821 96492 reg = ++pParse->nMem; 95822 96493 }else{ 95823 96494 reg = 0; 95824 96495 for(i=0; i<pIdx->nColumn; i++){ 95825 96496 if( aXRef[pIdx->aiColumn[i]]>=0 ){ 95826 96497 reg = ++pParse->nMem; 95827 96498 break; ................................................................................ 95837 96508 if( pParse->nested==0 ) sqlite3VdbeCountChanges(v); 95838 96509 sqlite3BeginWriteOperation(pParse, 1, iDb); 95839 96510 95840 96511 #ifndef SQLITE_OMIT_VIRTUALTABLE 95841 96512 /* Virtual tables must be handled separately */ 95842 96513 if( IsVirtual(pTab) ){ 95843 96514 updateVirtualTable(pParse, pTabList, pTab, pChanges, pRowidExpr, aXRef, 95844 - pWhere); 96515 + pWhere, onError); 95845 96516 pWhere = 0; 95846 96517 pTabList = 0; 95847 96518 goto update_cleanup; 95848 96519 } 95849 96520 #endif 95850 96521 95851 96522 /* Allocate required registers. */ ................................................................................ 96167 96838 static void updateVirtualTable( 96168 96839 Parse *pParse, /* The parsing context */ 96169 96840 SrcList *pSrc, /* The virtual table to be modified */ 96170 96841 Table *pTab, /* The virtual table */ 96171 96842 ExprList *pChanges, /* The columns to change in the UPDATE statement */ 96172 96843 Expr *pRowid, /* Expression used to recompute the rowid */ 96173 96844 int *aXRef, /* Mapping from columns of pTab to entries in pChanges */ 96174 - Expr *pWhere /* WHERE clause of the UPDATE statement */ 96845 + Expr *pWhere, /* WHERE clause of the UPDATE statement */ 96846 + int onError /* ON CONFLICT strategy */ 96175 96847 ){ 96176 96848 Vdbe *v = pParse->pVdbe; /* Virtual machine under construction */ 96177 96849 ExprList *pEList = 0; /* The result set of the SELECT statement */ 96178 96850 Select *pSelect = 0; /* The SELECT statement */ 96179 96851 Expr *pExpr; /* Temporary expression */ 96180 96852 int ephemTab; /* Table holding the result of the SELECT */ 96181 96853 int i; /* Loop counter */ ................................................................................ 96224 96896 sqlite3VdbeAddOp3(v, OP_Column, ephemTab, 0, iReg); 96225 96897 sqlite3VdbeAddOp3(v, OP_Column, ephemTab, (pRowid?1:0), iReg+1); 96226 96898 for(i=0; i<pTab->nCol; i++){ 96227 96899 sqlite3VdbeAddOp3(v, OP_Column, ephemTab, i+1+(pRowid!=0), iReg+2+i); 96228 96900 } 96229 96901 sqlite3VtabMakeWritable(pParse, pTab); 96230 96902 sqlite3VdbeAddOp4(v, OP_VUpdate, 0, pTab->nCol+2, iReg, pVTab, P4_VTAB); 96903 + sqlite3VdbeChangeP5(v, onError==OE_Default ? OE_Abort : onError); 96231 96904 sqlite3MayAbort(pParse); 96232 96905 sqlite3VdbeAddOp2(v, OP_Next, ephemTab, addr+1); 96233 96906 sqlite3VdbeJumpHere(v, addr); 96234 96907 sqlite3VdbeAddOp2(v, OP_Close, ephemTab, 0); 96235 96908 96236 96909 /* Cleanup */ 96237 96910 sqlite3SelectDelete(db, pSelect); ................................................................................ 96597 97270 ** May you share freely, never taking more than you give. 96598 97271 ** 96599 97272 ************************************************************************* 96600 97273 ** This file contains code used to help implement virtual tables. 96601 97274 */ 96602 97275 #ifndef SQLITE_OMIT_VIRTUALTABLE 96603 97276 97277 +/* 97278 +** Before a virtual table xCreate() or xConnect() method is invoked, the 97279 +** sqlite3.pVtabCtx member variable is set to point to an instance of 97280 +** this struct allocated on the stack. It is used by the implementation of 97281 +** the sqlite3_declare_vtab() and sqlite3_vtab_config() APIs, both of which 97282 +** are invoked only from within xCreate and xConnect methods. 97283 +*/ 97284 +struct VtabCtx { 97285 + Table *pTab; 97286 + VTable *pVTable; 97287 +}; 97288 + 96604 97289 /* 96605 97290 ** The actual function that does the work of creating a new module. 96606 97291 ** This function implements the sqlite3_create_module() and 96607 97292 ** sqlite3_create_module_v2() interfaces. 96608 97293 */ 96609 97294 static int createModule( 96610 97295 sqlite3 *db, /* Database in which module is registered */ ................................................................................ 96625 97310 memcpy(zCopy, zName, nName+1); 96626 97311 pMod->zName = zCopy; 96627 97312 pMod->pModule = pModule; 96628 97313 pMod->pAux = pAux; 96629 97314 pMod->xDestroy = xDestroy; 96630 97315 pDel = (Module *)sqlite3HashInsert(&db->aModule, zCopy, nName, (void*)pMod); 96631 97316 if( pDel && pDel->xDestroy ){ 97317 + sqlite3ResetInternalSchema(db, -1); 96632 97318 pDel->xDestroy(pDel->pAux); 96633 97319 } 96634 97320 sqlite3DbFree(db, pDel); 96635 97321 if( pDel==pMod ){ 96636 97322 db->mallocFailed = 1; 96637 97323 } 96638 - sqlite3ResetInternalSchema(db, -1); 96639 97324 }else if( xDestroy ){ 96640 97325 xDestroy(pAux); 96641 97326 } 96642 97327 rc = sqlite3ApiExit(db, SQLITE_OK); 96643 97328 sqlite3_mutex_leave(db->mutex); 96644 97329 return rc; 96645 97330 } ................................................................................ 96954 97639 ); 96955 97640 sqlite3DbFree(db, zStmt); 96956 97641 v = sqlite3GetVdbe(pParse); 96957 97642 sqlite3ChangeCookie(pParse, iDb); 96958 97643 96959 97644 sqlite3VdbeAddOp2(v, OP_Expire, 0, 0); 96960 97645 zWhere = sqlite3MPrintf(db, "name='%q' AND type='table'", pTab->zName); 96961 - sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC); 97646 + sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere); 96962 97647 sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0, 96963 97648 pTab->zName, sqlite3Strlen30(pTab->zName) + 1); 96964 97649 } 96965 97650 96966 97651 /* If we are rereading the sqlite_master table create the in-memory 96967 97652 ** record of the table. The xConnect() method is not called until 96968 97653 ** the first time the virtual table is used in an SQL statement. This ................................................................................ 97017 97702 static int vtabCallConstructor( 97018 97703 sqlite3 *db, 97019 97704 Table *pTab, 97020 97705 Module *pMod, 97021 97706 int (*xConstruct)(sqlite3*,void*,int,const char*const*,sqlite3_vtab**,char**), 97022 97707 char **pzErr 97023 97708 ){ 97709 + VtabCtx sCtx; 97024 97710 VTable *pVTable; 97025 97711 int rc; 97026 97712 const char *const*azArg = (const char *const*)pTab->azModuleArg; 97027 97713 int nArg = pTab->nModuleArg; 97028 97714 char *zErr = 0; 97029 97715 char *zModuleName = sqlite3MPrintf(db, "%s", pTab->zName); 97030 97716 ................................................................................ 97036 97722 if( !pVTable ){ 97037 97723 sqlite3DbFree(db, zModuleName); 97038 97724 return SQLITE_NOMEM; 97039 97725 } 97040 97726 pVTable->db = db; 97041 97727 pVTable->pMod = pMod; 97042 97728 97043 - assert( !db->pVTab ); 97729 + /* Invoke the virtual table constructor */ 97730 + assert( &db->pVtabCtx ); 97044 97731 assert( xConstruct ); 97045 - db->pVTab = pTab; 97046 - 97047 - /* Invoke the virtual table constructor */ 97732 + sCtx.pTab = pTab; 97733 + sCtx.pVTable = pVTable; 97734 + db->pVtabCtx = &sCtx; 97048 97735 rc = xConstruct(db, pMod->pAux, nArg, azArg, &pVTable->pVtab, &zErr); 97736 + db->pVtabCtx = 0; 97049 97737 if( rc==SQLITE_NOMEM ) db->mallocFailed = 1; 97050 97738 97051 97739 if( SQLITE_OK!=rc ){ 97052 97740 if( zErr==0 ){ 97053 97741 *pzErr = sqlite3MPrintf(db, "vtable constructor failed: %s", zModuleName); 97054 97742 }else { 97055 97743 *pzErr = sqlite3MPrintf(db, "%s", zErr); ................................................................................ 97057 97745 } 97058 97746 sqlite3DbFree(db, pVTable); 97059 97747 }else if( ALWAYS(pVTable->pVtab) ){ 97060 97748 /* Justification of ALWAYS(): A correct vtab constructor must allocate 97061 97749 ** the sqlite3_vtab object if successful. */ 97062 97750 pVTable->pVtab->pModule = pMod->pModule; 97063 97751 pVTable->nRef = 1; 97064 - if( db->pVTab ){ 97752 + if( sCtx.pTab ){ 97065 97753 const char *zFormat = "vtable constructor did not declare schema: %s"; 97066 97754 *pzErr = sqlite3MPrintf(db, zFormat, pTab->zName); 97067 97755 sqlite3VtabUnlock(pVTable); 97068 97756 rc = SQLITE_ERROR; 97069 97757 }else{ 97070 97758 int iCol; 97071 97759 /* If everything went according to plan, link the new VTable structure ................................................................................ 97105 97793 pTab->aCol[iCol].isHidden = 1; 97106 97794 } 97107 97795 } 97108 97796 } 97109 97797 } 97110 97798 97111 97799 sqlite3DbFree(db, zModuleName); 97112 - db->pVTab = 0; 97113 97800 return rc; 97114 97801 } 97115 97802 97116 97803 /* 97117 97804 ** This function is invoked by the parser to call the xConnect() method 97118 97805 ** of the virtual table pTab. If an error occurs, an error code is returned 97119 97806 ** and an error left in pParse. ................................................................................ 97146 97833 sqlite3ErrorMsg(pParse, "%s", zErr); 97147 97834 } 97148 97835 sqlite3DbFree(db, zErr); 97149 97836 } 97150 97837 97151 97838 return rc; 97152 97839 } 97153 - 97154 97840 /* 97155 -** Add the virtual table pVTab to the array sqlite3.aVTrans[]. 97841 +** Grow the db->aVTrans[] array so that there is room for at least one 97842 +** more v-table. Return SQLITE_NOMEM if a malloc fails, or SQLITE_OK otherwise. 97156 97843 */ 97157 -static int addToVTrans(sqlite3 *db, VTable *pVTab){ 97844 +static int growVTrans(sqlite3 *db){ 97158 97845 const int ARRAY_INCR = 5; 97159 97846 97160 97847 /* Grow the sqlite3.aVTrans array if required */ 97161 97848 if( (db->nVTrans%ARRAY_INCR)==0 ){ 97162 97849 VTable **aVTrans; 97163 97850 int nBytes = sizeof(sqlite3_vtab *) * (db->nVTrans + ARRAY_INCR); 97164 97851 aVTrans = sqlite3DbRealloc(db, (void *)db->aVTrans, nBytes); ................................................................................ 97165 97852 if( !aVTrans ){ 97166 97853 return SQLITE_NOMEM; 97167 97854 } 97168 97855 memset(&aVTrans[db->nVTrans], 0, sizeof(sqlite3_vtab *)*ARRAY_INCR); 97169 97856 db->aVTrans = aVTrans; 97170 97857 } 97171 97858 97859 + return SQLITE_OK; 97860 +} 97861 + 97862 +/* 97863 +** Add the virtual table pVTab to the array sqlite3.aVTrans[]. Space should 97864 +** have already been reserved using growVTrans(). 97865 +*/ 97866 +static void addToVTrans(sqlite3 *db, VTable *pVTab){ 97172 97867 /* Add pVtab to the end of sqlite3.aVTrans */ 97173 97868 db->aVTrans[db->nVTrans++] = pVTab; 97174 97869 sqlite3VtabLock(pVTab); 97175 - return SQLITE_OK; 97176 97870 } 97177 97871 97178 97872 /* 97179 97873 ** This function is invoked by the vdbe to call the xCreate method 97180 97874 ** of the virtual table named zTab in database iDb. 97181 97875 ** 97182 97876 ** If an error occurs, *pzErr is set to point an an English language ................................................................................ 97206 97900 }else{ 97207 97901 rc = vtabCallConstructor(db, pTab, pMod, pMod->pModule->xCreate, pzErr); 97208 97902 } 97209 97903 97210 97904 /* Justification of ALWAYS(): The xConstructor method is required to 97211 97905 ** create a valid sqlite3_vtab if it returns SQLITE_OK. */ 97212 97906 if( rc==SQLITE_OK && ALWAYS(sqlite3GetVTable(db, pTab)) ){ 97213 - rc = addToVTrans(db, sqlite3GetVTable(db, pTab)); 97907 + rc = growVTrans(db); 97908 + if( rc==SQLITE_OK ){ 97909 + addToVTrans(db, sqlite3GetVTable(db, pTab)); 97910 + } 97214 97911 } 97215 97912 97216 97913 return rc; 97217 97914 } 97218 97915 97219 97916 /* 97220 97917 ** This function is used to set the schema of a virtual table. It is only ................................................................................ 97225 97922 Parse *pParse; 97226 97923 97227 97924 int rc = SQLITE_OK; 97228 97925 Table *pTab; 97229 97926 char *zErr = 0; 97230 97927 97231 97928 sqlite3_mutex_enter(db->mutex); 97232 - pTab = db->pVTab; 97233 - if( !pTab ){ 97929 + if( !db->pVtabCtx || !(pTab = db->pVtabCtx->pTab) ){ 97234 97930 sqlite3Error(db, SQLITE_MISUSE, 0); 97235 97931 sqlite3_mutex_leave(db->mutex); 97236 97932 return SQLITE_MISUSE_BKPT; 97237 97933 } 97238 97934 assert( (pTab->tabFlags & TF_Virtual)!=0 ); 97239 97935 97240 97936 pParse = sqlite3StackAllocZero(db, sizeof(*pParse)); ................................................................................ 97253 97949 ){ 97254 97950 if( !pTab->aCol ){ 97255 97951 pTab->aCol = pParse->pNewTable->aCol; 97256 97952 pTab->nCol = pParse->pNewTable->nCol; 97257 97953 pParse->pNewTable->nCol = 0; 97258 97954 pParse->pNewTable->aCol = 0; 97259 97955 } 97260 - db->pVTab = 0; 97956 + db->pVtabCtx->pTab = 0; 97261 97957 }else{ 97262 97958 sqlite3Error(db, SQLITE_ERROR, (zErr ? "%s" : 0), zErr); 97263 97959 sqlite3DbFree(db, zErr); 97264 97960 rc = SQLITE_ERROR; 97265 97961 } 97266 97962 pParse->declareVtab = 0; 97267 97963 ................................................................................ 97323 98019 VTable *pVTab = db->aVTrans[i]; 97324 98020 sqlite3_vtab *p = pVTab->pVtab; 97325 98021 if( p ){ 97326 98022 int (*x)(sqlite3_vtab *); 97327 98023 x = *(int (**)(sqlite3_vtab *))((char *)p->pModule + offset); 97328 98024 if( x ) x(p); 97329 98025 } 98026 + pVTab->iSavepoint = 0; 97330 98027 sqlite3VtabUnlock(pVTab); 97331 98028 } 97332 98029 sqlite3DbFree(db, db->aVTrans); 97333 98030 db->nVTrans = 0; 97334 98031 db->aVTrans = 0; 97335 98032 } 97336 98033 } ................................................................................ 97405 98102 return SQLITE_OK; 97406 98103 } 97407 98104 pModule = pVTab->pVtab->pModule; 97408 98105 97409 98106 if( pModule->xBegin ){ 97410 98107 int i; 97411 98108 97412 - 97413 98109 /* If pVtab is already in the aVTrans array, return early */ 97414 98110 for(i=0; i<db->nVTrans; i++){ 97415 98111 if( db->aVTrans[i]==pVTab ){ 97416 98112 return SQLITE_OK; 97417 98113 } 97418 98114 } 97419 98115 97420 - /* Invoke the xBegin method */ 97421 - rc = pModule->xBegin(pVTab->pVtab); 98116 + /* Invoke the xBegin method. If successful, add the vtab to the 98117 + ** sqlite3.aVTrans[] array. */ 98118 + rc = growVTrans(db); 97422 98119 if( rc==SQLITE_OK ){ 97423 - rc = addToVTrans(db, pVTab); 98120 + rc = pModule->xBegin(pVTab->pVtab); 98121 + if( rc==SQLITE_OK ){ 98122 + addToVTrans(db, pVTab); 98123 + } 98124 + } 98125 + } 98126 + return rc; 98127 +} 98128 + 98129 +/* 98130 +** Invoke either the xSavepoint, xRollbackTo or xRelease method of all 98131 +** virtual tables that currently have an open transaction. Pass iSavepoint 98132 +** as the second argument to the virtual table method invoked. 98133 +** 98134 +** If op is SAVEPOINT_BEGIN, the xSavepoint method is invoked. If it is 98135 +** SAVEPOINT_ROLLBACK, the xRollbackTo method. Otherwise, if op is 98136 +** SAVEPOINT_RELEASE, then the xRelease method of each virtual table with 98137 +** an open transaction is invoked. 98138 +** 98139 +** If any virtual table method returns an error code other than SQLITE_OK, 98140 +** processing is abandoned and the error returned to the caller of this 98141 +** function immediately. If all calls to virtual table methods are successful, 98142 +** SQLITE_OK is returned. 98143 +*/ 98144 +SQLITE_PRIVATE int sqlite3VtabSavepoint(sqlite3 *db, int op, int iSavepoint){ 98145 + int rc = SQLITE_OK; 98146 + 98147 + assert( op==SAVEPOINT_RELEASE||op==SAVEPOINT_ROLLBACK||op==SAVEPOINT_BEGIN ); 98148 + assert( iSavepoint>=0 ); 98149 + if( db->aVTrans ){ 98150 + int i; 98151 + for(i=0; rc==SQLITE_OK && i<db->nVTrans; i++){ 98152 + VTable *pVTab = db->aVTrans[i]; 98153 + const sqlite3_module *pMod = pVTab->pMod->pModule; 98154 + if( pMod->iVersion>=2 ){ 98155 + int (*xMethod)(sqlite3_vtab *, int); 98156 + switch( op ){ 98157 + case SAVEPOINT_BEGIN: 98158 + xMethod = pMod->xSavepoint; 98159 + pVTab->iSavepoint = iSavepoint+1; 98160 + break; 98161 + case SAVEPOINT_ROLLBACK: 98162 + xMethod = pMod->xRollbackTo; 98163 + break; 98164 + default: 98165 + xMethod = pMod->xRelease; 98166 + break; 98167 + } 98168 + if( xMethod && pVTab->iSavepoint>iSavepoint ){ 98169 + rc = xMethod(db->aVTrans[i]->pVtab, iSavepoint); 98170 + } 98171 + } 97424 98172 } 97425 98173 } 97426 98174 return rc; 97427 98175 } 97428 98176 97429 98177 /* 97430 98178 ** The first parameter (pDef) is a function implementation. The ................................................................................ 97519 98267 if( apVtabLock ){ 97520 98268 pToplevel->apVtabLock = apVtabLock; 97521 98269 pToplevel->apVtabLock[pToplevel->nVtabLock++] = pTab; 97522 98270 }else{ 97523 98271 pToplevel->db->mallocFailed = 1; 97524 98272 } 97525 98273 } 98274 + 98275 +/* 98276 +** Return the ON CONFLICT resolution mode in effect for the virtual 98277 +** table update operation currently in progress. 98278 +** 98279 +** The results of this routine are undefined unless it is called from 98280 +** within an xUpdate method. 98281 +*/ 98282 +SQLITE_API int sqlite3_vtab_on_conflict(sqlite3 *db){ 98283 + static const unsigned char aMap[] = { 98284 + SQLITE_ROLLBACK, SQLITE_ABORT, SQLITE_FAIL, SQLITE_IGNORE, SQLITE_REPLACE 98285 + }; 98286 + assert( OE_Rollback==1 && OE_Abort==2 && OE_Fail==3 ); 98287 + assert( OE_Ignore==4 && OE_Replace==5 ); 98288 + assert( db->vtabOnConflict>=1 && db->vtabOnConflict<=5 ); 98289 + return (int)aMap[db->vtabOnConflict-1]; 98290 +} 98291 + 98292 +/* 98293 +** Call from within the xCreate() or xConnect() methods to provide 98294 +** the SQLite core with additional information about the behavior 98295 +** of the virtual table being implemented. 98296 +*/ 98297 +SQLITE_API int sqlite3_vtab_config(sqlite3 *db, int op, ...){ 98298 + va_list ap; 98299 + int rc = SQLITE_OK; 98300 + 98301 + sqlite3_mutex_enter(db->mutex); 98302 + 98303 + va_start(ap, op); 98304 + switch( op ){ 98305 + case SQLITE_VTAB_CONSTRAINT_SUPPORT: { 98306 + VtabCtx *p = db->pVtabCtx; 98307 + if( !p ){ 98308 + rc = SQLITE_MISUSE_BKPT; 98309 + }else{ 98310 + assert( p->pTab==0 || (p->pTab->tabFlags & TF_Virtual)!=0 ); 98311 + p->pVTable->bConstraint = (u8)va_arg(ap, int); 98312 + } 98313 + break; 98314 + } 98315 + default: 98316 + rc = SQLITE_MISUSE_BKPT; 98317 + break; 98318 + } 98319 + va_end(ap); 98320 + 98321 + if( rc!=SQLITE_OK ) sqlite3Error(db, rc, 0); 98322 + sqlite3_mutex_leave(db->mutex); 98323 + return rc; 98324 +} 97526 98325 97527 98326 #endif /* SQLITE_OMIT_VIRTUALTABLE */ 97528 98327 97529 98328 /************** End of vtab.c ************************************************/ 97530 98329 /************** Begin file where.c *******************************************/ 97531 98330 /* 97532 98331 ** 2001 September 15 ................................................................................ 106567 107366 return i; 106568 107367 } 106569 107368 #ifndef SQLITE_OMIT_BLOB_LITERAL 106570 107369 case 'x': case 'X': { 106571 107370 testcase( z[0]=='x' ); testcase( z[0]=='X' ); 106572 107371 if( z[1]=='\'' ){ 106573 107372 *tokenType = TK_BLOB; 106574 - for(i=2; (c=z[i])!=0 && c!='\''; i++){ 106575 - if( !sqlite3Isxdigit(c) ){ 106576 - *tokenType = TK_ILLEGAL; 106577 - } 107373 + for(i=2; sqlite3Isxdigit(z[i]); i++){} 107374 + if( z[i]!='\'' || i%2 ){ 107375 + *tokenType = TK_ILLEGAL; 107376 + while( z[i] && z[i]!='\'' ){ i++; } 106578 107377 } 106579 - if( i%2 || !c ) *tokenType = TK_ILLEGAL; 106580 - if( c ) i++; 107378 + if( z[i] ) i++; 106581 107379 return i; 106582 107380 } 106583 107381 /* Otherwise fall through to the next case */ 106584 107382 } 106585 107383 #endif 106586 107384 default: { 106587 107385 if( !IdChar(*z) ){ ................................................................................ 106626 107424 if( pEngine==0 ){ 106627 107425 db->mallocFailed = 1; 106628 107426 return SQLITE_NOMEM; 106629 107427 } 106630 107428 assert( pParse->pNewTable==0 ); 106631 107429 assert( pParse->pNewTrigger==0 ); 106632 107430 assert( pParse->nVar==0 ); 106633 - assert( pParse->nVarExpr==0 ); 106634 - assert( pParse->nVarExprAlloc==0 ); 106635 - assert( pParse->apVarExpr==0 ); 107431 + assert( pParse->nzVar==0 ); 107432 + assert( pParse->azVar==0 ); 106636 107433 enableLookaside = db->lookaside.bEnabled; 106637 107434 if( db->lookaside.pStart ) db->lookaside.bEnabled = 1; 106638 107435 while( !db->mallocFailed && zSql[i]!=0 ){ 106639 107436 assert( i>=0 ); 106640 107437 pParse->sLastToken.z = &zSql[i]; 106641 107438 pParse->sLastToken.n = sqlite3GetToken((unsigned char*)&zSql[i],&tokenType); 106642 107439 i += pParse->sLastToken.n; ................................................................................ 106722 107519 ** structure built up in pParse->pNewTable. The calling code (see vtab.c) 106723 107520 ** will take responsibility for freeing the Table structure. 106724 107521 */ 106725 107522 sqlite3DeleteTable(db, pParse->pNewTable); 106726 107523 } 106727 107524 106728 107525 sqlite3DeleteTrigger(db, pParse->pNewTrigger); 106729 - sqlite3DbFree(db, pParse->apVarExpr); 107526 + for(i=pParse->nzVar-1; i>=0; i--) sqlite3DbFree(db, pParse->azVar[i]); 107527 + sqlite3DbFree(db, pParse->azVar); 106730 107528 sqlite3DbFree(db, pParse->aAlias); 106731 107529 while( pParse->pAinc ){ 106732 107530 AutoincInfo *p = pParse->pAinc; 106733 107531 pParse->pAinc = p->pNext; 106734 107532 sqlite3DbFree(db, p); 106735 107533 } 106736 107534 while( pParse->pZombieTab ){ ................................................................................ 107541 108339 ** sqlite3GlobalConfig.xLog = va_arg(ap, void(*)(void*,int,const char*)); 107542 108340 */ 107543 108341 typedef void(*LOGFUNC_t)(void*,int,const char*); 107544 108342 sqlite3GlobalConfig.xLog = va_arg(ap, LOGFUNC_t); 107545 108343 sqlite3GlobalConfig.pLogArg = va_arg(ap, void*); 107546 108344 break; 107547 108345 } 108346 + 108347 + case SQLITE_CONFIG_URI: { 108348 + sqlite3GlobalConfig.bOpenUri = va_arg(ap, int); 108349 + break; 108350 + } 107548 108351 107549 108352 default: { 107550 108353 rc = SQLITE_ERROR; 107551 108354 break; 107552 108355 } 107553 108356 } 107554 108357 va_end(ap); ................................................................................ 108900 109703 if( newLimit>aHardLimit[limitId] ){ 108901 109704 newLimit = aHardLimit[limitId]; /* IMP: R-51463-25634 */ 108902 109705 } 108903 109706 db->aLimit[limitId] = newLimit; 108904 109707 } 108905 109708 return oldLimit; /* IMP: R-53341-35419 */ 108906 109709 } 109710 + 109711 +/* 109712 +** This function is used to parse both URIs and non-URI filenames passed by the 109713 +** user to API functions sqlite3_open() or sqlite3_open_v2(), and for database 109714 +** URIs specified as part of ATTACH statements. 109715 +** 109716 +** The first argument to this function is the name of the VFS to use (or 109717 +** a NULL to signify the default VFS) if the URI does not contain a "vfs=xxx" 109718 +** query parameter. The second argument contains the URI (or non-URI filename) 109719 +** itself. When this function is called the *pFlags variable should contain 109720 +** the default flags to open the database handle with. The value stored in 109721 +** *pFlags may be updated before returning if the URI filename contains 109722 +** "cache=xxx" or "mode=xxx" query parameters. 109723 +** 109724 +** If successful, SQLITE_OK is returned. In this case *ppVfs is set to point to 109725 +** the VFS that should be used to open the database file. *pzFile is set to 109726 +** point to a buffer containing the name of the file to open. It is the 109727 +** responsibility of the caller to eventually call sqlite3_free() to release 109728 +** this buffer. 109729 +** 109730 +** If an error occurs, then an SQLite error code is returned and *pzErrMsg 109731 +** may be set to point to a buffer containing an English language error 109732 +** message. It is the responsibility of the caller to eventually release 109733 +** this buffer by calling sqlite3_free(). 109734 +*/ 109735 +SQLITE_PRIVATE int sqlite3ParseUri( 109736 + const char *zDefaultVfs, /* VFS to use if no "vfs=xxx" query option */ 109737 + const char *zUri, /* Nul-terminated URI to parse */ 109738 + unsigned int *pFlags, /* IN/OUT: SQLITE_OPEN_XXX flags */ 109739 + sqlite3_vfs **ppVfs, /* OUT: VFS to use */ 109740 + char **pzFile, /* OUT: Filename component of URI */ 109741 + char **pzErrMsg /* OUT: Error message (if rc!=SQLITE_OK) */ 109742 +){ 109743 + int rc = SQLITE_OK; 109744 + unsigned int flags = *pFlags; 109745 + const char *zVfs = zDefaultVfs; 109746 + char *zFile; 109747 + char c; 109748 + int nUri = sqlite3Strlen30(zUri); 109749 + 109750 + assert( *pzErrMsg==0 ); 109751 + 109752 + if( ((flags & SQLITE_OPEN_URI) || sqlite3GlobalConfig.bOpenUri) 109753 + && nUri>=5 && memcmp(zUri, "file:", 5)==0 109754 + ){ 109755 + char *zOpt; 109756 + int eState; /* Parser state when parsing URI */ 109757 + int iIn; /* Input character index */ 109758 + int iOut = 0; /* Output character index */ 109759 + int nByte = nUri+2; /* Bytes of space to allocate */ 109760 + 109761 + /* Make sure the SQLITE_OPEN_URI flag is set to indicate to the VFS xOpen 109762 + ** method that there may be extra parameters following the file-name. */ 109763 + flags |= SQLITE_OPEN_URI; 109764 + 109765 + for(iIn=0; iIn<nUri; iIn++) nByte += (zUri[iIn]=='&'); 109766 + zFile = sqlite3_malloc(nByte); 109767 + if( !zFile ) return SQLITE_NOMEM; 109768 + 109769 + /* Discard the scheme and authority segments of the URI. */ 109770 + if( zUri[5]=='/' && zUri[6]=='/' ){ 109771 + iIn = 7; 109772 + while( zUri[iIn] && zUri[iIn]!='/' ) iIn++; 109773 + 109774 + if( iIn!=7 && (iIn!=16 || memcmp("localhost", &zUri[7], 9)) ){ 109775 + *pzErrMsg = sqlite3_mprintf("invalid uri authority: %.*s", 109776 + iIn-7, &zUri[7]); 109777 + rc = SQLITE_ERROR; 109778 + goto parse_uri_out; 109779 + } 109780 + }else{ 109781 + iIn = 5; 109782 + } 109783 + 109784 + /* Copy the filename and any query parameters into the zFile buffer. 109785 + ** Decode %HH escape codes along the way. 109786 + ** 109787 + ** Within this loop, variable eState may be set to 0, 1 or 2, depending 109788 + ** on the parsing context. As follows: 109789 + ** 109790 + ** 0: Parsing file-name. 109791 + ** 1: Parsing name section of a name=value query parameter. 109792 + ** 2: Parsing value section of a name=value query parameter. 109793 + */ 109794 + eState = 0; 109795 + while( (c = zUri[iIn])!=0 && c!='#' ){ 109796 + iIn++; 109797 + if( c=='%' 109798 + && sqlite3Isxdigit(zUri[iIn]) 109799 + && sqlite3Isxdigit(zUri[iIn+1]) 109800 + ){ 109801 + int octet = (sqlite3HexToInt(zUri[iIn++]) << 4); 109802 + octet += sqlite3HexToInt(zUri[iIn++]); 109803 + 109804 + assert( octet>=0 && octet<256 ); 109805 + if( octet==0 ){ 109806 + /* This branch is taken when "%00" appears within the URI. In this 109807 + ** case we ignore all text in the remainder of the path, name or 109808 + ** value currently being parsed. So ignore the current character 109809 + ** and skip to the next "?", "=" or "&", as appropriate. */ 109810 + while( (c = zUri[iIn])!=0 && c!='#' 109811 + && (eState!=0 || c!='?') 109812 + && (eState!=1 || (c!='=' && c!='&')) 109813 + && (eState!=2 || c!='&') 109814 + ){ 109815 + iIn++; 109816 + } 109817 + continue; 109818 + } 109819 + c = octet; 109820 + }else if( eState==1 && (c=='&' || c=='=') ){ 109821 + if( zFile[iOut-1]==0 ){ 109822 + /* An empty option name. Ignore this option altogether. */ 109823 + while( zUri[iIn] && zUri[iIn]!='#' && zUri[iIn-1]!='&' ) iIn++; 109824 + continue; 109825 + } 109826 + if( c=='&' ){ 109827 + zFile[iOut++] = '\0'; 109828 + }else{ 109829 + eState = 2; 109830 + } 109831 + c = 0; 109832 + }else if( (eState==0 && c=='?') || (eState==2 && c=='&') ){ 109833 + c = 0; 109834 + eState = 1; 109835 + } 109836 + zFile[iOut++] = c; 109837 + } 109838 + if( eState==1 ) zFile[iOut++] = '\0'; 109839 + zFile[iOut++] = '\0'; 109840 + zFile[iOut++] = '\0'; 109841 + 109842 + /* Check if there were any options specified that should be interpreted 109843 + ** here. Options that are interpreted here include "vfs" and those that 109844 + ** correspond to flags that may be passed to the sqlite3_open_v2() 109845 + ** method. */ 109846 + zOpt = &zFile[sqlite3Strlen30(zFile)+1]; 109847 + while( zOpt[0] ){ 109848 + int nOpt = sqlite3Strlen30(zOpt); 109849 + char *zVal = &zOpt[nOpt+1]; 109850 + int nVal = sqlite3Strlen30(zVal); 109851 + 109852 + if( nOpt==3 && memcmp("vfs", zOpt, 3)==0 ){ 109853 + zVfs = zVal; 109854 + }else{ 109855 + struct OpenMode { 109856 + const char *z; 109857 + int mode; 109858 + } *aMode = 0; 109859 + char *zModeType = 0; 109860 + int mask = 0; 109861 + int limit = 0; 109862 + 109863 + if( nOpt==5 && memcmp("cache", zOpt, 5)==0 ){ 109864 + static struct OpenMode aCacheMode[] = { 109865 + { "shared", SQLITE_OPEN_SHAREDCACHE }, 109866 + { "private", SQLITE_OPEN_PRIVATECACHE }, 109867 + { 0, 0 } 109868 + }; 109869 + 109870 + mask = SQLITE_OPEN_SHAREDCACHE|SQLITE_OPEN_PRIVATECACHE; 109871 + aMode = aCacheMode; 109872 + limit = mask; 109873 + zModeType = "cache"; 109874 + } 109875 + if( nOpt==4 && memcmp("mode", zOpt, 4)==0 ){ 109876 + static struct OpenMode aOpenMode[] = { 109877 + { "ro", SQLITE_OPEN_READONLY }, 109878 + { "rw", SQLITE_OPEN_READWRITE }, 109879 + { "rwc", SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE }, 109880 + { 0, 0 } 109881 + }; 109882 + 109883 + mask = SQLITE_OPEN_READONLY|SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE; 109884 + aMode = aOpenMode; 109885 + limit = mask & flags; 109886 + zModeType = "access"; 109887 + } 109888 + 109889 + if( aMode ){ 109890 + int i; 109891 + int mode = 0; 109892 + for(i=0; aMode[i].z; i++){ 109893 + const char *z = aMode[i].z; 109894 + if( nVal==sqlite3Strlen30(z) && 0==memcmp(zVal, z, nVal) ){ 109895 + mode = aMode[i].mode; 109896 + break; 109897 + } 109898 + } 109899 + if( mode==0 ){ 109900 + *pzErrMsg = sqlite3_mprintf("no such %s mode: %s", zModeType, zVal); 109901 + rc = SQLITE_ERROR; 109902 + goto parse_uri_out; 109903 + } 109904 + if( mode>limit ){ 109905 + *pzErrMsg = sqlite3_mprintf("%s mode not allowed: %s", 109906 + zModeType, zVal); 109907 + rc = SQLITE_PERM; 109908 + goto parse_uri_out; 109909 + } 109910 + flags = (flags & ~mask) | mode; 109911 + } 109912 + } 109913 + 109914 + zOpt = &zVal[nVal+1]; 109915 + } 109916 + 109917 + }else{ 109918 + zFile = sqlite3_malloc(nUri+2); 109919 + if( !zFile ) return SQLITE_NOMEM; 109920 + memcpy(zFile, zUri, nUri); 109921 + zFile[nUri] = '\0'; 109922 + zFile[nUri+1] = '\0'; 109923 + } 109924 + 109925 + *ppVfs = sqlite3_vfs_find(zVfs); 109926 + if( *ppVfs==0 ){ 109927 + *pzErrMsg = sqlite3_mprintf("no such vfs: %s", zVfs); 109928 + rc = SQLITE_ERROR; 109929 + } 109930 + parse_uri_out: 109931 + if( rc!=SQLITE_OK ){ 109932 + sqlite3_free(zFile); 109933 + zFile = 0; 109934 + } 109935 + *pFlags = flags; 109936 + *pzFile = zFile; 109937 + return rc; 109938 +} 109939 + 108907 109940 108908 109941 /* 108909 109942 ** This routine does the work of opening a database on behalf of 108910 109943 ** sqlite3_open() and sqlite3_open16(). The database filename "zFilename" 108911 109944 ** is UTF-8 encoded. 108912 109945 */ 108913 109946 static int openDatabase( 108914 109947 const char *zFilename, /* Database filename UTF-8 encoded */ 108915 109948 sqlite3 **ppDb, /* OUT: Returned database handle */ 108916 - unsigned flags, /* Operational flags */ 109949 + unsigned int flags, /* Operational flags */ 108917 109950 const char *zVfs /* Name of the VFS to use */ 108918 109951 ){ 108919 - sqlite3 *db; 108920 - int rc; 108921 - int isThreadsafe; 109952 + sqlite3 *db; /* Store allocated handle here */ 109953 + int rc; /* Return code */ 109954 + int isThreadsafe; /* True for threadsafe connections */ 109955 + char *zOpen = 0; /* Filename argument to pass to BtreeOpen() */ 109956 + char *zErrMsg = 0; /* Error message from sqlite3ParseUri() */ 108922 109957 108923 109958 *ppDb = 0; 108924 109959 #ifndef SQLITE_OMIT_AUTOINIT 108925 109960 rc = sqlite3_initialize(); 108926 109961 if( rc ) return rc; 108927 109962 #endif 108928 109963 ................................................................................ 108938 109973 */ 108939 109974 assert( SQLITE_OPEN_READONLY == 0x01 ); 108940 109975 assert( SQLITE_OPEN_READWRITE == 0x02 ); 108941 109976 assert( SQLITE_OPEN_CREATE == 0x04 ); 108942 109977 testcase( (1<<(flags&7))==0x02 ); /* READONLY */ 108943 109978 testcase( (1<<(flags&7))==0x04 ); /* READWRITE */ 108944 109979 testcase( (1<<(flags&7))==0x40 ); /* READWRITE | CREATE */ 108945 - if( ((1<<(flags&7)) & 0x46)==0 ) return SQLITE_MISUSE; 109980 + if( ((1<<(flags&7)) & 0x46)==0 ) return SQLITE_MISUSE_BKPT; 108946 109981 108947 109982 if( sqlite3GlobalConfig.bCoreMutex==0 ){ 108948 109983 isThreadsafe = 0; 108949 109984 }else if( flags & SQLITE_OPEN_NOMUTEX ){ 108950 109985 isThreadsafe = 0; 108951 109986 }else if( flags & SQLITE_OPEN_FULLMUTEX ){ 108952 109987 isThreadsafe = 1; ................................................................................ 109019 110054 #endif 109020 110055 ; 109021 110056 sqlite3HashInit(&db->aCollSeq); 109022 110057 #ifndef SQLITE_OMIT_VIRTUALTABLE 109023 110058 sqlite3HashInit(&db->aModule); 109024 110059 #endif 109025 110060 109026 - db->pVfs = sqlite3_vfs_find(zVfs); 109027 - if( !db->pVfs ){ 109028 - rc = SQLITE_ERROR; 109029 - sqlite3Error(db, rc, "no such vfs: %s", zVfs); 109030 - goto opendb_out; 109031 - } 109032 - 109033 110061 /* Add the default collation sequence BINARY. BINARY works for both UTF-8 109034 110062 ** and UTF-16, so add a version for each to avoid any unnecessary 109035 110063 ** conversions. The only error that can occur here is a malloc() failure. 109036 110064 */ 109037 110065 createCollation(db, "BINARY", SQLITE_UTF8, SQLITE_COLL_BINARY, 0, 109038 110066 binCollFunc, 0); 109039 110067 createCollation(db, "BINARY", SQLITE_UTF16BE, SQLITE_COLL_BINARY, 0, ................................................................................ 109048 110076 db->pDfltColl = sqlite3FindCollSeq(db, SQLITE_UTF8, "BINARY", 0); 109049 110077 assert( db->pDfltColl!=0 ); 109050 110078 109051 110079 /* Also add a UTF-8 case-insensitive collation sequence. */ 109052 110080 createCollation(db, "NOCASE", SQLITE_UTF8, SQLITE_COLL_NOCASE, 0, 109053 110081 nocaseCollatingFunc, 0); 109054 110082 109055 - /* Open the backend database driver */ 110083 + /* Parse the filename/URI argument. */ 109056 110084 db->openFlags = flags; 109057 - rc = sqlite3BtreeOpen(zFilename, db, &db->aDb[0].pBt, 0, 110085 + rc = sqlite3ParseUri(zVfs, zFilename, &flags, &db->pVfs, &zOpen, &zErrMsg); 110086 + if( rc!=SQLITE_OK ){ 110087 + if( rc==SQLITE_NOMEM ) db->mallocFailed = 1; 110088 + sqlite3Error(db, rc, zErrMsg ? "%s" : 0, zErrMsg); 110089 + sqlite3_free(zErrMsg); 110090 + goto opendb_out; 110091 + } 110092 + 110093 + /* Open the backend database driver */ 110094 + rc = sqlite3BtreeOpen(db->pVfs, zOpen, db, &db->aDb[0].pBt, 0, 109058 110095 flags | SQLITE_OPEN_MAIN_DB); 109059 110096 if( rc!=SQLITE_OK ){ 109060 110097 if( rc==SQLITE_IOERR_NOMEM ){ 109061 110098 rc = SQLITE_NOMEM; 109062 110099 } 109063 110100 sqlite3Error(db, rc, 0); 109064 110101 goto opendb_out; ................................................................................ 109143 110180 /* Enable the lookaside-malloc subsystem */ 109144 110181 setupLookaside(db, 0, sqlite3GlobalConfig.szLookaside, 109145 110182 sqlite3GlobalConfig.nLookaside); 109146 110183 109147 110184 sqlite3_wal_autocheckpoint(db, SQLITE_DEFAULT_WAL_AUTOCHECKPOINT); 109148 110185 109149 110186 opendb_out: 110187 + sqlite3_free(zOpen); 109150 110188 if( db ){ 109151 110189 assert( db->mutex!=0 || isThreadsafe==0 || sqlite3GlobalConfig.bFullMutex==0 ); 109152 110190 sqlite3_mutex_leave(db->mutex); 109153 110191 } 109154 110192 rc = sqlite3_errcode(db); 109155 110193 if( rc==SQLITE_NOMEM ){ 109156 110194 sqlite3_close(db); ................................................................................ 109174 110212 } 109175 110213 SQLITE_API int sqlite3_open_v2( 109176 110214 const char *filename, /* Database filename (UTF-8) */ 109177 110215 sqlite3 **ppDb, /* OUT: SQLite db handle */ 109178 110216 int flags, /* Flags */ 109179 110217 const char *zVfs /* Name of VFS module to use */ 109180 110218 ){ 109181 - return openDatabase(filename, ppDb, flags, zVfs); 110219 + return openDatabase(filename, ppDb, (unsigned int)flags, zVfs); 109182 110220 } 109183 110221 109184 110222 #ifndef SQLITE_OMIT_UTF16 109185 110223 /* 109186 110224 ** Open a new database handle. 109187 110225 */ 109188 110226 SQLITE_API int sqlite3_open16( ................................................................................ 109779 110817 ppNew = va_arg(ap, void**); 109780 110818 pFree = va_arg(ap, void*); 109781 110819 if( sz ) *ppNew = sqlite3ScratchMalloc(sz); 109782 110820 sqlite3ScratchFree(pFree); 109783 110821 break; 109784 110822 } 109785 110823 110824 + /* sqlite3_test_control(SQLITE_TESTCTRL_LOCALTIME_FAULT, int onoff); 110825 + ** 110826 + ** If parameter onoff is non-zero, configure the wrappers so that all 110827 + ** subsequent calls to localtime() and variants fail. If onoff is zero, 110828 + ** undo this setting. 110829 + */ 110830 + case SQLITE_TESTCTRL_LOCALTIME_FAULT: { 110831 + sqlite3GlobalConfig.bLocaltimeFault = va_arg(ap, int); 110832 + break; 110833 + } 110834 + 109786 110835 } 109787 110836 va_end(ap); 109788 110837 #endif /* SQLITE_OMIT_BUILTIN_TEST */ 109789 110838 return rc; 109790 110839 } 110840 + 110841 +/* 110842 +** This is a utility routine, useful to VFS implementations, that checks 110843 +** to see if a database file was a URI that contained a specific query 110844 +** parameter, and if so obtains the value of the query parameter. 110845 +** 110846 +** The zFilename argument is the filename pointer passed into the xOpen() 110847 +** method of a VFS implementation. The zParam argument is the name of the 110848 +** query parameter we seek. This routine returns the value of the zParam 110849 +** parameter if it exists. If the parameter does not exist, this routine 110850 +** returns a NULL pointer. 110851 +*/ 110852 +SQLITE_API const char *sqlite3_uri_parameter(const char *zFilename, const char *zParam){ 110853 + zFilename += sqlite3Strlen30(zFilename) + 1; 110854 + while( zFilename[0] ){ 110855 + int x = strcmp(zFilename, zParam); 110856 + zFilename += sqlite3Strlen30(zFilename) + 1; 110857 + if( x==0 ) return zFilename; 110858 + zFilename += sqlite3Strlen30(zFilename) + 1; 110859 + } 110860 + return 0; 110861 +} 109791 110862 109792 110863 /************** End of main.c ************************************************/ 109793 110864 /************** Begin file notify.c ******************************************/ 109794 110865 /* 109795 110866 ** 2009 March 3 109796 110867 ** 109797 110868 ** The author disclaims copyright to this source code. In place of ................................................................................ 110414 111485 ** older data. 110415 111486 ** 110416 111487 ** TODO(shess) Provide a VACUUM type operation to clear out all 110417 111488 ** deletions and duplications. This would basically be a forced merge 110418 111489 ** into a single segment. 110419 111490 */ 110420 111491 110421 -#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) 110422 - 110423 -#if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE) 110424 -# define SQLITE_CORE 1 110425 -#endif 110426 - 110427 111492 /************** Include fts3Int.h in the middle of fts3.c ********************/ 110428 111493 /************** Begin file fts3Int.h *****************************************/ 110429 111494 /* 110430 111495 ** 2009 Nov 12 110431 111496 ** 110432 111497 ** The author disclaims copyright to this source code. In place of 110433 111498 ** a legal notice, here is a blessing: ................................................................................ 110435 111500 ** May you do good and not evil. 110436 111501 ** May you find forgiveness for yourself and forgive others. 110437 111502 ** May you share freely, never taking more than you give. 110438 111503 ** 110439 111504 ****************************************************************************** 110440 111505 ** 110441 111506 */ 110442 - 110443 111507 #ifndef _FTSINT_H 110444 111508 #define _FTSINT_H 110445 111509 110446 111510 #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) 110447 111511 # define NDEBUG 1 110448 111512 #endif 110449 111513 111514 +/* 111515 +** FTS4 is really an extension for FTS3. It is enabled using the 111516 +** SQLITE_ENABLE_FTS3 macro. But to avoid confusion we also all 111517 +** the SQLITE_ENABLE_FTS4 macro to serve as an alisse for SQLITE_ENABLE_FTS3. 111518 +*/ 111519 +#if defined(SQLITE_ENABLE_FTS4) && !defined(SQLITE_ENABLE_FTS3) 111520 +# define SQLITE_ENABLE_FTS3 111521 +#endif 111522 + 111523 +#ifdef SQLITE_ENABLE_FTS3 110450 111524 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/ 110451 111525 /************** Begin file fts3_tokenizer.h **********************************/ 110452 111526 /* 110453 111527 ** 2006 July 10 110454 111528 ** 110455 111529 ** The author disclaims copyright to this source code. 110456 111530 ** ................................................................................ 110741 111815 /* 110742 111816 ** Macro to return the number of elements in an array. SQLite has a 110743 111817 ** similar macro called ArraySize(). Use a different name to avoid 110744 111818 ** a collision when building an amalgamation with built-in FTS3. 110745 111819 */ 110746 111820 #define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0]))) 110747 111821 111822 + 111823 +#ifndef MIN 111824 +# define MIN(x,y) ((x)<(y)?(x):(y)) 111825 +#endif 111826 + 110748 111827 /* 110749 111828 ** Maximum length of a varint encoded integer. The varint format is different 110750 111829 ** from that used by SQLite, so the maximum length is 10, not 9. 110751 111830 */ 110752 111831 #define FTS3_VARINT_MAX 10 110753 111832 111833 +/* 111834 +** FTS4 virtual tables may maintain multiple indexes - one index of all terms 111835 +** in the document set and zero or more prefix indexes. All indexes are stored 111836 +** as one or more b+-trees in the %_segments and %_segdir tables. 111837 +** 111838 +** It is possible to determine which index a b+-tree belongs to based on the 111839 +** value stored in the "%_segdir.level" column. Given this value L, the index 111840 +** that the b+-tree belongs to is (L<<10). In other words, all b+-trees with 111841 +** level values between 0 and 1023 (inclusive) belong to index 0, all levels 111842 +** between 1024 and 2047 to index 1, and so on. 111843 +** 111844 +** It is considered impossible for an index to use more than 1024 levels. In 111845 +** theory though this may happen, but only after at least 111846 +** (FTS3_MERGE_COUNT^1024) separate flushes of the pending-terms tables. 111847 +*/ 111848 +#define FTS3_SEGDIR_MAXLEVEL 1024 111849 +#define FTS3_SEGDIR_MAXLEVEL_STR "1024" 111850 + 110754 111851 /* 110755 111852 ** The testcase() macro is only used by the amalgamation. If undefined, 110756 111853 ** make it a no-op. 110757 111854 */ 110758 111855 #ifndef testcase 110759 111856 # define testcase(X) 110760 111857 #endif ................................................................................ 110786 111883 /* 110787 111884 ** Internal types used by SQLite. 110788 111885 */ 110789 111886 typedef unsigned char u8; /* 1-byte (or larger) unsigned integer */ 110790 111887 typedef short int i16; /* 2-byte (or larger) signed integer */ 110791 111888 typedef unsigned int u32; /* 4-byte unsigned integer */ 110792 111889 typedef sqlite3_uint64 u64; /* 8-byte unsigned integer */ 111890 + 110793 111891 /* 110794 111892 ** Macro used to suppress compiler warnings for unused parameters. 110795 111893 */ 110796 111894 #define UNUSED_PARAMETER(x) (void)(x) 111895 + 111896 +/* 111897 +** Activate assert() only if SQLITE_TEST is enabled. 111898 +*/ 111899 +#if !defined(NDEBUG) && !defined(SQLITE_DEBUG) 111900 +# define NDEBUG 1 110797 111901 #endif 110798 111902 111903 +/* 111904 +** The TESTONLY macro is used to enclose variable declarations or 111905 +** other bits of code that are needed to support the arguments 111906 +** within testcase() and assert() macros. 111907 +*/ 111908 +#if defined(SQLITE_DEBUG) || defined(SQLITE_COVERAGE_TEST) 111909 +# define TESTONLY(X) X 111910 +#else 111911 +# define TESTONLY(X) 111912 +#endif 111913 + 111914 +#endif /* SQLITE_AMALGAMATION */ 111915 + 110799 111916 typedef struct Fts3Table Fts3Table; 110800 111917 typedef struct Fts3Cursor Fts3Cursor; 110801 111918 typedef struct Fts3Expr Fts3Expr; 110802 111919 typedef struct Fts3Phrase Fts3Phrase; 110803 111920 typedef struct Fts3PhraseToken Fts3PhraseToken; 110804 111921 111922 +typedef struct Fts3Doclist Fts3Doclist; 110805 111923 typedef struct Fts3SegFilter Fts3SegFilter; 110806 111924 typedef struct Fts3DeferredToken Fts3DeferredToken; 110807 111925 typedef struct Fts3SegReader Fts3SegReader; 110808 -typedef struct Fts3SegReaderCursor Fts3SegReaderCursor; 111926 +typedef struct Fts3MultiSegReader Fts3MultiSegReader; 110809 111927 110810 111928 /* 110811 111929 ** A connection to a fulltext index is an instance of the following 110812 111930 ** structure. The xCreate and xConnect methods create an instance 110813 111931 ** of this structure and xDestroy and xDisconnect free that instance. 110814 111932 ** All other methods receive a pointer to the structure as one of their 110815 111933 ** arguments. ................................................................................ 110822 111940 int nColumn; /* number of named columns in virtual table */ 110823 111941 char **azColumn; /* column names. malloced */ 110824 111942 sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */ 110825 111943 110826 111944 /* Precompiled statements used by the implementation. Each of these 110827 111945 ** statements is run and reset within a single virtual table API call. 110828 111946 */ 110829 - sqlite3_stmt *aStmt[24]; 111947 + sqlite3_stmt *aStmt[27]; 110830 111948 110831 111949 char *zReadExprlist; 110832 111950 char *zWriteExprlist; 110833 111951 110834 111952 int nNodeSize; /* Soft limit for node size */ 110835 111953 u8 bHasStat; /* True if %_stat table exists */ 110836 111954 u8 bHasDocsize; /* True if %_docsize table exists */ 111955 + u8 bDescIdx; /* True if doclists are in reverse order */ 110837 111956 int nPgsz; /* Page size for host database */ 110838 111957 char *zSegmentsTbl; /* Name of %_segments table */ 110839 111958 sqlite3_blob *pSegments; /* Blob handle open on %_segments table */ 110840 111959 110841 - /* The following hash table is used to buffer pending index updates during 111960 + /* TODO: Fix the first paragraph of this comment. 111961 + ** 111962 + ** The following hash table is used to buffer pending index updates during 110842 111963 ** transactions. Variable nPendingData estimates the memory size of the 110843 111964 ** pending data, including hash table overhead, but not malloc overhead. 110844 111965 ** When nPendingData exceeds nMaxPendingData, the buffer is flushed 110845 111966 ** automatically. Variable iPrevDocid is the docid of the most recently 110846 111967 ** inserted record. 111968 + ** 111969 + ** A single FTS4 table may have multiple full-text indexes. For each index 111970 + ** there is an entry in the aIndex[] array. Index 0 is an index of all the 111971 + ** terms that appear in the document set. Each subsequent index in aIndex[] 111972 + ** is an index of prefixes of a specific length. 110847 111973 */ 110848 - int nMaxPendingData; 110849 - int nPendingData; 110850 - sqlite_int64 iPrevDocid; 110851 - Fts3Hash pendingTerms; 111974 + int nIndex; /* Size of aIndex[] */ 111975 + struct Fts3Index { 111976 + int nPrefix; /* Prefix length (0 for main terms index) */ 111977 + Fts3Hash hPending; /* Pending terms table for this index */ 111978 + } *aIndex; 111979 + int nMaxPendingData; /* Max pending data before flush to disk */ 111980 + int nPendingData; /* Current bytes of pending data */ 111981 + sqlite_int64 iPrevDocid; /* Docid of most recently inserted document */ 111982 + 111983 +#if defined(SQLITE_DEBUG) 111984 + /* State variables used for validating that the transaction control 111985 + ** methods of the virtual table are called at appropriate times. These 111986 + ** values do not contribution to the FTS computation; they are used for 111987 + ** verifying the SQLite core. 111988 + */ 111989 + int inTransaction; /* True after xBegin but before xCommit/xRollback */ 111990 + int mxSavepoint; /* Largest valid xSavepoint integer */ 111991 +#endif 110852 111992 }; 110853 111993 110854 111994 /* 110855 111995 ** When the core wants to read from the virtual table, it creates a 110856 111996 ** virtual table cursor (an instance of the following structure) using 110857 111997 ** the xOpen method. Cursors are destroyed using the xClose method. 110858 111998 */ ................................................................................ 110865 112005 Fts3Expr *pExpr; /* Parsed MATCH query string */ 110866 112006 int nPhrase; /* Number of matchable phrases in query */ 110867 112007 Fts3DeferredToken *pDeferred; /* Deferred search tokens, if any */ 110868 112008 sqlite3_int64 iPrevId; /* Previous id read from aDoclist */ 110869 112009 char *pNextId; /* Pointer into the body of aDoclist */ 110870 112010 char *aDoclist; /* List of docids for full-text queries */ 110871 112011 int nDoclist; /* Size of buffer at aDoclist */ 112012 + u8 bDesc; /* True to sort in descending order */ 110872 112013 int eEvalmode; /* An FTS3_EVAL_XX constant */ 110873 112014 int nRowAvg; /* Average size of database rows, in pages */ 112015 + sqlite3_int64 nDoc; /* Documents in table */ 110874 112016 110875 112017 int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */ 110876 112018 u32 *aMatchinfo; /* Information about most recent match */ 110877 112019 int nMatchinfo; /* Number of elements in aMatchinfo[] */ 110878 112020 char *zMatchinfo; /* Matchinfo specification */ 110879 112021 }; 110880 112022 ................................................................................ 110897 112039 ** indicating that all columns should be searched, 110898 112040 ** then eSearch would be set to FTS3_FULLTEXT_SEARCH+4. 110899 112041 */ 110900 112042 #define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */ 110901 112043 #define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */ 110902 112044 #define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */ 110903 112045 112046 + 112047 +struct Fts3Doclist { 112048 + char *aAll; /* Array containing doclist (or NULL) */ 112049 + int nAll; /* Size of a[] in bytes */ 112050 + char *pNextDocid; /* Pointer to next docid */ 112051 + 112052 + sqlite3_int64 iDocid; /* Current docid (if pList!=0) */ 112053 + int bFreeList; /* True if pList should be sqlite3_free()d */ 112054 + char *pList; /* Pointer to position list following iDocid */ 112055 + int nList; /* Length of position list */ 112056 +} doclist; 112057 + 110904 112058 /* 110905 112059 ** A "phrase" is a sequence of one or more tokens that must match in 110906 112060 ** sequence. A single token is the base case and the most common case. 110907 112061 ** For a sequence of tokens contained in double-quotes (i.e. "one two three") 110908 112062 ** nToken will be the number of tokens in the string. 110909 -** 110910 -** The nDocMatch and nMatch variables contain data that may be used by the 110911 -** matchinfo() function. They are populated when the full-text index is 110912 -** queried for hits on the phrase. If one or more tokens in the phrase 110913 -** are deferred, the nDocMatch and nMatch variables are populated based 110914 -** on the assumption that the 110915 112063 */ 110916 112064 struct Fts3PhraseToken { 110917 112065 char *z; /* Text of the token */ 110918 112066 int n; /* Number of bytes in buffer z */ 110919 112067 int isPrefix; /* True if token ends with a "*" character */ 110920 - int bFulltext; /* True if full-text index was used */ 110921 - Fts3SegReaderCursor *pSegcsr; /* Segment-reader for this token */ 112068 + 112069 + /* Variables above this point are populated when the expression is 112070 + ** parsed (by code in fts3_expr.c). Below this point the variables are 112071 + ** used when evaluating the expression. */ 110922 112072 Fts3DeferredToken *pDeferred; /* Deferred token object for this token */ 112073 + Fts3MultiSegReader *pSegcsr; /* Segment-reader for this token */ 110923 112074 }; 110924 112075 110925 112076 struct Fts3Phrase { 110926 - /* Variables populated by fts3_expr.c when parsing a MATCH expression */ 112077 + /* Cache of doclist for this phrase. */ 112078 + Fts3Doclist doclist; 112079 + int bIncr; /* True if doclist is loaded incrementally */ 112080 + int iDoclistToken; 112081 + 112082 + /* Variables below this point are populated by fts3_expr.c when parsing 112083 + ** a MATCH expression. Everything above is part of the evaluation phase. 112084 + */ 110927 112085 int nToken; /* Number of tokens in the phrase */ 110928 112086 int iColumn; /* Index of column this phrase must match */ 110929 - int isNot; /* Phrase prefixed by unary not (-) operator */ 110930 112087 Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */ 110931 112088 }; 110932 112089 110933 112090 /* 110934 112091 ** A tree of these objects forms the RHS of a MATCH operator. 110935 112092 ** 110936 -** If Fts3Expr.eType is either FTSQUERY_NEAR or FTSQUERY_PHRASE and isLoaded 110937 -** is true, then aDoclist points to a malloced buffer, size nDoclist bytes, 110938 -** containing the results of the NEAR or phrase query in FTS3 doclist 110939 -** format. As usual, the initial "Length" field found in doclists stored 110940 -** on disk is omitted from this buffer. 112093 +** If Fts3Expr.eType is FTSQUERY_PHRASE and isLoaded is true, then aDoclist 112094 +** points to a malloced buffer, size nDoclist bytes, containing the results 112095 +** of this phrase query in FTS3 doclist format. As usual, the initial 112096 +** "Length" field found in doclists stored on disk is omitted from this 112097 +** buffer. 110941 112098 ** 110942 -** Variable pCurrent always points to the start of a docid field within 110943 -** aDoclist. Since the doclist is usually scanned in docid order, this can 110944 -** be used to accelerate seeking to the required docid within the doclist. 112099 +** Variable aMI is used only for FTSQUERY_NEAR nodes to store the global 112100 +** matchinfo data. If it is not NULL, it points to an array of size nCol*3, 112101 +** where nCol is the number of columns in the queried FTS table. The array 112102 +** is populated as follows: 112103 +** 112104 +** aMI[iCol*3 + 0] = Undefined 112105 +** aMI[iCol*3 + 1] = Number of occurrences 112106 +** aMI[iCol*3 + 2] = Number of rows containing at least one instance 112107 +** 112108 +** The aMI array is allocated using sqlite3_malloc(). It should be freed 112109 +** when the expression node is. 110945 112110 */ 110946 112111 struct Fts3Expr { 110947 112112 int eType; /* One of the FTSQUERY_XXX values defined below */ 110948 112113 int nNear; /* Valid if eType==FTSQUERY_NEAR */ 110949 112114 Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */ 110950 112115 Fts3Expr *pLeft; /* Left operand */ 110951 112116 Fts3Expr *pRight; /* Right operand */ 110952 112117 Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */ 110953 112118 110954 - int isLoaded; /* True if aDoclist/nDoclist are initialized. */ 110955 - char *aDoclist; /* Buffer containing doclist */ 110956 - int nDoclist; /* Size of aDoclist in bytes */ 112119 + /* The following are used by the fts3_eval.c module. */ 112120 + sqlite3_int64 iDocid; /* Current docid */ 112121 + u8 bEof; /* True this expression is at EOF already */ 112122 + u8 bStart; /* True if iDocid is valid */ 112123 + u8 bDeferred; /* True if this expression is entirely deferred */ 110957 112124 110958 - sqlite3_int64 iCurrent; 110959 - char *pCurrent; 112125 + u32 *aMI; 110960 112126 }; 110961 112127 110962 112128 /* 110963 112129 ** Candidate values for Fts3Query.eType. Note that the order of the first 110964 112130 ** four values is in order of precedence when parsing expressions. For 110965 112131 ** example, the following: 110966 112132 ** ................................................................................ 110980 112146 /* fts3_write.c */ 110981 112147 SQLITE_PRIVATE int sqlite3Fts3UpdateMethod(sqlite3_vtab*,int,sqlite3_value**,sqlite3_int64*); 110982 112148 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *); 110983 112149 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *); 110984 112150 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *); 110985 112151 SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(int, sqlite3_int64, 110986 112152 sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**); 110987 -SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(Fts3Table*,const char*,int,int,Fts3SegReader**); 112153 +SQLITE_PRIVATE int sqlite3Fts3SegReaderPending( 112154 + Fts3Table*,int,const char*,int,int,Fts3SegReader**); 110988 112155 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *); 110989 -SQLITE_PRIVATE int sqlite3Fts3SegReaderCost(Fts3Cursor *, Fts3SegReader *, int *); 110990 -SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, sqlite3_stmt **); 112156 +SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, int, sqlite3_stmt **); 110991 112157 SQLITE_PRIVATE int sqlite3Fts3ReadLock(Fts3Table *); 110992 -SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*); 112158 +SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*, int*); 110993 112159 110994 112160 SQLITE_PRIVATE int sqlite3Fts3SelectDoctotal(Fts3Table *, sqlite3_stmt **); 110995 112161 SQLITE_PRIVATE int sqlite3Fts3SelectDocsize(Fts3Table *, sqlite3_int64, sqlite3_stmt **); 110996 112162 110997 112163 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *); 110998 112164 SQLITE_PRIVATE int sqlite3Fts3DeferToken(Fts3Cursor *, Fts3PhraseToken *, int); 110999 112165 SQLITE_PRIVATE int sqlite3Fts3CacheDeferredDoclists(Fts3Cursor *); 111000 112166 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *); 111001 -SQLITE_PRIVATE char *sqlite3Fts3DeferredDoclist(Fts3DeferredToken *, int *); 111002 112167 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *); 111003 112168 111004 -#define FTS3_SEGCURSOR_PENDING -1 111005 -#define FTS3_SEGCURSOR_ALL -2 112169 +/* Special values interpreted by sqlite3SegReaderCursor() */ 112170 +#define FTS3_SEGCURSOR_PENDING -1 112171 +#define FTS3_SEGCURSOR_ALL -2 111006 112172 111007 -SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3SegReaderCursor*, Fts3SegFilter*); 111008 -SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3SegReaderCursor *); 111009 -SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3SegReaderCursor *); 112173 +SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3MultiSegReader*, Fts3SegFilter*); 112174 +SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3MultiSegReader *); 112175 +SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3MultiSegReader *); 112176 + 111010 112177 SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor( 111011 - Fts3Table *, int, const char *, int, int, int, Fts3SegReaderCursor *); 112178 + Fts3Table *, int, int, const char *, int, int, int, Fts3MultiSegReader *); 111012 112179 111013 112180 /* Flags allowed as part of the 4th argument to SegmentReaderIterate() */ 111014 112181 #define FTS3_SEGMENT_REQUIRE_POS 0x00000001 111015 112182 #define FTS3_SEGMENT_IGNORE_EMPTY 0x00000002 111016 112183 #define FTS3_SEGMENT_COLUMN_FILTER 0x00000004 111017 112184 #define FTS3_SEGMENT_PREFIX 0x00000008 111018 112185 #define FTS3_SEGMENT_SCAN 0x00000010 ................................................................................ 111021 112188 struct Fts3SegFilter { 111022 112189 const char *zTerm; 111023 112190 int nTerm; 111024 112191 int iCol; 111025 112192 int flags; 111026 112193 }; 111027 112194 111028 -struct Fts3SegReaderCursor { 112195 +struct Fts3MultiSegReader { 111029 112196 /* Used internally by sqlite3Fts3SegReaderXXX() calls */ 111030 112197 Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */ 111031 112198 int nSegment; /* Size of apSegment array */ 111032 112199 int nAdvance; /* How many seg-readers to advance */ 111033 112200 Fts3SegFilter *pFilter; /* Pointer to filter object */ 111034 112201 char *aBuffer; /* Buffer to merge doclists in */ 111035 112202 int nBuffer; /* Allocated size of aBuffer[] in bytes */ 111036 112203 111037 - /* Cost of running this iterator. Used by fts3.c only. */ 111038 - int nCost; 112204 + int iColFilter; /* If >=0, filter for this column */ 112205 + int bRestart; 112206 + 112207 + /* Used by fts3.c only. */ 112208 + int nCost; /* Cost of running iterator */ 112209 + int bLookup; /* True if a lookup of a single entry. */ 111039 112210 111040 112211 /* Output values. Valid only after Fts3SegReaderStep() returns SQLITE_ROW. */ 111041 112212 char *zTerm; /* Pointer to term buffer */ 111042 112213 int nTerm; /* Size of zTerm in bytes */ 111043 112214 char *aDoclist; /* Pointer to doclist buffer */ 111044 112215 int nDoclist; /* Size of aDoclist[] in bytes */ 111045 112216 }; ................................................................................ 111046 112217 111047 112218 /* fts3.c */ 111048 112219 SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *, sqlite3_int64); 111049 112220 SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *, sqlite_int64 *); 111050 112221 SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *, int *); 111051 112222 SQLITE_PRIVATE int sqlite3Fts3VarintLen(sqlite3_uint64); 111052 112223 SQLITE_PRIVATE void sqlite3Fts3Dequote(char *); 112224 +SQLITE_PRIVATE void sqlite3Fts3DoclistPrev(int,char*,int,char**,sqlite3_int64*,int*,u8*); 111053 112225 111054 -SQLITE_PRIVATE char *sqlite3Fts3FindPositions(Fts3Expr *, sqlite3_int64, int); 111055 -SQLITE_PRIVATE int sqlite3Fts3ExprLoadDoclist(Fts3Cursor *, Fts3Expr *); 111056 -SQLITE_PRIVATE int sqlite3Fts3ExprLoadFtDoclist(Fts3Cursor *, Fts3Expr *, char **, int *); 111057 -SQLITE_PRIVATE int sqlite3Fts3ExprNearTrim(Fts3Expr *, Fts3Expr *, int); 112226 +SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *); 111058 112227 111059 112228 /* fts3_tokenizer.c */ 111060 112229 SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *, int *); 111061 112230 SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *); 111062 112231 SQLITE_PRIVATE int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *, 111063 112232 sqlite3_tokenizer **, char ** 111064 112233 ); ................................................................................ 111074 112243 /* fts3_expr.c */ 111075 112244 SQLITE_PRIVATE int sqlite3Fts3ExprParse(sqlite3_tokenizer *, 111076 112245 char **, int, int, const char *, int, Fts3Expr ** 111077 112246 ); 111078 112247 SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *); 111079 112248 #ifdef SQLITE_TEST 111080 112249 SQLITE_PRIVATE int sqlite3Fts3ExprInitTestInterface(sqlite3 *db); 112250 +SQLITE_PRIVATE int sqlite3Fts3InitTerm(sqlite3 *db); 111081 112251 #endif 111082 112252 111083 112253 /* fts3_aux.c */ 111084 112254 SQLITE_PRIVATE int sqlite3Fts3InitAux(sqlite3 *db); 111085 112255 112256 +SQLITE_PRIVATE int sqlite3Fts3TermSegReaderCursor( 112257 + Fts3Cursor *pCsr, /* Virtual table cursor handle */ 112258 + const char *zTerm, /* Term to query for */ 112259 + int nTerm, /* Size of zTerm in bytes */ 112260 + int isPrefix, /* True for a prefix search */ 112261 + Fts3MultiSegReader **ppSegcsr /* OUT: Allocated seg-reader cursor */ 112262 +); 112263 + 112264 +SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *); 112265 + 112266 +SQLITE_PRIVATE int sqlite3Fts3EvalStart(Fts3Cursor *, Fts3Expr *, int); 112267 +SQLITE_PRIVATE int sqlite3Fts3EvalNext(Fts3Cursor *pCsr); 112268 + 112269 +SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart( 112270 + Fts3Table*, Fts3MultiSegReader*, int, const char*, int); 112271 +SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext( 112272 + Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *); 112273 +SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol); 112274 +SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *); 112275 +SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr); 112276 + 112277 +SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(Fts3DeferredToken *, char **, int *); 112278 + 112279 +#endif /* SQLITE_ENABLE_FTS3 */ 111086 112280 #endif /* _FTSINT_H */ 111087 112281 111088 112282 /************** End of fts3Int.h *********************************************/ 111089 112283 /************** Continuing where we left off in fts3.c ***********************/ 112284 +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) 112285 + 112286 +#if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE) 112287 +# define SQLITE_CORE 1 112288 +#endif 111090 112289 111091 112290 111092 112291 #ifndef SQLITE_CORE 111093 112292 SQLITE_EXTENSION_INIT1 111094 112293 #endif 111095 112294 111096 112295 /* ................................................................................ 111196 112395 static void fts3GetDeltaVarint(char **pp, sqlite3_int64 *pVal){ 111197 112396 sqlite3_int64 iVal; 111198 112397 *pp += sqlite3Fts3GetVarint(*pp, &iVal); 111199 112398 *pVal += iVal; 111200 112399 } 111201 112400 111202 112401 /* 111203 -** As long as *pp has not reached its end (pEnd), then do the same 111204 -** as fts3GetDeltaVarint(): read a single varint and add it to *pVal. 111205 -** But if we have reached the end of the varint, just set *pp=0 and 111206 -** leave *pVal unchanged. 112402 +** When this function is called, *pp points to the first byte following a 112403 +** varint that is part of a doclist (or position-list, or any other list 112404 +** of varints). This function moves *pp to point to the start of that varint, 112405 +** and sets *pVal by the varint value. 112406 +** 112407 +** Argument pStart points to the first byte of the doclist that the 112408 +** varint is part of. 111207 112409 */ 111208 -static void fts3GetDeltaVarint2(char **pp, char *pEnd, sqlite3_int64 *pVal){ 111209 - if( *pp>=pEnd ){ 111210 - *pp = 0; 111211 - }else{ 111212 - fts3GetDeltaVarint(pp, pVal); 111213 - } 112410 +static void fts3GetReverseVarint( 112411 + char **pp, 112412 + char *pStart, 112413 + sqlite3_int64 *pVal 112414 +){ 112415 + sqlite3_int64 iVal; 112416 + char *p = *pp; 112417 + 112418 + /* Pointer p now points at the first byte past the varint we are 112419 + ** interested in. So, unless the doclist is corrupt, the 0x80 bit is 112420 + ** clear on character p[-1]. */ 112421 + for(p = (*pp)-2; p>=pStart && *p&0x80; p--); 112422 + p++; 112423 + *pp = p; 112424 + 112425 + sqlite3Fts3GetVarint(p, &iVal); 112426 + *pVal = iVal; 111214 112427 } 111215 112428 111216 112429 /* 111217 112430 ** The xDisconnect() virtual table method. 111218 112431 */ 111219 112432 static int fts3DisconnectMethod(sqlite3_vtab *pVtab){ 111220 112433 Fts3Table *p = (Fts3Table *)pVtab; ................................................................................ 111299 112512 */ 111300 112513 static void fts3DeclareVtab(int *pRc, Fts3Table *p){ 111301 112514 if( *pRc==SQLITE_OK ){ 111302 112515 int i; /* Iterator variable */ 111303 112516 int rc; /* Return code */ 111304 112517 char *zSql; /* SQL statement passed to declare_vtab() */ 111305 112518 char *zCols; /* List of user defined columns */ 112519 + 112520 + sqlite3_vtab_config(p->db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1); 111306 112521 111307 112522 /* Create a list of user columns for the virtual table */ 111308 112523 zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]); 111309 112524 for(i=1; zCols && i<p->nColumn; i++){ 111310 112525 zCols = sqlite3_mprintf("%z%Q, ", zCols, p->azColumn[i]); 111311 112526 } 111312 112527 ................................................................................ 111405 112620 rc = SQLITE_NOMEM; 111406 112621 }else{ 111407 112622 rc = sqlite3_prepare(p->db, zSql, -1, &pStmt, 0); 111408 112623 if( rc==SQLITE_OK ){ 111409 112624 sqlite3_step(pStmt); 111410 112625 p->nPgsz = sqlite3_column_int(pStmt, 0); 111411 112626 rc = sqlite3_finalize(pStmt); 112627 + }else if( rc==SQLITE_AUTH ){ 112628 + p->nPgsz = 1024; 112629 + rc = SQLITE_OK; 111412 112630 } 111413 112631 } 111414 112632 assert( p->nPgsz>0 || rc!=SQLITE_OK ); 111415 112633 sqlite3_free(zSql); 111416 112634 *pRc = rc; 111417 112635 } 111418 112636 } ................................................................................ 111577 112795 fts3Appendf(pRc, &zRet, "?"); 111578 112796 for(i=0; i<p->nColumn; i++){ 111579 112797 fts3Appendf(pRc, &zRet, ",%s(?)", zFunction); 111580 112798 } 111581 112799 sqlite3_free(zFree); 111582 112800 return zRet; 111583 112801 } 112802 + 112803 +static int fts3GobbleInt(const char **pp, int *pnOut){ 112804 + const char *p = *pp; 112805 + int nInt = 0; 112806 + for(p=*pp; p[0]>='0' && p[0]<='9'; p++){ 112807 + nInt = nInt * 10 + (p[0] - '0'); 112808 + } 112809 + if( p==*pp ) return SQLITE_ERROR; 112810 + *pnOut = nInt; 112811 + *pp = p; 112812 + return SQLITE_OK; 112813 +} 112814 + 112815 + 112816 +static int fts3PrefixParameter( 112817 + const char *zParam, /* ABC in prefix=ABC parameter to parse */ 112818 + int *pnIndex, /* OUT: size of *apIndex[] array */ 112819 + struct Fts3Index **apIndex, /* OUT: Array of indexes for this table */ 112820 + struct Fts3Index **apFree /* OUT: Free this with sqlite3_free() */ 112821 +){ 112822 + struct Fts3Index *aIndex; 112823 + int nIndex = 1; 112824 + 112825 + if( zParam && zParam[0] ){ 112826 + const char *p; 112827 + nIndex++; 112828 + for(p=zParam; *p; p++){ 112829 + if( *p==',' ) nIndex++; 112830 + } 112831 + } 112832 + 112833 + aIndex = sqlite3_malloc(sizeof(struct Fts3Index) * nIndex); 112834 + *apIndex = *apFree = aIndex; 112835 + *pnIndex = nIndex; 112836 + if( !aIndex ){ 112837 + return SQLITE_NOMEM; 112838 + } 112839 + 112840 + memset(aIndex, 0, sizeof(struct Fts3Index) * nIndex); 112841 + if( zParam ){ 112842 + const char *p = zParam; 112843 + int i; 112844 + for(i=1; i<nIndex; i++){ 112845 + int nPrefix; 112846 + if( fts3GobbleInt(&p, &nPrefix) ) return SQLITE_ERROR; 112847 + aIndex[i].nPrefix = nPrefix; 112848 + p++; 112849 + } 112850 + } 112851 + 112852 + return SQLITE_OK; 112853 +} 111584 112854 111585 112855 /* 111586 112856 ** This function is the implementation of both the xConnect and xCreate 111587 112857 ** methods of the FTS3 virtual table. 111588 112858 ** 111589 112859 ** The argv[] array contains the following: 111590 112860 ** ................................................................................ 111610 112880 int iCol; /* Column index */ 111611 112881 int nString = 0; /* Bytes required to hold all column names */ 111612 112882 int nCol = 0; /* Number of columns in the FTS table */ 111613 112883 char *zCsr; /* Space for holding column names */ 111614 112884 int nDb; /* Bytes required to hold database name */ 111615 112885 int nName; /* Bytes required to hold table name */ 111616 112886 int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */ 111617 - int bNoDocsize = 0; /* True to omit %_docsize table */ 111618 112887 const char **aCol; /* Array of column names */ 111619 112888 sqlite3_tokenizer *pTokenizer = 0; /* Tokenizer for this table */ 111620 112889 111621 - char *zCompress = 0; 111622 - char *zUncompress = 0; 112890 + int nIndex; /* Size of aIndex[] array */ 112891 + struct Fts3Index *aIndex; /* Array of indexes for this table */ 112892 + struct Fts3Index *aFree = 0; /* Free this before returning */ 112893 + 112894 + /* The results of parsing supported FTS4 key=value options: */ 112895 + int bNoDocsize = 0; /* True to omit %_docsize table */ 112896 + int bDescIdx = 0; /* True to store descending indexes */ 112897 + char *zPrefix = 0; /* Prefix parameter value (or NULL) */ 112898 + char *zCompress = 0; /* compress=? parameter (or NULL) */ 112899 + char *zUncompress = 0; /* uncompress=? parameter (or NULL) */ 111623 112900 111624 112901 assert( strlen(argv[0])==4 ); 111625 112902 assert( (sqlite3_strnicmp(argv[0], "fts4", 4)==0 && isFts4) 111626 112903 || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4) 111627 112904 ); 111628 112905 111629 112906 nDb = (int)strlen(argv[1]) + 1; ................................................................................ 111656 112933 && 0==sqlite3Fts3IsIdChar(z[8]) 111657 112934 ){ 111658 112935 rc = sqlite3Fts3InitTokenizer(pHash, &z[9], &pTokenizer, pzErr); 111659 112936 } 111660 112937 111661 112938 /* Check if it is an FTS4 special argument. */ 111662 112939 else if( isFts4 && fts3IsSpecialColumn(z, &nKey, &zVal) ){ 112940 + struct Fts4Option { 112941 + const char *zOpt; 112942 + int nOpt; 112943 + char **pzVar; 112944 + } aFts4Opt[] = { 112945 + { "matchinfo", 9, 0 }, /* 0 -> MATCHINFO */ 112946 + { "prefix", 6, 0 }, /* 1 -> PREFIX */ 112947 + { "compress", 8, 0 }, /* 2 -> COMPRESS */ 112948 + { "uncompress", 10, 0 }, /* 3 -> UNCOMPRESS */ 112949 + { "order", 5, 0 } /* 4 -> ORDER */ 112950 + }; 112951 + 112952 + int iOpt; 111663 112953 if( !zVal ){ 111664 112954 rc = SQLITE_NOMEM; 111665 - goto fts3_init_out; 111666 - } 111667 - if( nKey==9 && 0==sqlite3_strnicmp(z, "matchinfo", 9) ){ 111668 - if( strlen(zVal)==4 && 0==sqlite3_strnicmp(zVal, "fts3", 4) ){ 111669 - bNoDocsize = 1; 111670 - }else{ 111671 - *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal); 112955 + }else{ 112956 + for(iOpt=0; iOpt<SizeofArray(aFts4Opt); iOpt++){ 112957 + struct Fts4Option *pOp = &aFts4Opt[iOpt]; 112958 + if( nKey==pOp->nOpt && !sqlite3_strnicmp(z, pOp->zOpt, pOp->nOpt) ){ 112959 + break; 112960 + } 112961 + } 112962 + if( iOpt==SizeofArray(aFts4Opt) ){ 112963 + *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z); 111672 112964 rc = SQLITE_ERROR; 111673 - } 111674 - }else if( nKey==8 && 0==sqlite3_strnicmp(z, "compress", 8) ){ 111675 - zCompress = zVal; 111676 - zVal = 0; 111677 - }else if( nKey==10 && 0==sqlite3_strnicmp(z, "uncompress", 10) ){ 111678 - zUncompress = zVal; 111679 - zVal = 0; 111680 - }else{ 111681 - *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z); 111682 - rc = SQLITE_ERROR; 111683 - } 111684 - sqlite3_free(zVal); 112965 + }else{ 112966 + switch( iOpt ){ 112967 + case 0: /* MATCHINFO */ 112968 + if( strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "fts3", 4) ){ 112969 + *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal); 112970 + rc = SQLITE_ERROR; 112971 + } 112972 + bNoDocsize = 1; 112973 + break; 112974 + 112975 + case 1: /* PREFIX */ 112976 + sqlite3_free(zPrefix); 112977 + zPrefix = zVal; 112978 + zVal = 0; 112979 + break; 112980 + 112981 + case 2: /* COMPRESS */ 112982 + sqlite3_free(zCompress); 112983 + zCompress = zVal; 112984 + zVal = 0; 112985 + break; 112986 + 112987 + case 3: /* UNCOMPRESS */ 112988 + sqlite3_free(zUncompress); 112989 + zUncompress = zVal; 112990 + zVal = 0; 112991 + break; 112992 + 112993 + case 4: /* ORDER */ 112994 + if( (strlen(zVal)!=3 || sqlite3_strnicmp(zVal, "asc", 3)) 112995 + && (strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "desc", 3)) 112996 + ){ 112997 + *pzErr = sqlite3_mprintf("unrecognized order: %s", zVal); 112998 + rc = SQLITE_ERROR; 112999 + } 113000 + bDescIdx = (zVal[0]=='d' || zVal[0]=='D'); 113001 + break; 113002 + } 113003 + } 113004 + sqlite3_free(zVal); 113005 + } 111685 113006 } 111686 113007 111687 113008 /* Otherwise, the argument is a column name. */ 111688 113009 else { 111689 113010 nString += (int)(strlen(z) + 1); 111690 113011 aCol[nCol++] = z; 111691 113012 } ................................................................................ 111701 113022 111702 113023 if( pTokenizer==0 ){ 111703 113024 rc = sqlite3Fts3InitTokenizer(pHash, "simple", &pTokenizer, pzErr); 111704 113025 if( rc!=SQLITE_OK ) goto fts3_init_out; 111705 113026 } 111706 113027 assert( pTokenizer ); 111707 113028 113029 + rc = fts3PrefixParameter(zPrefix, &nIndex, &aIndex, &aFree); 113030 + if( rc==SQLITE_ERROR ){ 113031 + assert( zPrefix ); 113032 + *pzErr = sqlite3_mprintf("error parsing prefix parameter: %s", zPrefix); 113033 + } 113034 + if( rc!=SQLITE_OK ) goto fts3_init_out; 111708 113035 111709 113036 /* Allocate and populate the Fts3Table structure. */ 111710 - nByte = sizeof(Fts3Table) + /* Fts3Table */ 113037 + nByte = sizeof(Fts3Table) + /* Fts3Table */ 111711 113038 nCol * sizeof(char *) + /* azColumn */ 113039 + nIndex * sizeof(struct Fts3Index) + /* aIndex */ 111712 113040 nName + /* zName */ 111713 113041 nDb + /* zDb */ 111714 113042 nString; /* Space for azColumn strings */ 111715 113043 p = (Fts3Table*)sqlite3_malloc(nByte); 111716 113044 if( p==0 ){ 111717 113045 rc = SQLITE_NOMEM; 111718 113046 goto fts3_init_out; ................................................................................ 111719 113047 } 111720 113048 memset(p, 0, nByte); 111721 113049 p->db = db; 111722 113050 p->nColumn = nCol; 111723 113051 p->nPendingData = 0; 111724 113052 p->azColumn = (char **)&p[1]; 111725 113053 p->pTokenizer = pTokenizer; 111726 - p->nNodeSize = 1000; 111727 113054 p->nMaxPendingData = FTS3_MAX_PENDING_DATA; 111728 113055 p->bHasDocsize = (isFts4 && bNoDocsize==0); 111729 113056 p->bHasStat = isFts4; 111730 - fts3HashInit(&p->pendingTerms, FTS3_HASH_STRING, 1); 113057 + p->bDescIdx = bDescIdx; 113058 + TESTONLY( p->inTransaction = -1 ); 113059 + TESTONLY( p->mxSavepoint = -1 ); 113060 + 113061 + p->aIndex = (struct Fts3Index *)&p->azColumn[nCol]; 113062 + memcpy(p->aIndex, aIndex, sizeof(struct Fts3Index) * nIndex); 113063 + p->nIndex = nIndex; 113064 + for(i=0; i<nIndex; i++){ 113065 + fts3HashInit(&p->aIndex[i].hPending, FTS3_HASH_STRING, 1); 113066 + } 111731 113067 111732 113068 /* Fill in the zName and zDb fields of the vtab structure. */ 111733 - zCsr = (char *)&p->azColumn[nCol]; 113069 + zCsr = (char *)&p->aIndex[nIndex]; 111734 113070 p->zName = zCsr; 111735 113071 memcpy(zCsr, argv[2], nName); 111736 113072 zCsr += nName; 111737 113073 p->zDb = zCsr; 111738 113074 memcpy(zCsr, argv[1], nDb); 111739 113075 zCsr += nDb; 111740 113076 111741 113077 /* Fill in the azColumn array */ 111742 113078 for(iCol=0; iCol<nCol; iCol++){ 111743 113079 char *z; 111744 - int n; 113080 + int n = 0; 111745 113081 z = (char *)sqlite3Fts3NextToken(aCol[iCol], &n); 111746 113082 memcpy(zCsr, z, n); 111747 113083 zCsr[n] = '\0'; 111748 113084 sqlite3Fts3Dequote(zCsr); 111749 113085 p->azColumn[iCol] = zCsr; 111750 113086 zCsr += n+1; 111751 113087 assert( zCsr <= &((char *)p)[nByte] ); ................................................................................ 111764 113100 ** database. TODO: For xConnect(), it could verify that said tables exist. 111765 113101 */ 111766 113102 if( isCreate ){ 111767 113103 rc = fts3CreateTables(p); 111768 113104 } 111769 113105 111770 113106 /* Figure out the page-size for the database. This is required in order to 111771 - ** estimate the cost of loading large doclists from the database (see 111772 - ** function sqlite3Fts3SegReaderCost() for details). 111773 - */ 113107 + ** estimate the cost of loading large doclists from the database. */ 111774 113108 fts3DatabasePageSize(&rc, p); 113109 + p->nNodeSize = p->nPgsz-35; 111775 113110 111776 113111 /* Declare the table schema to SQLite. */ 111777 113112 fts3DeclareVtab(&rc, p); 111778 113113 111779 113114 fts3_init_out: 113115 + sqlite3_free(zPrefix); 113116 + sqlite3_free(aFree); 111780 113117 sqlite3_free(zCompress); 111781 113118 sqlite3_free(zUncompress); 111782 113119 sqlite3_free((void *)aCol); 111783 113120 if( rc!=SQLITE_OK ){ 111784 113121 if( p ){ 111785 113122 fts3DisconnectMethod((sqlite3_vtab *)p); 111786 113123 }else if( pTokenizer ){ 111787 113124 pTokenizer->pModule->xDestroy(pTokenizer); 111788 113125 } 111789 113126 }else{ 113127 + assert( p->pSegments==0 ); 111790 113128 *ppVTab = &p->base; 111791 113129 } 111792 113130 return rc; 111793 113131 } 111794 113132 111795 113133 /* 111796 113134 ** The xConnect() and xCreate() methods for the virtual table. All the ................................................................................ 111868 113206 } 111869 113207 } 111870 113208 111871 113209 if( iCons>=0 ){ 111872 113210 pInfo->aConstraintUsage[iCons].argvIndex = 1; 111873 113211 pInfo->aConstraintUsage[iCons].omit = 1; 111874 113212 } 113213 + 113214 + /* Regardless of the strategy selected, FTS can deliver rows in rowid (or 113215 + ** docid) order. Both ascending and descending are possible. 113216 + */ 113217 + if( pInfo->nOrderBy==1 ){ 113218 + struct sqlite3_index_orderby *pOrder = &pInfo->aOrderBy[0]; 113219 + if( pOrder->iColumn<0 || pOrder->iColumn==p->nColumn+1 ){ 113220 + if( pOrder->desc ){ 113221 + pInfo->idxStr = "DESC"; 113222 + }else{ 113223 + pInfo->idxStr = "ASC"; 113224 + } 113225 + pInfo->orderByConsumed = 1; 113226 + } 113227 + } 113228 + 113229 + assert( p->pSegments==0 ); 111875 113230 return SQLITE_OK; 111876 113231 } 111877 113232 111878 113233 /* 111879 113234 ** Implementation of xOpen method. 111880 113235 */ 111881 113236 static int fts3OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){ ................................................................................ 111903 113258 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; 111904 113259 assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); 111905 113260 sqlite3_finalize(pCsr->pStmt); 111906 113261 sqlite3Fts3ExprFree(pCsr->pExpr); 111907 113262 sqlite3Fts3FreeDeferredTokens(pCsr); 111908 113263 sqlite3_free(pCsr->aDoclist); 111909 113264 sqlite3_free(pCsr->aMatchinfo); 113265 + assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); 111910 113266 sqlite3_free(pCsr); 111911 113267 return SQLITE_OK; 111912 113268 } 111913 113269 111914 113270 /* 111915 113271 ** Position the pCsr->pStmt statement so that it is on the row 111916 113272 ** of the %_content table that contains the last match. Return 111917 113273 ** SQLITE_OK on success. 111918 113274 */ 111919 113275 static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){ 111920 113276 if( pCsr->isRequireSeek ){ 111921 - pCsr->isRequireSeek = 0; 111922 113277 sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId); 113278 + pCsr->isRequireSeek = 0; 111923 113279 if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){ 111924 113280 return SQLITE_OK; 111925 113281 }else{ 111926 113282 int rc = sqlite3_reset(pCsr->pStmt); 111927 113283 if( rc==SQLITE_OK ){ 111928 113284 /* If no row was found and no error has occured, then the %_content 111929 113285 ** table is missing a row that is present in the full-text index. 111930 113286 ** The data structures are corrupt. 111931 113287 */ 111932 - rc = SQLITE_CORRUPT; 113288 + rc = SQLITE_CORRUPT_VTAB; 111933 113289 } 111934 113290 pCsr->isEof = 1; 111935 113291 if( pContext ){ 111936 113292 sqlite3_result_error_code(pContext, rc); 111937 113293 } 111938 113294 return rc; 111939 113295 } ................................................................................ 111985 113341 ** contents, or two zero bytes. Or, if the node is read from the %_segments 111986 113342 ** table, then there are always 20 bytes of zeroed padding following the 111987 113343 ** nNode bytes of content (see sqlite3Fts3ReadBlock() for details). 111988 113344 */ 111989 113345 zCsr += sqlite3Fts3GetVarint(zCsr, &iChild); 111990 113346 zCsr += sqlite3Fts3GetVarint(zCsr, &iChild); 111991 113347 if( zCsr>zEnd ){ 111992 - return SQLITE_CORRUPT; 113348 + return SQLITE_CORRUPT_VTAB; 111993 113349 } 111994 113350 111995 113351 while( zCsr<zEnd && (piFirst || piLast) ){ 111996 113352 int cmp; /* memcmp() result */ 111997 113353 int nSuffix; /* Size of term suffix */ 111998 113354 int nPrefix = 0; /* Size of term prefix */ 111999 113355 int nBuffer; /* Total term size */ ................................................................................ 112003 113359 if( !isFirstTerm ){ 112004 113360 zCsr += sqlite3Fts3GetVarint32(zCsr, &nPrefix); 112005 113361 } 112006 113362 isFirstTerm = 0; 112007 113363 zCsr += sqlite3Fts3GetVarint32(zCsr, &nSuffix); 112008 113364 112009 113365 if( nPrefix<0 || nSuffix<0 || &zCsr[nSuffix]>zEnd ){ 112010 - rc = SQLITE_CORRUPT; 113366 + rc = SQLITE_CORRUPT_VTAB; 112011 113367 goto finish_scan; 112012 113368 } 112013 113369 if( nPrefix+nSuffix>nAlloc ){ 112014 113370 char *zNew; 112015 113371 nAlloc = (nPrefix+nSuffix) * 2; 112016 113372 zNew = (char *)sqlite3_realloc(zBuffer, nAlloc); 112017 113373 if( !zNew ){ ................................................................................ 112096 113452 assert( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) ); 112097 113453 112098 113454 if( rc==SQLITE_OK && iHeight>1 ){ 112099 113455 char *zBlob = 0; /* Blob read from %_segments table */ 112100 113456 int nBlob; /* Size of zBlob in bytes */ 112101 113457 112102 113458 if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){ 112103 - rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob); 113459 + rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob, 0); 112104 113460 if( rc==SQLITE_OK ){ 112105 113461 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, 0); 112106 113462 } 112107 113463 sqlite3_free(zBlob); 112108 113464 piLeaf = 0; 112109 113465 zBlob = 0; 112110 113466 } 112111 113467 112112 113468 if( rc==SQLITE_OK ){ 112113 - rc = sqlite3Fts3ReadBlock(p, piLeaf ? *piLeaf : *piLeaf2, &zBlob, &nBlob); 113469 + rc = sqlite3Fts3ReadBlock(p, piLeaf?*piLeaf:*piLeaf2, &zBlob, &nBlob, 0); 112114 113470 } 112115 113471 if( rc==SQLITE_OK ){ 112116 113472 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2); 112117 113473 } 112118 113474 sqlite3_free(zBlob); 112119 113475 } 112120 113476 ................................................................................ 112482 113838 } 112483 113839 *p++ = 0x00; 112484 113840 *pp = p; 112485 113841 return 1; 112486 113842 } 112487 113843 112488 113844 /* 112489 -** Merge two position-lists as required by the NEAR operator. 113845 +** Merge two position-lists as required by the NEAR operator. The argument 113846 +** position lists correspond to the left and right phrases of an expression 113847 +** like: 113848 +** 113849 +** "phrase 1" NEAR "phrase number 2" 113850 +** 113851 +** Position list *pp1 corresponds to the left-hand side of the NEAR 113852 +** expression and *pp2 to the right. As usual, the indexes in the position 113853 +** lists are the offsets of the last token in each phrase (tokens "1" and "2" 113854 +** in the example above). 113855 +** 113856 +** The output position list - written to *pp - is a copy of *pp2 with those 113857 +** entries that are not sufficiently NEAR entries in *pp1 removed. 112490 113858 */ 112491 113859 static int fts3PoslistNearMerge( 112492 113860 char **pp, /* Output buffer */ 112493 113861 char *aTmp, /* Temporary buffer space */ 112494 113862 int nRight, /* Maximum difference in token positions */ 112495 113863 int nLeft, /* Maximum difference in token positions */ 112496 113864 char **pp1, /* IN/OUT: Left input list */ 112497 113865 char **pp2 /* IN/OUT: Right input list */ 112498 113866 ){ 112499 113867 char *p1 = *pp1; 112500 113868 char *p2 = *pp2; 112501 113869 112502 - if( !pp ){ 112503 - if( fts3PoslistPhraseMerge(0, nRight, 0, 0, pp1, pp2) ) return 1; 112504 - *pp1 = p1; 112505 - *pp2 = p2; 112506 - return fts3PoslistPhraseMerge(0, nLeft, 0, 0, pp2, pp1); 112507 - }else{ 112508 - char *pTmp1 = aTmp; 112509 - char *pTmp2; 112510 - char *aTmp2; 112511 - int res = 1; 112512 - 112513 - fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2); 112514 - aTmp2 = pTmp2 = pTmp1; 112515 - *pp1 = p1; 112516 - *pp2 = p2; 112517 - fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1); 112518 - if( pTmp1!=aTmp && pTmp2!=aTmp2 ){ 112519 - fts3PoslistMerge(pp, &aTmp, &aTmp2); 112520 - }else if( pTmp1!=aTmp ){ 112521 - fts3PoslistCopy(pp, &aTmp); 112522 - }else if( pTmp2!=aTmp2 ){ 112523 - fts3PoslistCopy(pp, &aTmp2); 112524 - }else{ 112525 - res = 0; 112526 - } 112527 - 112528 - return res; 112529 - } 112530 -} 112531 - 112532 -/* 112533 -** Values that may be used as the first parameter to fts3DoclistMerge(). 112534 -*/ 112535 -#define MERGE_NOT 2 /* D + D -> D */ 112536 -#define MERGE_AND 3 /* D + D -> D */ 112537 -#define MERGE_OR 4 /* D + D -> D */ 112538 -#define MERGE_POS_OR 5 /* P + P -> P */ 112539 -#define MERGE_PHRASE 6 /* P + P -> D */ 112540 -#define MERGE_POS_PHRASE 7 /* P + P -> P */ 112541 -#define MERGE_NEAR 8 /* P + P -> D */ 112542 -#define MERGE_POS_NEAR 9 /* P + P -> P */ 112543 - 112544 -/* 112545 -** Merge the two doclists passed in buffer a1 (size n1 bytes) and a2 112546 -** (size n2 bytes). The output is written to pre-allocated buffer aBuffer, 112547 -** which is guaranteed to be large enough to hold the results. The number 112548 -** of bytes written to aBuffer is stored in *pnBuffer before returning. 112549 -** 112550 -** If successful, SQLITE_OK is returned. Otherwise, if a malloc error 112551 -** occurs while allocating a temporary buffer as part of the merge operation, 112552 -** SQLITE_NOMEM is returned. 112553 -*/ 112554 -static int fts3DoclistMerge( 112555 - int mergetype, /* One of the MERGE_XXX constants */ 112556 - int nParam1, /* Used by MERGE_NEAR and MERGE_POS_NEAR */ 112557 - int nParam2, /* Used by MERGE_NEAR and MERGE_POS_NEAR */ 112558 - char *aBuffer, /* Pre-allocated output buffer */ 112559 - int *pnBuffer, /* OUT: Bytes written to aBuffer */ 112560 - char *a1, /* Buffer containing first doclist */ 112561 - int n1, /* Size of buffer a1 */ 112562 - char *a2, /* Buffer containing second doclist */ 112563 - int n2, /* Size of buffer a2 */ 112564 - int *pnDoc /* OUT: Number of docids in output */ 112565 -){ 112566 - sqlite3_int64 i1 = 0; 112567 - sqlite3_int64 i2 = 0; 112568 - sqlite3_int64 iPrev = 0; 112569 - 112570 - char *p = aBuffer; 112571 - char *p1 = a1; 112572 - char *p2 = a2; 112573 - char *pEnd1 = &a1[n1]; 112574 - char *pEnd2 = &a2[n2]; 112575 - int nDoc = 0; 112576 - 112577 - assert( mergetype==MERGE_OR || mergetype==MERGE_POS_OR 112578 - || mergetype==MERGE_AND || mergetype==MERGE_NOT 112579 - || mergetype==MERGE_PHRASE || mergetype==MERGE_POS_PHRASE 112580 - || mergetype==MERGE_NEAR || mergetype==MERGE_POS_NEAR 112581 - ); 112582 - 112583 - if( !aBuffer ){ 112584 - *pnBuffer = 0; 112585 - return SQLITE_NOMEM; 112586 - } 112587 - 112588 - /* Read the first docid from each doclist */ 112589 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112590 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112591 - 112592 - switch( mergetype ){ 112593 - case MERGE_OR: 112594 - case MERGE_POS_OR: 112595 - while( p1 || p2 ){ 112596 - if( p2 && p1 && i1==i2 ){ 112597 - fts3PutDeltaVarint(&p, &iPrev, i1); 112598 - if( mergetype==MERGE_POS_OR ) fts3PoslistMerge(&p, &p1, &p2); 112599 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112600 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112601 - }else if( !p2 || (p1 && i1<i2) ){ 112602 - fts3PutDeltaVarint(&p, &iPrev, i1); 112603 - if( mergetype==MERGE_POS_OR ) fts3PoslistCopy(&p, &p1); 112604 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112605 - }else{ 112606 - fts3PutDeltaVarint(&p, &iPrev, i2); 112607 - if( mergetype==MERGE_POS_OR ) fts3PoslistCopy(&p, &p2); 112608 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112609 - } 112610 - } 112611 - break; 112612 - 112613 - case MERGE_AND: 112614 - while( p1 && p2 ){ 112615 - if( i1==i2 ){ 112616 - fts3PutDeltaVarint(&p, &iPrev, i1); 112617 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112618 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112619 - nDoc++; 112620 - }else if( i1<i2 ){ 112621 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112622 - }else{ 112623 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112624 - } 112625 - } 112626 - break; 112627 - 112628 - case MERGE_NOT: 112629 - while( p1 ){ 112630 - if( p2 && i1==i2 ){ 112631 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112632 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112633 - }else if( !p2 || i1<i2 ){ 112634 - fts3PutDeltaVarint(&p, &iPrev, i1); 112635 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112636 - }else{ 112637 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112638 - } 112639 - } 112640 - break; 112641 - 112642 - case MERGE_POS_PHRASE: 112643 - case MERGE_PHRASE: { 112644 - char **ppPos = (mergetype==MERGE_PHRASE ? 0 : &p); 112645 - while( p1 && p2 ){ 112646 - if( i1==i2 ){ 112647 - char *pSave = p; 112648 - sqlite3_int64 iPrevSave = iPrev; 112649 - fts3PutDeltaVarint(&p, &iPrev, i1); 112650 - if( 0==fts3PoslistPhraseMerge(ppPos, nParam1, 0, 1, &p1, &p2) ){ 112651 - p = pSave; 112652 - iPrev = iPrevSave; 112653 - }else{ 112654 - nDoc++; 112655 - } 112656 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112657 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112658 - }else if( i1<i2 ){ 112659 - fts3PoslistCopy(0, &p1); 112660 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112661 - }else{ 112662 - fts3PoslistCopy(0, &p2); 112663 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112664 - } 112665 - } 112666 - break; 112667 - } 112668 - 112669 - default: assert( mergetype==MERGE_POS_NEAR || mergetype==MERGE_NEAR ); { 112670 - char *aTmp = 0; 112671 - char **ppPos = 0; 112672 - 112673 - if( mergetype==MERGE_POS_NEAR ){ 112674 - ppPos = &p; 112675 - aTmp = sqlite3_malloc(2*(n1+n2+1)); 112676 - if( !aTmp ){ 112677 - return SQLITE_NOMEM; 112678 - } 112679 - } 112680 - 112681 - while( p1 && p2 ){ 112682 - if( i1==i2 ){ 112683 - char *pSave = p; 112684 - sqlite3_int64 iPrevSave = iPrev; 112685 - fts3PutDeltaVarint(&p, &iPrev, i1); 112686 - 112687 - if( !fts3PoslistNearMerge(ppPos, aTmp, nParam1, nParam2, &p1, &p2) ){ 112688 - iPrev = iPrevSave; 112689 - p = pSave; 112690 - } 112691 - 112692 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112693 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112694 - }else if( i1<i2 ){ 112695 - fts3PoslistCopy(0, &p1); 112696 - fts3GetDeltaVarint2(&p1, pEnd1, &i1); 112697 - }else{ 112698 - fts3PoslistCopy(0, &p2); 112699 - fts3GetDeltaVarint2(&p2, pEnd2, &i2); 112700 - } 112701 - } 112702 - sqlite3_free(aTmp); 112703 - break; 112704 - } 112705 - } 112706 - 112707 - if( pnDoc ) *pnDoc = nDoc; 112708 - *pnBuffer = (int)(p-aBuffer); 112709 - return SQLITE_OK; 113870 + char *pTmp1 = aTmp; 113871 + char *pTmp2; 113872 + char *aTmp2; 113873 + int res = 1; 113874 + 113875 + fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2); 113876 + aTmp2 = pTmp2 = pTmp1; 113877 + *pp1 = p1; 113878 + *pp2 = p2; 113879 + fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1); 113880 + if( pTmp1!=aTmp && pTmp2!=aTmp2 ){ 113881 + fts3PoslistMerge(pp, &aTmp, &aTmp2); 113882 + }else if( pTmp1!=aTmp ){ 113883 + fts3PoslistCopy(pp, &aTmp); 113884 + }else if( pTmp2!=aTmp2 ){ 113885 + fts3PoslistCopy(pp, &aTmp2); 113886 + }else{ 113887 + res = 0; 113888 + } 113889 + 113890 + return res; 112710 113891 } 112711 113892 112712 113893 /* 112713 113894 ** A pointer to an instance of this structure is used as the context 112714 113895 ** argument to sqlite3Fts3SegReaderIterate() 112715 113896 */ 112716 113897 typedef struct TermSelect TermSelect; 112717 113898 struct TermSelect { 112718 113899 int isReqPos; 112719 113900 char *aaOutput[16]; /* Malloc'd output buffer */ 112720 113901 int anOutput[16]; /* Size of output in bytes */ 112721 113902 }; 113903 + 113904 + 113905 +static void fts3GetDeltaVarint3( 113906 + char **pp, 113907 + char *pEnd, 113908 + int bDescIdx, 113909 + sqlite3_int64 *pVal 113910 +){ 113911 + if( *pp>=pEnd ){ 113912 + *pp = 0; 113913 + }else{ 113914 + sqlite3_int64 iVal; 113915 + *pp += sqlite3Fts3GetVarint(*pp, &iVal); 113916 + if( bDescIdx ){ 113917 + *pVal -= iVal; 113918 + }else{ 113919 + *pVal += iVal; 113920 + } 113921 + } 113922 +} 113923 + 113924 +static void fts3PutDeltaVarint3( 113925 + char **pp, /* IN/OUT: Output pointer */ 113926 + int bDescIdx, /* True for descending docids */ 113927 + sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */ 113928 + int *pbFirst, /* IN/OUT: True after first int written */ 113929 + sqlite3_int64 iVal /* Write this value to the list */ 113930 +){ 113931 + sqlite3_int64 iWrite; 113932 + if( bDescIdx==0 || *pbFirst==0 ){ 113933 + iWrite = iVal - *piPrev; 113934 + }else{ 113935 + iWrite = *piPrev - iVal; 113936 + } 113937 + assert( *pbFirst || *piPrev==0 ); 113938 + assert( *pbFirst==0 || iWrite>0 ); 113939 + *pp += sqlite3Fts3PutVarint(*pp, iWrite); 113940 + *piPrev = iVal; 113941 + *pbFirst = 1; 113942 +} 113943 + 113944 +#define COMPARE_DOCID(i1, i2) ((bDescIdx?-1:1) * (i1-i2)) 113945 + 113946 +static int fts3DoclistOrMerge( 113947 + int bDescIdx, /* True if arguments are desc */ 113948 + char *a1, int n1, /* First doclist */ 113949 + char *a2, int n2, /* Second doclist */ 113950 + char **paOut, int *pnOut /* OUT: Malloc'd doclist */ 113951 +){ 113952 + sqlite3_int64 i1 = 0; 113953 + sqlite3_int64 i2 = 0; 113954 + sqlite3_int64 iPrev = 0; 113955 + char *pEnd1 = &a1[n1]; 113956 + char *pEnd2 = &a2[n2]; 113957 + char *p1 = a1; 113958 + char *p2 = a2; 113959 + char *p; 113960 + char *aOut; 113961 + int bFirstOut = 0; 113962 + 113963 + *paOut = 0; 113964 + *pnOut = 0; 113965 + aOut = sqlite3_malloc(n1+n2); 113966 + if( !aOut ) return SQLITE_NOMEM; 113967 + 113968 + p = aOut; 113969 + fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); 113970 + fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2); 113971 + while( p1 || p2 ){ 113972 + sqlite3_int64 iDiff = COMPARE_DOCID(i1, i2); 113973 + 113974 + if( p2 && p1 && iDiff==0 ){ 113975 + fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i1); 113976 + fts3PoslistMerge(&p, &p1, &p2); 113977 + fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); 113978 + fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); 113979 + }else if( !p2 || (p1 && iDiff<0) ){ 113980 + fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i1); 113981 + fts3PoslistCopy(&p, &p1); 113982 + fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); 113983 + }else{ 113984 + fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i2); 113985 + fts3PoslistCopy(&p, &p2); 113986 + fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); 113987 + } 113988 + } 113989 + 113990 + *paOut = aOut; 113991 + *pnOut = (p-aOut); 113992 + return SQLITE_OK; 113993 +} 113994 + 113995 +static void fts3DoclistPhraseMerge( 113996 + int bDescIdx, /* True if arguments are desc */ 113997 + int nDist, /* Distance from left to right (1=adjacent) */ 113998 + char *aLeft, int nLeft, /* Left doclist */ 113999 + char *aRight, int *pnRight /* IN/OUT: Right/output doclist */ 114000 +){ 114001 + sqlite3_int64 i1 = 0; 114002 + sqlite3_int64 i2 = 0; 114003 + sqlite3_int64 iPrev = 0; 114004 + char *pEnd1 = &aLeft[nLeft]; 114005 + char *pEnd2 = &aRight[*pnRight]; 114006 + char *p1 = aLeft; 114007 + char *p2 = aRight; 114008 + char *p; 114009 + int bFirstOut = 0; 114010 + char *aOut = aRight; 114011 + 114012 + assert( nDist>0 ); 114013 + 114014 + p = aOut; 114015 + fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); 114016 + fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2); 114017 + 114018 + while( p1 && p2 ){ 114019 + sqlite3_int64 iDiff = COMPARE_DOCID(i1, i2); 114020 + if( iDiff==0 ){ 114021 + char *pSave = p; 114022 + sqlite3_int64 iPrevSave = iPrev; 114023 + int bFirstOutSave = bFirstOut; 114024 + 114025 + fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i1); 114026 + if( 0==fts3PoslistPhraseMerge(&p, nDist, 0, 1, &p1, &p2) ){ 114027 + p = pSave; 114028 + iPrev = iPrevSave; 114029 + bFirstOut = bFirstOutSave; 114030 + } 114031 + fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); 114032 + fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); 114033 + }else if( iDiff<0 ){ 114034 + fts3PoslistCopy(0, &p1); 114035 + fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); 114036 + }else{ 114037 + fts3PoslistCopy(0, &p2); 114038 + fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); 114039 + } 114040 + } 114041 + 114042 + *pnRight = p - aOut; 114043 +} 114044 + 112722 114045 112723 114046 /* 112724 114047 ** Merge all doclists in the TermSelect.aaOutput[] array into a single 112725 114048 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all 112726 114049 ** other doclists (except the aaOutput[0] one) and return SQLITE_OK. 112727 114050 ** 112728 114051 ** If an OOM error occurs, return SQLITE_NOMEM. In this case it is 112729 114052 ** the responsibility of the caller to free any doclists left in the 112730 114053 ** TermSelect.aaOutput[] array. 112731 114054 */ 112732 -static int fts3TermSelectMerge(TermSelect *pTS){ 112733 - int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR); 114055 +static int fts3TermSelectMerge(Fts3Table *p, TermSelect *pTS){ 112734 114056 char *aOut = 0; 112735 114057 int nOut = 0; 112736 114058 int i; 112737 114059 112738 114060 /* Loop through the doclists in the aaOutput[] array. Merge them all 112739 114061 ** into a single doclist. 112740 114062 */ ................................................................................ 112741 114063 for(i=0; i<SizeofArray(pTS->aaOutput); i++){ 112742 114064 if( pTS->aaOutput[i] ){ 112743 114065 if( !aOut ){ 112744 114066 aOut = pTS->aaOutput[i]; 112745 114067 nOut = pTS->anOutput[i]; 112746 114068 pTS->aaOutput[i] = 0; 112747 114069 }else{ 112748 - int nNew = nOut + pTS->anOutput[i]; 112749 - char *aNew = sqlite3_malloc(nNew); 112750 - if( !aNew ){ 114070 + int nNew; 114071 + char *aNew; 114072 + 114073 + int rc = fts3DoclistOrMerge(p->bDescIdx, 114074 + pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, &aNew, &nNew 114075 + ); 114076 + if( rc!=SQLITE_OK ){ 112751 114077 sqlite3_free(aOut); 112752 - return SQLITE_NOMEM; 114078 + return rc; 112753 114079 } 112754 - fts3DoclistMerge(mergetype, 0, 0, 112755 - aNew, &nNew, pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, 0 112756 - ); 114080 + 112757 114081 sqlite3_free(pTS->aaOutput[i]); 112758 114082 sqlite3_free(aOut); 112759 114083 pTS->aaOutput[i] = 0; 112760 114084 aOut = aNew; 112761 114085 nOut = nNew; 112762 114086 } 112763 114087 } ................................................................................ 112785 114109 112786 114110 UNUSED_PARAMETER(p); 112787 114111 UNUSED_PARAMETER(zTerm); 112788 114112 UNUSED_PARAMETER(nTerm); 112789 114113 112790 114114 if( pTS->aaOutput[0]==0 ){ 112791 114115 /* If this is the first term selected, copy the doclist to the output 112792 - ** buffer using memcpy(). TODO: Add a way to transfer control of the 112793 - ** aDoclist buffer from the caller so as to avoid the memcpy(). 112794 - */ 114116 + ** buffer using memcpy(). */ 112795 114117 pTS->aaOutput[0] = sqlite3_malloc(nDoclist); 112796 114118 pTS->anOutput[0] = nDoclist; 112797 114119 if( pTS->aaOutput[0] ){ 112798 114120 memcpy(pTS->aaOutput[0], aDoclist, nDoclist); 112799 114121 }else{ 112800 114122 return SQLITE_NOMEM; 112801 114123 } 112802 114124 }else{ 112803 - int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR); 112804 114125 char *aMerge = aDoclist; 112805 114126 int nMerge = nDoclist; 112806 114127 int iOut; 112807 114128 112808 114129 for(iOut=0; iOut<SizeofArray(pTS->aaOutput); iOut++){ 112809 - char *aNew; 112810 - int nNew; 112811 114130 if( pTS->aaOutput[iOut]==0 ){ 112812 114131 assert( iOut>0 ); 112813 114132 pTS->aaOutput[iOut] = aMerge; 112814 114133 pTS->anOutput[iOut] = nMerge; 112815 114134 break; 112816 - } 112817 - 112818 - nNew = nMerge + pTS->anOutput[iOut]; 112819 - aNew = sqlite3_malloc(nNew); 112820 - if( !aNew ){ 112821 - if( aMerge!=aDoclist ){ 112822 - sqlite3_free(aMerge); 112823 - } 112824 - return SQLITE_NOMEM; 112825 - } 112826 - fts3DoclistMerge(mergetype, 0, 0, aNew, &nNew, 112827 - pTS->aaOutput[iOut], pTS->anOutput[iOut], aMerge, nMerge, 0 112828 - ); 112829 - 112830 - if( iOut>0 ) sqlite3_free(aMerge); 112831 - sqlite3_free(pTS->aaOutput[iOut]); 112832 - pTS->aaOutput[iOut] = 0; 112833 - 112834 - aMerge = aNew; 112835 - nMerge = nNew; 112836 - if( (iOut+1)==SizeofArray(pTS->aaOutput) ){ 112837 - pTS->aaOutput[iOut] = aMerge; 112838 - pTS->anOutput[iOut] = nMerge; 112839 - } 112840 - } 112841 - } 112842 - return SQLITE_OK; 112843 -} 112844 - 112845 -static int fts3DeferredTermSelect( 112846 - Fts3DeferredToken *pToken, /* Phrase token */ 112847 - int isTermPos, /* True to include positions */ 112848 - int *pnOut, /* OUT: Size of list */ 112849 - char **ppOut /* OUT: Body of list */ 112850 -){ 112851 - char *aSource; 112852 - int nSource; 112853 - 112854 - aSource = sqlite3Fts3DeferredDoclist(pToken, &nSource); 112855 - if( !aSource ){ 112856 - *pnOut = 0; 112857 - *ppOut = 0; 112858 - }else if( isTermPos ){ 112859 - *ppOut = sqlite3_malloc(nSource); 112860 - if( !*ppOut ) return SQLITE_NOMEM; 112861 - memcpy(*ppOut, aSource, nSource); 112862 - *pnOut = nSource; 112863 - }else{ 112864 - sqlite3_int64 docid; 112865 - *pnOut = sqlite3Fts3GetVarint(aSource, &docid); 112866 - *ppOut = sqlite3_malloc(*pnOut); 112867 - if( !*ppOut ) return SQLITE_NOMEM; 112868 - sqlite3Fts3PutVarint(*ppOut, docid); 112869 - } 112870 - 112871 - return SQLITE_OK; 112872 -} 112873 - 112874 -SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor( 114135 + }else{ 114136 + char *aNew; 114137 + int nNew; 114138 + 114139 + int rc = fts3DoclistOrMerge(p->bDescIdx, aMerge, nMerge, 114140 + pTS->aaOutput[iOut], pTS->anOutput[iOut], &aNew, &nNew 114141 + ); 114142 + if( rc!=SQLITE_OK ){ 114143 + if( aMerge!=aDoclist ) sqlite3_free(aMerge); 114144 + return rc; 114145 + } 114146 + 114147 + if( aMerge!=aDoclist ) sqlite3_free(aMerge); 114148 + sqlite3_free(pTS->aaOutput[iOut]); 114149 + pTS->aaOutput[iOut] = 0; 114150 + 114151 + aMerge = aNew; 114152 + nMerge = nNew; 114153 + if( (iOut+1)==SizeofArray(pTS->aaOutput) ){ 114154 + pTS->aaOutput[iOut] = aMerge; 114155 + pTS->anOutput[iOut] = nMerge; 114156 + } 114157 + } 114158 + } 114159 + } 114160 + return SQLITE_OK; 114161 +} 114162 + 114163 +/* 114164 +** Append SegReader object pNew to the end of the pCsr->apSegment[] array. 114165 +*/ 114166 +static int fts3SegReaderCursorAppend( 114167 + Fts3MultiSegReader *pCsr, 114168 + Fts3SegReader *pNew 114169 +){ 114170 + if( (pCsr->nSegment%16)==0 ){ 114171 + Fts3SegReader **apNew; 114172 + int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*); 114173 + apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte); 114174 + if( !apNew ){ 114175 + sqlite3Fts3SegReaderFree(pNew); 114176 + return SQLITE_NOMEM; 114177 + } 114178 + pCsr->apSegment = apNew; 114179 + } 114180 + pCsr->apSegment[pCsr->nSegment++] = pNew; 114181 + return SQLITE_OK; 114182 +} 114183 + 114184 +static int fts3SegReaderCursor( 112875 114185 Fts3Table *p, /* FTS3 table handle */ 114186 + int iIndex, /* Index to search (from 0 to p->nIndex-1) */ 112876 114187 int iLevel, /* Level of segments to scan */ 112877 114188 const char *zTerm, /* Term to query for */ 112878 114189 int nTerm, /* Size of zTerm in bytes */ 112879 114190 int isPrefix, /* True for a prefix search */ 112880 114191 int isScan, /* True to scan from zTerm to EOF */ 112881 - Fts3SegReaderCursor *pCsr /* Cursor object to populate */ 114192 + Fts3MultiSegReader *pCsr /* Cursor object to populate */ 112882 114193 ){ 112883 114194 int rc = SQLITE_OK; 112884 114195 int rc2; 112885 - int iAge = 0; 112886 114196 sqlite3_stmt *pStmt = 0; 112887 - Fts3SegReader *pPending = 0; 112888 114197 112889 - assert( iLevel==FTS3_SEGCURSOR_ALL 112890 - || iLevel==FTS3_SEGCURSOR_PENDING 112891 - || iLevel>=0 112892 - ); 112893 - assert( FTS3_SEGCURSOR_PENDING<0 ); 112894 - assert( FTS3_SEGCURSOR_ALL<0 ); 112895 - assert( iLevel==FTS3_SEGCURSOR_ALL || (zTerm==0 && isPrefix==1) ); 112896 - assert( isPrefix==0 || isScan==0 ); 112897 - 112898 - 112899 - memset(pCsr, 0, sizeof(Fts3SegReaderCursor)); 112900 - 112901 - /* If iLevel is less than 0, include a seg-reader for the pending-terms. */ 112902 - assert( isScan==0 || fts3HashCount(&p->pendingTerms)==0 ); 112903 - if( iLevel<0 && isScan==0 ){ 112904 - rc = sqlite3Fts3SegReaderPending(p, zTerm, nTerm, isPrefix, &pPending); 112905 - if( rc==SQLITE_OK && pPending ){ 112906 - int nByte = (sizeof(Fts3SegReader *) * 16); 112907 - pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc(nByte); 112908 - if( pCsr->apSegment==0 ){ 112909 - rc = SQLITE_NOMEM; 112910 - }else{ 112911 - pCsr->apSegment[0] = pPending; 112912 - pCsr->nSegment = 1; 112913 - pPending = 0; 112914 - } 114198 + /* If iLevel is less than 0 and this is not a scan, include a seg-reader 114199 + ** for the pending-terms. If this is a scan, then this call must be being 114200 + ** made by an fts4aux module, not an FTS table. In this case calling 114201 + ** Fts3SegReaderPending might segfault, as the data structures used by 114202 + ** fts4aux are not completely populated. So it's easiest to filter these 114203 + ** calls out here. */ 114204 + if( iLevel<0 && p->aIndex ){ 114205 + Fts3SegReader *pSeg = 0; 114206 + rc = sqlite3Fts3SegReaderPending(p, iIndex, zTerm, nTerm, isPrefix, &pSeg); 114207 + if( rc==SQLITE_OK && pSeg ){ 114208 + rc = fts3SegReaderCursorAppend(pCsr, pSeg); 112915 114209 } 112916 114210 } 112917 114211 112918 114212 if( iLevel!=FTS3_SEGCURSOR_PENDING ){ 112919 114213 if( rc==SQLITE_OK ){ 112920 - rc = sqlite3Fts3AllSegdirs(p, iLevel, &pStmt); 114214 + rc = sqlite3Fts3AllSegdirs(p, iIndex, iLevel, &pStmt); 112921 114215 } 114216 + 112922 114217 while( rc==SQLITE_OK && SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){ 114218 + Fts3SegReader *pSeg = 0; 112923 114219 112924 114220 /* Read the values returned by the SELECT into local variables. */ 112925 114221 sqlite3_int64 iStartBlock = sqlite3_column_int64(pStmt, 1); 112926 114222 sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2); 112927 114223 sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3); 112928 114224 int nRoot = sqlite3_column_bytes(pStmt, 4); 112929 114225 char const *zRoot = sqlite3_column_blob(pStmt, 4); 112930 114226 112931 - /* If nSegment is a multiple of 16 the array needs to be extended. */ 112932 - if( (pCsr->nSegment%16)==0 ){ 112933 - Fts3SegReader **apNew; 112934 - int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*); 112935 - apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte); 112936 - if( !apNew ){ 112937 - rc = SQLITE_NOMEM; 112938 - goto finished; 112939 - } 112940 - pCsr->apSegment = apNew; 112941 - } 112942 - 112943 114227 /* If zTerm is not NULL, and this segment is not stored entirely on its 112944 114228 ** root node, the range of leaves scanned can be reduced. Do this. */ 112945 114229 if( iStartBlock && zTerm ){ 112946 114230 sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0); 112947 114231 rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi); 112948 114232 if( rc!=SQLITE_OK ) goto finished; 112949 114233 if( isPrefix==0 && isScan==0 ) iLeavesEndBlock = iStartBlock; 112950 114234 } 112951 114235 112952 - rc = sqlite3Fts3SegReaderNew(iAge, iStartBlock, iLeavesEndBlock, 112953 - iEndBlock, zRoot, nRoot, &pCsr->apSegment[pCsr->nSegment] 114236 + rc = sqlite3Fts3SegReaderNew(pCsr->nSegment+1, 114237 + iStartBlock, iLeavesEndBlock, iEndBlock, zRoot, nRoot, &pSeg 112954 114238 ); 112955 114239 if( rc!=SQLITE_OK ) goto finished; 112956 - pCsr->nSegment++; 112957 - iAge++; 114240 + rc = fts3SegReaderCursorAppend(pCsr, pSeg); 112958 114241 } 112959 114242 } 112960 114243 112961 114244 finished: 112962 114245 rc2 = sqlite3_reset(pStmt); 112963 114246 if( rc==SQLITE_DONE ) rc = rc2; 112964 - sqlite3Fts3SegReaderFree(pPending); 112965 114247 112966 114248 return rc; 112967 114249 } 112968 114250 114251 +/* 114252 +** Set up a cursor object for iterating through a full-text index or a 114253 +** single level therein. 114254 +*/ 114255 +SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor( 114256 + Fts3Table *p, /* FTS3 table handle */ 114257 + int iIndex, /* Index to search (from 0 to p->nIndex-1) */ 114258 + int iLevel, /* Level of segments to scan */ 114259 + const char *zTerm, /* Term to query for */ 114260 + int nTerm, /* Size of zTerm in bytes */ 114261 + int isPrefix, /* True for a prefix search */ 114262 + int isScan, /* True to scan from zTerm to EOF */ 114263 + Fts3MultiSegReader *pCsr /* Cursor object to populate */ 114264 +){ 114265 + assert( iIndex>=0 && iIndex<p->nIndex ); 114266 + assert( iLevel==FTS3_SEGCURSOR_ALL 114267 + || iLevel==FTS3_SEGCURSOR_PENDING 114268 + || iLevel>=0 114269 + ); 114270 + assert( iLevel<FTS3_SEGDIR_MAXLEVEL ); 114271 + assert( FTS3_SEGCURSOR_ALL<0 && FTS3_SEGCURSOR_PENDING<0 ); 114272 + assert( isPrefix==0 || isScan==0 ); 112969 114273 112970 -static int fts3TermSegReaderCursor( 114274 + /* "isScan" is only set to true by the ft4aux module, an ordinary 114275 + ** full-text tables. */ 114276 + assert( isScan==0 || p->aIndex==0 ); 114277 + 114278 + memset(pCsr, 0, sizeof(Fts3MultiSegReader)); 114279 + 114280 + return fts3SegReaderCursor( 114281 + p, iIndex, iLevel, zTerm, nTerm, isPrefix, isScan, pCsr 114282 + ); 114283 +} 114284 + 114285 +static int fts3SegReaderCursorAddZero( 114286 + Fts3Table *p, 114287 + const char *zTerm, 114288 + int nTerm, 114289 + Fts3MultiSegReader *pCsr 114290 +){ 114291 + return fts3SegReaderCursor(p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0,pCsr); 114292 +} 114293 + 114294 + 114295 +SQLITE_PRIVATE int sqlite3Fts3TermSegReaderCursor( 112971 114296 Fts3Cursor *pCsr, /* Virtual table cursor handle */ 112972 114297 const char *zTerm, /* Term to query for */ 112973 114298 int nTerm, /* Size of zTerm in bytes */ 112974 114299 int isPrefix, /* True for a prefix search */ 112975 - Fts3SegReaderCursor **ppSegcsr /* OUT: Allocated seg-reader cursor */ 114300 + Fts3MultiSegReader **ppSegcsr /* OUT: Allocated seg-reader cursor */ 112976 114301 ){ 112977 - Fts3SegReaderCursor *pSegcsr; /* Object to allocate and return */ 114302 + Fts3MultiSegReader *pSegcsr; /* Object to allocate and return */ 112978 114303 int rc = SQLITE_NOMEM; /* Return code */ 112979 114304 112980 - pSegcsr = sqlite3_malloc(sizeof(Fts3SegReaderCursor)); 114305 + pSegcsr = sqlite3_malloc(sizeof(Fts3MultiSegReader)); 112981 114306 if( pSegcsr ){ 112982 - Fts3Table *p = (Fts3Table *)pCsr->base.pVtab; 112983 114307 int i; 112984 - int nCost = 0; 112985 - rc = sqlite3Fts3SegReaderCursor( 112986 - p, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr); 112987 - 112988 - for(i=0; rc==SQLITE_OK && i<pSegcsr->nSegment; i++){ 112989 - rc = sqlite3Fts3SegReaderCost(pCsr, pSegcsr->apSegment[i], &nCost); 114308 + int bFound = 0; /* True once an index has been found */ 114309 + Fts3Table *p = (Fts3Table *)pCsr->base.pVtab; 114310 + 114311 + if( isPrefix ){ 114312 + for(i=1; bFound==0 && i<p->nIndex; i++){ 114313 + if( p->aIndex[i].nPrefix==nTerm ){ 114314 + bFound = 1; 114315 + rc = sqlite3Fts3SegReaderCursor( 114316 + p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0, pSegcsr); 114317 + pSegcsr->bLookup = 1; 114318 + } 114319 + } 114320 + 114321 + for(i=1; bFound==0 && i<p->nIndex; i++){ 114322 + if( p->aIndex[i].nPrefix==nTerm+1 ){ 114323 + bFound = 1; 114324 + rc = sqlite3Fts3SegReaderCursor( 114325 + p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 1, 0, pSegcsr 114326 + ); 114327 + if( rc==SQLITE_OK ){ 114328 + rc = fts3SegReaderCursorAddZero(p, zTerm, nTerm, pSegcsr); 114329 + } 114330 + } 114331 + } 112990 114332 } 112991 - pSegcsr->nCost = nCost; 114333 + 114334 + if( bFound==0 ){ 114335 + rc = sqlite3Fts3SegReaderCursor( 114336 + p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr 114337 + ); 114338 + pSegcsr->bLookup = !isPrefix; 114339 + } 112992 114340 } 112993 114341 112994 114342 *ppSegcsr = pSegcsr; 112995 114343 return rc; 112996 114344 } 112997 114345 112998 -static void fts3SegReaderCursorFree(Fts3SegReaderCursor *pSegcsr){ 114346 +static void fts3SegReaderCursorFree(Fts3MultiSegReader *pSegcsr){ 112999 114347 sqlite3Fts3SegReaderFinish(pSegcsr); 113000 114348 sqlite3_free(pSegcsr); 113001 114349 } 113002 114350 113003 114351 /* 113004 114352 ** This function retreives the doclist for the specified term (or term 113005 114353 ** prefix) from the database. ................................................................................ 113016 114364 Fts3PhraseToken *pTok, /* Token to query for */ 113017 114365 int iColumn, /* Column to query (or -ve for all columns) */ 113018 114366 int isReqPos, /* True to include position lists in output */ 113019 114367 int *pnOut, /* OUT: Size of buffer at *ppOut */ 113020 114368 char **ppOut /* OUT: Malloced result buffer */ 113021 114369 ){ 113022 114370 int rc; /* Return code */ 113023 - Fts3SegReaderCursor *pSegcsr; /* Seg-reader cursor for this term */ 114371 + Fts3MultiSegReader *pSegcsr; /* Seg-reader cursor for this term */ 113024 114372 TermSelect tsc; /* Context object for fts3TermSelectCb() */ 113025 114373 Fts3SegFilter filter; /* Segment term filter configuration */ 113026 114374 113027 114375 pSegcsr = pTok->pSegcsr; 113028 114376 memset(&tsc, 0, sizeof(TermSelect)); 113029 114377 tsc.isReqPos = isReqPos; 113030 114378 ................................................................................ 113042 114390 ){ 113043 114391 rc = fts3TermSelectCb(p, (void *)&tsc, 113044 114392 pSegcsr->zTerm, pSegcsr->nTerm, pSegcsr->aDoclist, pSegcsr->nDoclist 113045 114393 ); 113046 114394 } 113047 114395 113048 114396 if( rc==SQLITE_OK ){ 113049 - rc = fts3TermSelectMerge(&tsc); 114397 + rc = fts3TermSelectMerge(p, &tsc); 113050 114398 } 113051 114399 if( rc==SQLITE_OK ){ 113052 114400 *ppOut = tsc.aaOutput[0]; 113053 114401 *pnOut = tsc.anOutput[0]; 113054 114402 }else{ 113055 114403 int i; 113056 114404 for(i=0; i<SizeofArray(tsc.aaOutput); i++){ ................................................................................ 113092 114440 } 113093 114441 } 113094 114442 } 113095 114443 113096 114444 return nDoc; 113097 114445 } 113098 114446 113099 -/* 113100 -** Call sqlite3Fts3DeferToken() for each token in the expression pExpr. 113101 -*/ 113102 -static int fts3DeferExpression(Fts3Cursor *pCsr, Fts3Expr *pExpr){ 113103 - int rc = SQLITE_OK; 113104 - if( pExpr ){ 113105 - rc = fts3DeferExpression(pCsr, pExpr->pLeft); 113106 - if( rc==SQLITE_OK ){ 113107 - rc = fts3DeferExpression(pCsr, pExpr->pRight); 113108 - } 113109 - if( pExpr->eType==FTSQUERY_PHRASE ){ 113110 - int iCol = pExpr->pPhrase->iColumn; 113111 - int i; 113112 - for(i=0; rc==SQLITE_OK && i<pExpr->pPhrase->nToken; i++){ 113113 - Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i]; 113114 - if( pToken->pDeferred==0 ){ 113115 - rc = sqlite3Fts3DeferToken(pCsr, pToken, iCol); 113116 - } 113117 - } 113118 - } 113119 - } 113120 - return rc; 113121 -} 113122 - 113123 -/* 113124 -** This function removes the position information from a doclist. When 113125 -** called, buffer aList (size *pnList bytes) contains a doclist that includes 113126 -** position information. This function removes the position information so 113127 -** that aList contains only docids, and adjusts *pnList to reflect the new 113128 -** (possibly reduced) size of the doclist. 113129 -*/ 113130 -static void fts3DoclistStripPositions( 113131 - char *aList, /* IN/OUT: Buffer containing doclist */ 113132 - int *pnList /* IN/OUT: Size of doclist in bytes */ 113133 -){ 113134 - if( aList ){ 113135 - char *aEnd = &aList[*pnList]; /* Pointer to one byte after EOF */ 113136 - char *p = aList; /* Input cursor */ 113137 - char *pOut = aList; /* Output cursor */ 113138 - 113139 - while( p<aEnd ){ 113140 - sqlite3_int64 delta; 113141 - p += sqlite3Fts3GetVarint(p, &delta); 113142 - fts3PoslistCopy(0, &p); 113143 - pOut += sqlite3Fts3PutVarint(pOut, delta); 113144 - } 113145 - 113146 - *pnList = (int)(pOut - aList); 113147 - } 113148 -} 113149 - 113150 -/* 113151 -** Return a DocList corresponding to the phrase *pPhrase. 113152 -** 113153 -** If this function returns SQLITE_OK, but *pnOut is set to a negative value, 113154 -** then no tokens in the phrase were looked up in the full-text index. This 113155 -** is only possible when this function is called from within xFilter(). The 113156 -** caller should assume that all documents match the phrase. The actual 113157 -** filtering will take place in xNext(). 113158 -*/ 113159 -static int fts3PhraseSelect( 113160 - Fts3Cursor *pCsr, /* Virtual table cursor handle */ 113161 - Fts3Phrase *pPhrase, /* Phrase to return a doclist for */ 113162 - int isReqPos, /* True if output should contain positions */ 113163 - char **paOut, /* OUT: Pointer to malloc'd result buffer */ 113164 - int *pnOut /* OUT: Size of buffer at *paOut */ 113165 -){ 113166 - char *pOut = 0; 113167 - int nOut = 0; 113168 - int rc = SQLITE_OK; 113169 - int ii; 113170 - int iCol = pPhrase->iColumn; 113171 - int isTermPos = (pPhrase->nToken>1 || isReqPos); 113172 - Fts3Table *p = (Fts3Table *)pCsr->base.pVtab; 113173 - int isFirst = 1; 113174 - 113175 - int iPrevTok = 0; 113176 - int nDoc = 0; 113177 - 113178 - /* If this is an xFilter() evaluation, create a segment-reader for each 113179 - ** phrase token. Or, if this is an xNext() or snippet/offsets/matchinfo 113180 - ** evaluation, only create segment-readers if there are no Fts3DeferredToken 113181 - ** objects attached to the phrase-tokens. 113182 - */ 113183 - for(ii=0; ii<pPhrase->nToken; ii++){ 113184 - Fts3PhraseToken *pTok = &pPhrase->aToken[ii]; 113185 - if( pTok->pSegcsr==0 ){ 113186 - if( (pCsr->eEvalmode==FTS3_EVAL_FILTER) 113187 - || (pCsr->eEvalmode==FTS3_EVAL_NEXT && pCsr->pDeferred==0) 113188 - || (pCsr->eEvalmode==FTS3_EVAL_MATCHINFO && pTok->bFulltext) 113189 - ){ 113190 - rc = fts3TermSegReaderCursor( 113191 - pCsr, pTok->z, pTok->n, pTok->isPrefix, &pTok->pSegcsr 113192 - ); 113193 - if( rc!=SQLITE_OK ) return rc; 113194 - } 113195 - } 113196 - } 113197 - 113198 - for(ii=0; ii<pPhrase->nToken; ii++){ 113199 - Fts3PhraseToken *pTok; /* Token to find doclist for */ 113200 - int iTok = 0; /* The token being queried this iteration */ 113201 - char *pList = 0; /* Pointer to token doclist */ 113202 - int nList = 0; /* Size of buffer at pList */ 113203 - 113204 - /* Select a token to process. If this is an xFilter() call, then tokens 113205 - ** are processed in order from least to most costly. Otherwise, tokens 113206 - ** are processed in the order in which they occur in the phrase. 113207 - */ 113208 - if( pCsr->eEvalmode==FTS3_EVAL_MATCHINFO ){ 113209 - assert( isReqPos ); 113210 - iTok = ii; 113211 - pTok = &pPhrase->aToken[iTok]; 113212 - if( pTok->bFulltext==0 ) continue; 113213 - }else if( pCsr->eEvalmode==FTS3_EVAL_NEXT || isReqPos ){ 113214 - iTok = ii; 113215 - pTok = &pPhrase->aToken[iTok]; 113216 - }else{ 113217 - int nMinCost = 0x7FFFFFFF; 113218 - int jj; 113219 - 113220 - /* Find the remaining token with the lowest cost. */ 113221 - for(jj=0; jj<pPhrase->nToken; jj++){ 113222 - Fts3SegReaderCursor *pSegcsr = pPhrase->aToken[jj].pSegcsr; 113223 - if( pSegcsr && pSegcsr->nCost<nMinCost ){ 113224 - iTok = jj; 113225 - nMinCost = pSegcsr->nCost; 113226 - } 113227 - } 113228 - pTok = &pPhrase->aToken[iTok]; 113229 - 113230 - /* This branch is taken if it is determined that loading the doclist 113231 - ** for the next token would require more IO than loading all documents 113232 - ** currently identified by doclist pOut/nOut. No further doclists will 113233 - ** be loaded from the full-text index for this phrase. 113234 - */ 113235 - if( nMinCost>nDoc && ii>0 ){ 113236 - rc = fts3DeferExpression(pCsr, pCsr->pExpr); 113237 - break; 113238 - } 113239 - } 113240 - 113241 - if( pCsr->eEvalmode==FTS3_EVAL_NEXT && pTok->pDeferred ){ 113242 - rc = fts3DeferredTermSelect(pTok->pDeferred, isTermPos, &nList, &pList); 113243 - }else{ 113244 - if( pTok->pSegcsr ){ 113245 - rc = fts3TermSelect(p, pTok, iCol, isTermPos, &nList, &pList); 113246 - } 113247 - pTok->bFulltext = 1; 113248 - } 113249 - assert( rc!=SQLITE_OK || pCsr->eEvalmode || pTok->pSegcsr==0 ); 113250 - if( rc!=SQLITE_OK ) break; 113251 - 113252 - if( isFirst ){ 113253 - pOut = pList; 113254 - nOut = nList; 113255 - if( pCsr->eEvalmode==FTS3_EVAL_FILTER && pPhrase->nToken>1 ){ 113256 - nDoc = fts3DoclistCountDocids(1, pOut, nOut); 113257 - } 113258 - isFirst = 0; 113259 - iPrevTok = iTok; 113260 - }else{ 113261 - /* Merge the new term list and the current output. */ 113262 - char *aLeft, *aRight; 113263 - int nLeft, nRight; 113264 - int nDist; 113265 - int mt; 113266 - 113267 - /* If this is the final token of the phrase, and positions were not 113268 - ** requested by the caller, use MERGE_PHRASE instead of POS_PHRASE. 113269 - ** This drops the position information from the output list. 113270 - */ 113271 - mt = MERGE_POS_PHRASE; 113272 - if( ii==pPhrase->nToken-1 && !isReqPos ) mt = MERGE_PHRASE; 113273 - 113274 - assert( iPrevTok!=iTok ); 113275 - if( iPrevTok<iTok ){ 113276 - aLeft = pOut; 113277 - nLeft = nOut; 113278 - aRight = pList; 113279 - nRight = nList; 113280 - nDist = iTok-iPrevTok; 113281 - iPrevTok = iTok; 113282 - }else{ 113283 - aRight = pOut; 113284 - nRight = nOut; 113285 - aLeft = pList; 113286 - nLeft = nList; 113287 - nDist = iPrevTok-iTok; 113288 - } 113289 - pOut = aRight; 113290 - fts3DoclistMerge( 113291 - mt, nDist, 0, pOut, &nOut, aLeft, nLeft, aRight, nRight, &nDoc 113292 - ); 113293 - sqlite3_free(aLeft); 113294 - } 113295 - assert( nOut==0 || pOut!=0 ); 113296 - } 113297 - 113298 - if( rc==SQLITE_OK ){ 113299 - if( ii!=pPhrase->nToken ){ 113300 - assert( pCsr->eEvalmode==FTS3_EVAL_FILTER && isReqPos==0 ); 113301 - fts3DoclistStripPositions(pOut, &nOut); 113302 - } 113303 - *paOut = pOut; 113304 - *pnOut = nOut; 113305 - }else{ 113306 - sqlite3_free(pOut); 113307 - } 113308 - return rc; 113309 -} 113310 - 113311 -/* 113312 -** This function merges two doclists according to the requirements of a 113313 -** NEAR operator. 113314 -** 113315 -** Both input doclists must include position information. The output doclist 113316 -** includes position information if the first argument to this function 113317 -** is MERGE_POS_NEAR, or does not if it is MERGE_NEAR. 113318 -*/ 113319 -static int fts3NearMerge( 113320 - int mergetype, /* MERGE_POS_NEAR or MERGE_NEAR */ 113321 - int nNear, /* Parameter to NEAR operator */ 113322 - int nTokenLeft, /* Number of tokens in LHS phrase arg */ 113323 - char *aLeft, /* Doclist for LHS (incl. positions) */ 113324 - int nLeft, /* Size of LHS doclist in bytes */ 113325 - int nTokenRight, /* As nTokenLeft */ 113326 - char *aRight, /* As aLeft */ 113327 - int nRight, /* As nRight */ 113328 - char **paOut, /* OUT: Results of merge (malloced) */ 113329 - int *pnOut /* OUT: Sized of output buffer */ 113330 -){ 113331 - char *aOut; /* Buffer to write output doclist to */ 113332 - int rc; /* Return code */ 113333 - 113334 - assert( mergetype==MERGE_POS_NEAR || MERGE_NEAR ); 113335 - 113336 - aOut = sqlite3_malloc(nLeft+nRight+1); 113337 - if( aOut==0 ){ 113338 - rc = SQLITE_NOMEM; 113339 - }else{ 113340 - rc = fts3DoclistMerge(mergetype, nNear+nTokenRight, nNear+nTokenLeft, 113341 - aOut, pnOut, aLeft, nLeft, aRight, nRight, 0 113342 - ); 113343 - if( rc!=SQLITE_OK ){ 113344 - sqlite3_free(aOut); 113345 - aOut = 0; 113346 - } 113347 - } 113348 - 113349 - *paOut = aOut; 113350 - return rc; 113351 -} 113352 - 113353 -/* 113354 -** This function is used as part of the processing for the snippet() and 113355 -** offsets() functions. 113356 -** 113357 -** Both pLeft and pRight are expression nodes of type FTSQUERY_PHRASE. Both 113358 -** have their respective doclists (including position information) loaded 113359 -** in Fts3Expr.aDoclist/nDoclist. This function removes all entries from 113360 -** each doclist that are not within nNear tokens of a corresponding entry 113361 -** in the other doclist. 113362 -*/ 113363 -SQLITE_PRIVATE int sqlite3Fts3ExprNearTrim(Fts3Expr *pLeft, Fts3Expr *pRight, int nNear){ 113364 - int rc; /* Return code */ 113365 - 113366 - assert( pLeft->eType==FTSQUERY_PHRASE ); 113367 - assert( pRight->eType==FTSQUERY_PHRASE ); 113368 - assert( pLeft->isLoaded && pRight->isLoaded ); 113369 - 113370 - if( pLeft->aDoclist==0 || pRight->aDoclist==0 ){ 113371 - sqlite3_free(pLeft->aDoclist); 113372 - sqlite3_free(pRight->aDoclist); 113373 - pRight->aDoclist = 0; 113374 - pLeft->aDoclist = 0; 113375 - rc = SQLITE_OK; 113376 - }else{ 113377 - char *aOut; /* Buffer in which to assemble new doclist */ 113378 - int nOut; /* Size of buffer aOut in bytes */ 113379 - 113380 - rc = fts3NearMerge(MERGE_POS_NEAR, nNear, 113381 - pLeft->pPhrase->nToken, pLeft->aDoclist, pLeft->nDoclist, 113382 - pRight->pPhrase->nToken, pRight->aDoclist, pRight->nDoclist, 113383 - &aOut, &nOut 113384 - ); 113385 - if( rc!=SQLITE_OK ) return rc; 113386 - sqlite3_free(pRight->aDoclist); 113387 - pRight->aDoclist = aOut; 113388 - pRight->nDoclist = nOut; 113389 - 113390 - rc = fts3NearMerge(MERGE_POS_NEAR, nNear, 113391 - pRight->pPhrase->nToken, pRight->aDoclist, pRight->nDoclist, 113392 - pLeft->pPhrase->nToken, pLeft->aDoclist, pLeft->nDoclist, 113393 - &aOut, &nOut 113394 - ); 113395 - sqlite3_free(pLeft->aDoclist); 113396 - pLeft->aDoclist = aOut; 113397 - pLeft->nDoclist = nOut; 113398 - } 113399 - return rc; 113400 -} 113401 - 113402 - 113403 -/* 113404 -** Allocate an Fts3SegReaderArray for each token in the expression pExpr. 113405 -** The allocated objects are stored in the Fts3PhraseToken.pArray member 113406 -** variables of each token structure. 113407 -*/ 113408 -static int fts3ExprAllocateSegReaders( 113409 - Fts3Cursor *pCsr, /* FTS3 table */ 113410 - Fts3Expr *pExpr, /* Expression to create seg-readers for */ 113411 - int *pnExpr /* OUT: Number of AND'd expressions */ 113412 -){ 113413 - int rc = SQLITE_OK; /* Return code */ 113414 - 113415 - assert( pCsr->eEvalmode==FTS3_EVAL_FILTER ); 113416 - if( pnExpr && pExpr->eType!=FTSQUERY_AND ){ 113417 - (*pnExpr)++; 113418 - pnExpr = 0; 113419 - } 113420 - 113421 - if( pExpr->eType==FTSQUERY_PHRASE ){ 113422 - Fts3Phrase *pPhrase = pExpr->pPhrase; 113423 - int ii; 113424 - 113425 - for(ii=0; rc==SQLITE_OK && ii<pPhrase->nToken; ii++){ 113426 - Fts3PhraseToken *pTok = &pPhrase->aToken[ii]; 113427 - if( pTok->pSegcsr==0 ){ 113428 - rc = fts3TermSegReaderCursor( 113429 - pCsr, pTok->z, pTok->n, pTok->isPrefix, &pTok->pSegcsr 113430 - ); 113431 - } 113432 - } 113433 - }else{ 113434 - rc = fts3ExprAllocateSegReaders(pCsr, pExpr->pLeft, pnExpr); 113435 - if( rc==SQLITE_OK ){ 113436 - rc = fts3ExprAllocateSegReaders(pCsr, pExpr->pRight, pnExpr); 113437 - } 113438 - } 113439 - return rc; 113440 -} 113441 - 113442 -/* 113443 -** Free the Fts3SegReaderArray objects associated with each token in the 113444 -** expression pExpr. In other words, this function frees the resources 113445 -** allocated by fts3ExprAllocateSegReaders(). 113446 -*/ 113447 -static void fts3ExprFreeSegReaders(Fts3Expr *pExpr){ 113448 - if( pExpr ){ 113449 - Fts3Phrase *pPhrase = pExpr->pPhrase; 113450 - if( pPhrase ){ 113451 - int kk; 113452 - for(kk=0; kk<pPhrase->nToken; kk++){ 113453 - fts3SegReaderCursorFree(pPhrase->aToken[kk].pSegcsr); 113454 - pPhrase->aToken[kk].pSegcsr = 0; 113455 - } 113456 - } 113457 - fts3ExprFreeSegReaders(pExpr->pLeft); 113458 - fts3ExprFreeSegReaders(pExpr->pRight); 113459 - } 113460 -} 113461 - 113462 -/* 113463 -** Return the sum of the costs of all tokens in the expression pExpr. This 113464 -** function must be called after Fts3SegReaderArrays have been allocated 113465 -** for all tokens using fts3ExprAllocateSegReaders(). 113466 -*/ 113467 -static int fts3ExprCost(Fts3Expr *pExpr){ 113468 - int nCost; /* Return value */ 113469 - if( pExpr->eType==FTSQUERY_PHRASE ){ 113470 - Fts3Phrase *pPhrase = pExpr->pPhrase; 113471 - int ii; 113472 - nCost = 0; 113473 - for(ii=0; ii<pPhrase->nToken; ii++){ 113474 - Fts3SegReaderCursor *pSegcsr = pPhrase->aToken[ii].pSegcsr; 113475 - if( pSegcsr ) nCost += pSegcsr->nCost; 113476 - } 113477 - }else{ 113478 - nCost = fts3ExprCost(pExpr->pLeft) + fts3ExprCost(pExpr->pRight); 113479 - } 113480 - return nCost; 113481 -} 113482 - 113483 -/* 113484 -** The following is a helper function (and type) for fts3EvalExpr(). It 113485 -** must be called after Fts3SegReaders have been allocated for every token 113486 -** in the expression. See the context it is called from in fts3EvalExpr() 113487 -** for further explanation. 113488 -*/ 113489 -typedef struct ExprAndCost ExprAndCost; 113490 -struct ExprAndCost { 113491 - Fts3Expr *pExpr; 113492 - int nCost; 113493 -}; 113494 -static void fts3ExprAssignCosts( 113495 - Fts3Expr *pExpr, /* Expression to create seg-readers for */ 113496 - ExprAndCost **ppExprCost /* OUT: Write to *ppExprCost */ 113497 -){ 113498 - if( pExpr->eType==FTSQUERY_AND ){ 113499 - fts3ExprAssignCosts(pExpr->pLeft, ppExprCost); 113500 - fts3ExprAssignCosts(pExpr->pRight, ppExprCost); 113501 - }else{ 113502 - (*ppExprCost)->pExpr = pExpr; 113503 - (*ppExprCost)->nCost = fts3ExprCost(pExpr); 113504 - (*ppExprCost)++; 113505 - } 113506 -} 113507 - 113508 -/* 113509 -** Evaluate the full-text expression pExpr against FTS3 table pTab. Store 113510 -** the resulting doclist in *paOut and *pnOut. This routine mallocs for 113511 -** the space needed to store the output. The caller is responsible for 113512 -** freeing the space when it has finished. 113513 -** 113514 -** This function is called in two distinct contexts: 113515 -** 113516 -** * From within the virtual table xFilter() method. In this case, the 113517 -** output doclist contains entries for all rows in the table, based on 113518 -** data read from the full-text index. 113519 -** 113520 -** In this case, if the query expression contains one or more tokens that 113521 -** are very common, then the returned doclist may contain a superset of 113522 -** the documents that actually match the expression. 113523 -** 113524 -** * From within the virtual table xNext() method. This call is only made 113525 -** if the call from within xFilter() found that there were very common 113526 -** tokens in the query expression and did return a superset of the 113527 -** matching documents. In this case the returned doclist contains only 113528 -** entries that correspond to the current row of the table. Instead of 113529 -** reading the data for each token from the full-text index, the data is 113530 -** already available in-memory in the Fts3PhraseToken.pDeferred structures. 113531 -** See fts3EvalDeferred() for how it gets there. 113532 -** 113533 -** In the first case above, Fts3Cursor.doDeferred==0. In the second (if it is 113534 -** required) Fts3Cursor.doDeferred==1. 113535 -** 113536 -** If the SQLite invokes the snippet(), offsets() or matchinfo() function 113537 -** as part of a SELECT on an FTS3 table, this function is called on each 113538 -** individual phrase expression in the query. If there were very common tokens 113539 -** found in the xFilter() call, then this function is called once for phrase 113540 -** for each row visited, and the returned doclist contains entries for the 113541 -** current row only. Otherwise, if there were no very common tokens, then this 113542 -** function is called once only for each phrase in the query and the returned 113543 -** doclist contains entries for all rows of the table. 113544 -** 113545 -** Fts3Cursor.doDeferred==1 when this function is called on phrases as a 113546 -** result of a snippet(), offsets() or matchinfo() invocation. 113547 -*/ 113548 -static int fts3EvalExpr( 113549 - Fts3Cursor *p, /* Virtual table cursor handle */ 113550 - Fts3Expr *pExpr, /* Parsed fts3 expression */ 113551 - char **paOut, /* OUT: Pointer to malloc'd result buffer */ 113552 - int *pnOut, /* OUT: Size of buffer at *paOut */ 113553 - int isReqPos /* Require positions in output buffer */ 113554 -){ 113555 - int rc = SQLITE_OK; /* Return code */ 113556 - 113557 - /* Zero the output parameters. */ 113558 - *paOut = 0; 113559 - *pnOut = 0; 113560 - 113561 - if( pExpr ){ 113562 - assert( pExpr->eType==FTSQUERY_NEAR || pExpr->eType==FTSQUERY_OR 113563 - || pExpr->eType==FTSQUERY_AND || pExpr->eType==FTSQUERY_NOT 113564 - || pExpr->eType==FTSQUERY_PHRASE 113565 - ); 113566 - assert( pExpr->eType==FTSQUERY_PHRASE || isReqPos==0 ); 113567 - 113568 - if( pExpr->eType==FTSQUERY_PHRASE ){ 113569 - rc = fts3PhraseSelect(p, pExpr->pPhrase, 113570 - isReqPos || (pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR), 113571 - paOut, pnOut 113572 - ); 113573 - fts3ExprFreeSegReaders(pExpr); 113574 - }else if( p->eEvalmode==FTS3_EVAL_FILTER && pExpr->eType==FTSQUERY_AND ){ 113575 - ExprAndCost *aExpr = 0; /* Array of AND'd expressions and costs */ 113576 - int nExpr = 0; /* Size of aExpr[] */ 113577 - char *aRet = 0; /* Doclist to return to caller */ 113578 - int nRet = 0; /* Length of aRet[] in bytes */ 113579 - int nDoc = 0x7FFFFFFF; 113580 - 113581 - assert( !isReqPos ); 113582 - 113583 - rc = fts3ExprAllocateSegReaders(p, pExpr, &nExpr); 113584 - if( rc==SQLITE_OK ){ 113585 - assert( nExpr>1 ); 113586 - aExpr = sqlite3_malloc(sizeof(ExprAndCost) * nExpr); 113587 - if( !aExpr ) rc = SQLITE_NOMEM; 113588 - } 113589 - if( rc==SQLITE_OK ){ 113590 - int ii; /* Used to iterate through expressions */ 113591 - 113592 - fts3ExprAssignCosts(pExpr, &aExpr); 113593 - aExpr -= nExpr; 113594 - for(ii=0; ii<nExpr; ii++){ 113595 - char *aNew; 113596 - int nNew; 113597 - int jj; 113598 - ExprAndCost *pBest = 0; 113599 - 113600 - for(jj=0; jj<nExpr; jj++){ 113601 - ExprAndCost *pCand = &aExpr[jj]; 113602 - if( pCand->pExpr && (pBest==0 || pCand->nCost<pBest->nCost) ){ 113603 - pBest = pCand; 113604 - } 113605 - } 113606 - 113607 - if( pBest->nCost>nDoc ){ 113608 - rc = fts3DeferExpression(p, p->pExpr); 113609 - break; 113610 - }else{ 113611 - rc = fts3EvalExpr(p, pBest->pExpr, &aNew, &nNew, 0); 113612 - if( rc!=SQLITE_OK ) break; 113613 - pBest->pExpr = 0; 113614 - if( ii==0 ){ 113615 - aRet = aNew; 113616 - nRet = nNew; 113617 - nDoc = fts3DoclistCountDocids(0, aRet, nRet); 113618 - }else{ 113619 - fts3DoclistMerge( 113620 - MERGE_AND, 0, 0, aRet, &nRet, aRet, nRet, aNew, nNew, &nDoc 113621 - ); 113622 - sqlite3_free(aNew); 113623 - } 113624 - } 113625 - } 113626 - } 113627 - 113628 - if( rc==SQLITE_OK ){ 113629 - *paOut = aRet; 113630 - *pnOut = nRet; 113631 - }else{ 113632 - assert( *paOut==0 ); 113633 - sqlite3_free(aRet); 113634 - } 113635 - sqlite3_free(aExpr); 113636 - fts3ExprFreeSegReaders(pExpr); 113637 - 113638 - }else{ 113639 - char *aLeft; 113640 - char *aRight; 113641 - int nLeft; 113642 - int nRight; 113643 - 113644 - assert( pExpr->eType==FTSQUERY_NEAR 113645 - || pExpr->eType==FTSQUERY_OR 113646 - || pExpr->eType==FTSQUERY_NOT 113647 - || (pExpr->eType==FTSQUERY_AND && p->eEvalmode==FTS3_EVAL_NEXT) 113648 - ); 113649 - 113650 - if( 0==(rc = fts3EvalExpr(p, pExpr->pRight, &aRight, &nRight, isReqPos)) 113651 - && 0==(rc = fts3EvalExpr(p, pExpr->pLeft, &aLeft, &nLeft, isReqPos)) 113652 - ){ 113653 - switch( pExpr->eType ){ 113654 - case FTSQUERY_NEAR: { 113655 - Fts3Expr *pLeft; 113656 - Fts3Expr *pRight; 113657 - int mergetype = MERGE_NEAR; 113658 - if( pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR ){ 113659 - mergetype = MERGE_POS_NEAR; 113660 - } 113661 - pLeft = pExpr->pLeft; 113662 - while( pLeft->eType==FTSQUERY_NEAR ){ 113663 - pLeft=pLeft->pRight; 113664 - } 113665 - pRight = pExpr->pRight; 113666 - assert( pRight->eType==FTSQUERY_PHRASE ); 113667 - assert( pLeft->eType==FTSQUERY_PHRASE ); 113668 - 113669 - rc = fts3NearMerge(mergetype, pExpr->nNear, 113670 - pLeft->pPhrase->nToken, aLeft, nLeft, 113671 - pRight->pPhrase->nToken, aRight, nRight, 113672 - paOut, pnOut 113673 - ); 113674 - sqlite3_free(aLeft); 113675 - break; 113676 - } 113677 - 113678 - case FTSQUERY_OR: { 113679 - /* Allocate a buffer for the output. The maximum size is the 113680 - ** sum of the sizes of the two input buffers. The +1 term is 113681 - ** so that a buffer of zero bytes is never allocated - this can 113682 - ** cause fts3DoclistMerge() to incorrectly return SQLITE_NOMEM. 113683 - */ 113684 - char *aBuffer = sqlite3_malloc(nRight+nLeft+1); 113685 - rc = fts3DoclistMerge(MERGE_OR, 0, 0, aBuffer, pnOut, 113686 - aLeft, nLeft, aRight, nRight, 0 113687 - ); 113688 - *paOut = aBuffer; 113689 - sqlite3_free(aLeft); 113690 - break; 113691 - } 113692 - 113693 - default: { 113694 - assert( FTSQUERY_NOT==MERGE_NOT && FTSQUERY_AND==MERGE_AND ); 113695 - fts3DoclistMerge(pExpr->eType, 0, 0, aLeft, pnOut, 113696 - aLeft, nLeft, aRight, nRight, 0 113697 - ); 113698 - *paOut = aLeft; 113699 - break; 113700 - } 113701 - } 113702 - } 113703 - sqlite3_free(aRight); 113704 - } 113705 - } 113706 - 113707 - assert( rc==SQLITE_OK || *paOut==0 ); 113708 - return rc; 113709 -} 113710 - 113711 -/* 113712 -** This function is called from within xNext() for each row visited by 113713 -** an FTS3 query. If evaluating the FTS3 query expression within xFilter() 113714 -** was able to determine the exact set of matching rows, this function sets 113715 -** *pbRes to true and returns SQLITE_IO immediately. 113716 -** 113717 -** Otherwise, if evaluating the query expression within xFilter() returned a 113718 -** superset of the matching documents instead of an exact set (this happens 113719 -** when the query includes very common tokens and it is deemed too expensive to 113720 -** load their doclists from disk), this function tests if the current row 113721 -** really does match the FTS3 query. 113722 -** 113723 -** If an error occurs, an SQLite error code is returned. Otherwise, SQLITE_OK 113724 -** is returned and *pbRes is set to true if the current row matches the 113725 -** FTS3 query (and should be included in the results returned to SQLite), or 113726 -** false otherwise. 113727 -*/ 113728 -static int fts3EvalDeferred( 113729 - Fts3Cursor *pCsr, /* FTS3 cursor pointing at row to test */ 113730 - int *pbRes /* OUT: Set to true if row is a match */ 113731 -){ 113732 - int rc = SQLITE_OK; 113733 - if( pCsr->pDeferred==0 ){ 113734 - *pbRes = 1; 113735 - }else{ 113736 - rc = fts3CursorSeek(0, pCsr); 113737 - if( rc==SQLITE_OK ){ 113738 - sqlite3Fts3FreeDeferredDoclists(pCsr); 113739 - rc = sqlite3Fts3CacheDeferredDoclists(pCsr); 113740 - } 113741 - if( rc==SQLITE_OK ){ 113742 - char *a = 0; 113743 - int n = 0; 113744 - rc = fts3EvalExpr(pCsr, pCsr->pExpr, &a, &n, 0); 113745 - assert( n>=0 ); 113746 - *pbRes = (n>0); 113747 - sqlite3_free(a); 113748 - } 113749 - } 113750 - return rc; 113751 -} 113752 - 113753 114447 /* 113754 114448 ** Advance the cursor to the next row in the %_content table that 113755 114449 ** matches the search criteria. For a MATCH search, this will be 113756 114450 ** the next row that matches. For a full-table scan, this will be 113757 114451 ** simply the next row in the %_content table. For a docid lookup, 113758 114452 ** this routine simply sets the EOF flag. 113759 114453 ** 113760 114454 ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned 113761 114455 ** even if we reach end-of-file. The fts3EofMethod() will be called 113762 114456 ** subsequently to determine whether or not an EOF was hit. 113763 114457 */ 113764 114458 static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){ 113765 - int res; 113766 - int rc = SQLITE_OK; /* Return code */ 114459 + int rc; 113767 114460 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; 113768 - 113769 - pCsr->eEvalmode = FTS3_EVAL_NEXT; 113770 - do { 113771 - if( pCsr->aDoclist==0 ){ 113772 - if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){ 113773 - pCsr->isEof = 1; 113774 - rc = sqlite3_reset(pCsr->pStmt); 113775 - break; 113776 - } 114461 + if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){ 114462 + if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){ 114463 + pCsr->isEof = 1; 114464 + rc = sqlite3_reset(pCsr->pStmt); 114465 + }else{ 113777 114466 pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0); 113778 - }else{ 113779 - if( pCsr->pNextId>=&pCsr->aDoclist[pCsr->nDoclist] ){ 113780 - pCsr->isEof = 1; 113781 - break; 113782 - } 113783 - sqlite3_reset(pCsr->pStmt); 113784 - fts3GetDeltaVarint(&pCsr->pNextId, &pCsr->iPrevId); 113785 - pCsr->isRequireSeek = 1; 113786 - pCsr->isMatchinfoNeeded = 1; 114467 + rc = SQLITE_OK; 113787 114468 } 113788 - }while( SQLITE_OK==(rc = fts3EvalDeferred(pCsr, &res)) && res==0 ); 113789 - 114469 + }else{ 114470 + rc = sqlite3Fts3EvalNext((Fts3Cursor *)pCursor); 114471 + } 114472 + assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); 113790 114473 return rc; 113791 114474 } 113792 114475 113793 114476 /* 113794 114477 ** This is the xFilter interface for the virtual table. See 113795 114478 ** the virtual table xFilter method documentation for additional 113796 114479 ** information. ................................................................................ 113809 114492 static int fts3FilterMethod( 113810 114493 sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ 113811 114494 int idxNum, /* Strategy index */ 113812 114495 const char *idxStr, /* Unused */ 113813 114496 int nVal, /* Number of elements in apVal */ 113814 114497 sqlite3_value **apVal /* Arguments for the indexing scheme */ 113815 114498 ){ 113816 - const char *azSql[] = { 113817 - "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?", /* non-full-scan */ 113818 - "SELECT %s FROM %Q.'%q_content' AS x ", /* full-scan */ 113819 - }; 113820 - int rc; /* Return code */ 114499 + int rc; 113821 114500 char *zSql; /* SQL statement used to access %_content */ 113822 114501 Fts3Table *p = (Fts3Table *)pCursor->pVtab; 113823 114502 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; 113824 114503 113825 114504 UNUSED_PARAMETER(idxStr); 113826 114505 UNUSED_PARAMETER(nVal); 113827 114506 ................................................................................ 113832 114511 113833 114512 /* In case the cursor has been used before, clear it now. */ 113834 114513 sqlite3_finalize(pCsr->pStmt); 113835 114514 sqlite3_free(pCsr->aDoclist); 113836 114515 sqlite3Fts3ExprFree(pCsr->pExpr); 113837 114516 memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor)); 113838 114517 114518 + if( idxStr ){ 114519 + pCsr->bDesc = (idxStr[0]=='D'); 114520 + }else{ 114521 + pCsr->bDesc = p->bDescIdx; 114522 + } 114523 + pCsr->eSearch = (i16)idxNum; 114524 + 113839 114525 if( idxNum!=FTS3_DOCID_SEARCH && idxNum!=FTS3_FULLSCAN_SEARCH ){ 113840 114526 int iCol = idxNum-FTS3_FULLTEXT_SEARCH; 113841 114527 const char *zQuery = (const char *)sqlite3_value_text(apVal[0]); 113842 114528 113843 114529 if( zQuery==0 && sqlite3_value_type(apVal[0])!=SQLITE_NULL ){ 113844 114530 return SQLITE_NOMEM; 113845 114531 } 113846 114532 113847 114533 rc = sqlite3Fts3ExprParse(p->pTokenizer, p->azColumn, p->nColumn, 113848 114534 iCol, zQuery, -1, &pCsr->pExpr 113849 114535 ); 113850 114536 if( rc!=SQLITE_OK ){ 113851 114537 if( rc==SQLITE_ERROR ){ 113852 - p->base.zErrMsg = sqlite3_mprintf("malformed MATCH expression: [%s]", 113853 - zQuery); 114538 + static const char *zErr = "malformed MATCH expression: [%s]"; 114539 + p->base.zErrMsg = sqlite3_mprintf(zErr, zQuery); 113854 114540 } 113855 114541 return rc; 113856 114542 } 113857 114543 113858 114544 rc = sqlite3Fts3ReadLock(p); 113859 114545 if( rc!=SQLITE_OK ) return rc; 113860 114546 113861 - rc = fts3EvalExpr(pCsr, pCsr->pExpr, &pCsr->aDoclist, &pCsr->nDoclist, 0); 114547 + rc = sqlite3Fts3EvalStart(pCsr, pCsr->pExpr, 1); 114548 + 113862 114549 sqlite3Fts3SegmentsClose(p); 113863 114550 if( rc!=SQLITE_OK ) return rc; 113864 114551 pCsr->pNextId = pCsr->aDoclist; 113865 114552 pCsr->iPrevId = 0; 113866 114553 } 113867 114554 113868 114555 /* Compile a SELECT statement for this cursor. For a full-table-scan, the 113869 114556 ** statement loops through all rows of the %_content table. For a 113870 114557 ** full-text query or docid lookup, the statement retrieves a single 113871 114558 ** row by docid. 113872 114559 */ 113873 - zSql = (char *)azSql[idxNum==FTS3_FULLSCAN_SEARCH]; 113874 - zSql = sqlite3_mprintf(zSql, p->zReadExprlist, p->zDb, p->zName); 113875 - if( !zSql ){ 113876 - rc = SQLITE_NOMEM; 114560 + if( idxNum==FTS3_FULLSCAN_SEARCH ){ 114561 + const char *zSort = (pCsr->bDesc ? "DESC" : "ASC"); 114562 + const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x ORDER BY docid %s"; 114563 + zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName, zSort); 113877 114564 }else{ 113878 - rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0); 113879 - sqlite3_free(zSql); 114565 + const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?"; 114566 + zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName); 113880 114567 } 113881 - if( rc==SQLITE_OK && idxNum==FTS3_DOCID_SEARCH ){ 114568 + if( !zSql ) return SQLITE_NOMEM; 114569 + rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0); 114570 + sqlite3_free(zSql); 114571 + if( rc!=SQLITE_OK ) return rc; 114572 + 114573 + if( idxNum==FTS3_DOCID_SEARCH ){ 113882 114574 rc = sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]); 114575 + if( rc!=SQLITE_OK ) return rc; 113883 114576 } 113884 - pCsr->eSearch = (i16)idxNum; 113885 114577 113886 - if( rc!=SQLITE_OK ) return rc; 113887 114578 return fts3NextMethod(pCursor); 113888 114579 } 113889 114580 113890 114581 /* 113891 114582 ** This is the xEof method of the virtual table. SQLite calls this 113892 114583 ** routine to find out if it has reached the end of a result set. 113893 114584 */ ................................................................................ 113899 114590 ** This is the xRowid method. The SQLite core calls this routine to 113900 114591 ** retrieve the rowid for the current row of the result set. fts3 113901 114592 ** exposes %_content.docid as the rowid for the virtual table. The 113902 114593 ** rowid should be written to *pRowid. 113903 114594 */ 113904 114595 static int fts3RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ 113905 114596 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor; 113906 - if( pCsr->aDoclist ){ 113907 - *pRowid = pCsr->iPrevId; 113908 - }else{ 113909 - /* This branch runs if the query is implemented using a full-table scan 113910 - ** (not using the full-text index). In this case grab the rowid from the 113911 - ** SELECT statement. 113912 - */ 113913 - assert( pCsr->isRequireSeek==0 ); 113914 - *pRowid = sqlite3_column_int64(pCsr->pStmt, 0); 113915 - } 114597 + *pRowid = pCsr->iPrevId; 113916 114598 return SQLITE_OK; 113917 114599 } 113918 114600 113919 114601 /* 113920 114602 ** This is the xColumn method, called by SQLite to request a value from 113921 114603 ** the row that the supplied cursor currently points to. 113922 114604 */ 113923 114605 static int fts3ColumnMethod( 113924 114606 sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ 113925 114607 sqlite3_context *pContext, /* Context for sqlite3_result_xxx() calls */ 113926 114608 int iCol /* Index of column to read value from */ 113927 114609 ){ 113928 - int rc; /* Return Code */ 114610 + int rc = SQLITE_OK; /* Return Code */ 113929 114611 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor; 113930 114612 Fts3Table *p = (Fts3Table *)pCursor->pVtab; 113931 114613 113932 114614 /* The column value supplied by SQLite must be in range. */ 113933 114615 assert( iCol>=0 && iCol<=p->nColumn+1 ); 113934 114616 113935 114617 if( iCol==p->nColumn+1 ){ 113936 114618 /* This call is a request for the "docid" column. Since "docid" is an 113937 114619 ** alias for "rowid", use the xRowid() method to obtain the value. 113938 114620 */ 113939 - sqlite3_int64 iRowid; 113940 - rc = fts3RowidMethod(pCursor, &iRowid); 113941 - sqlite3_result_int64(pContext, iRowid); 114621 + sqlite3_result_int64(pContext, pCsr->iPrevId); 113942 114622 }else if( iCol==p->nColumn ){ 113943 114623 /* The extra column whose name is the same as the table. 113944 114624 ** Return a blob which is a pointer to the cursor. 113945 114625 */ 113946 114626 sqlite3_result_blob(pContext, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT); 113947 - rc = SQLITE_OK; 113948 114627 }else{ 113949 114628 rc = fts3CursorSeek(0, pCsr); 113950 114629 if( rc==SQLITE_OK ){ 113951 114630 sqlite3_result_value(pContext, sqlite3_column_value(pCsr->pStmt, iCol+1)); 113952 114631 } 113953 114632 } 114633 + 114634 + assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); 113954 114635 return rc; 113955 114636 } 113956 114637 113957 114638 /* 113958 114639 ** This function is the implementation of the xUpdate callback used by 113959 114640 ** FTS3 virtual tables. It is invoked by SQLite each time a row is to be 113960 114641 ** inserted, updated or deleted. ................................................................................ 113978 114659 return rc; 113979 114660 } 113980 114661 113981 114662 /* 113982 114663 ** Implementation of xBegin() method. This is a no-op. 113983 114664 */ 113984 114665 static int fts3BeginMethod(sqlite3_vtab *pVtab){ 114666 + TESTONLY( Fts3Table *p = (Fts3Table*)pVtab ); 113985 114667 UNUSED_PARAMETER(pVtab); 113986 - assert( ((Fts3Table *)pVtab)->nPendingData==0 ); 114668 + assert( p->pSegments==0 ); 114669 + assert( p->nPendingData==0 ); 114670 + assert( p->inTransaction!=1 ); 114671 + TESTONLY( p->inTransaction = 1 ); 114672 + TESTONLY( p->mxSavepoint = -1; ); 113987 114673 return SQLITE_OK; 113988 114674 } 113989 114675 113990 114676 /* 113991 114677 ** Implementation of xCommit() method. This is a no-op. The contents of 113992 114678 ** the pending-terms hash-table have already been flushed into the database 113993 114679 ** by fts3SyncMethod(). 113994 114680 */ 113995 114681 static int fts3CommitMethod(sqlite3_vtab *pVtab){ 114682 + TESTONLY( Fts3Table *p = (Fts3Table*)pVtab ); 113996 114683 UNUSED_PARAMETER(pVtab); 113997 - assert( ((Fts3Table *)pVtab)->nPendingData==0 ); 114684 + assert( p->nPendingData==0 ); 114685 + assert( p->inTransaction!=0 ); 114686 + assert( p->pSegments==0 ); 114687 + TESTONLY( p->inTransaction = 0 ); 114688 + TESTONLY( p->mxSavepoint = -1; ); 113998 114689 return SQLITE_OK; 113999 114690 } 114000 114691 114001 114692 /* 114002 114693 ** Implementation of xRollback(). Discard the contents of the pending-terms 114003 114694 ** hash-table. Any changes made to the database are reverted by SQLite. 114004 114695 */ 114005 114696 static int fts3RollbackMethod(sqlite3_vtab *pVtab){ 114006 - sqlite3Fts3PendingTermsClear((Fts3Table *)pVtab); 114007 - return SQLITE_OK; 114008 -} 114009 - 114010 -/* 114011 -** Load the doclist associated with expression pExpr to pExpr->aDoclist. 114012 -** The loaded doclist contains positions as well as the document ids. 114013 -** This is used by the matchinfo(), snippet() and offsets() auxillary 114014 -** functions. 114015 -*/ 114016 -SQLITE_PRIVATE int sqlite3Fts3ExprLoadDoclist(Fts3Cursor *pCsr, Fts3Expr *pExpr){ 114017 - int rc; 114018 - assert( pExpr->eType==FTSQUERY_PHRASE && pExpr->pPhrase ); 114019 - assert( pCsr->eEvalmode==FTS3_EVAL_NEXT ); 114020 - rc = fts3EvalExpr(pCsr, pExpr, &pExpr->aDoclist, &pExpr->nDoclist, 1); 114021 - return rc; 114022 -} 114023 - 114024 -SQLITE_PRIVATE int sqlite3Fts3ExprLoadFtDoclist( 114025 - Fts3Cursor *pCsr, 114026 - Fts3Expr *pExpr, 114027 - char **paDoclist, 114028 - int *pnDoclist 114029 -){ 114030 - int rc; 114031 - assert( pCsr->eEvalmode==FTS3_EVAL_NEXT ); 114032 - assert( pExpr->eType==FTSQUERY_PHRASE && pExpr->pPhrase ); 114033 - pCsr->eEvalmode = FTS3_EVAL_MATCHINFO; 114034 - rc = fts3EvalExpr(pCsr, pExpr, paDoclist, pnDoclist, 1); 114035 - pCsr->eEvalmode = FTS3_EVAL_NEXT; 114036 - return rc; 114037 -} 114038 - 114039 -/* 114040 -** After ExprLoadDoclist() (see above) has been called, this function is 114041 -** used to iterate/search through the position lists that make up the doclist 114042 -** stored in pExpr->aDoclist. 114043 -*/ 114044 -SQLITE_PRIVATE char *sqlite3Fts3FindPositions( 114045 - Fts3Expr *pExpr, /* Access this expressions doclist */ 114046 - sqlite3_int64 iDocid, /* Docid associated with requested pos-list */ 114047 - int iCol /* Column of requested pos-list */ 114048 -){ 114049 - assert( pExpr->isLoaded ); 114050 - if( pExpr->aDoclist ){ 114051 - char *pEnd = &pExpr->aDoclist[pExpr->nDoclist]; 114052 - char *pCsr; 114053 - 114054 - if( pExpr->pCurrent==0 ){ 114055 - pExpr->pCurrent = pExpr->aDoclist; 114056 - pExpr->iCurrent = 0; 114057 - pExpr->pCurrent += sqlite3Fts3GetVarint(pExpr->pCurrent,&pExpr->iCurrent); 114058 - } 114059 - pCsr = pExpr->pCurrent; 114060 - assert( pCsr ); 114061 - 114062 - while( pCsr<pEnd ){ 114063 - if( pExpr->iCurrent<iDocid ){ 114064 - fts3PoslistCopy(0, &pCsr); 114065 - if( pCsr<pEnd ){ 114066 - fts3GetDeltaVarint(&pCsr, &pExpr->iCurrent); 114067 - } 114068 - pExpr->pCurrent = pCsr; 114069 - }else{ 114070 - if( pExpr->iCurrent==iDocid ){ 114071 - int iThis = 0; 114072 - if( iCol<0 ){ 114073 - /* If iCol is negative, return a pointer to the start of the 114074 - ** position-list (instead of a pointer to the start of a list 114075 - ** of offsets associated with a specific column). 114076 - */ 114077 - return pCsr; 114078 - } 114079 - while( iThis<iCol ){ 114080 - fts3ColumnlistCopy(0, &pCsr); 114081 - if( *pCsr==0x00 ) return 0; 114082 - pCsr++; 114083 - pCsr += sqlite3Fts3GetVarint32(pCsr, &iThis); 114084 - } 114085 - if( iCol==iThis && (*pCsr&0xFE) ) return pCsr; 114086 - } 114087 - return 0; 114088 - } 114089 - } 114090 - } 114091 - 114092 - return 0; 114697 + Fts3Table *p = (Fts3Table*)pVtab; 114698 + sqlite3Fts3PendingTermsClear(p); 114699 + assert( p->inTransaction!=0 ); 114700 + TESTONLY( p->inTransaction = 0 ); 114701 + TESTONLY( p->mxSavepoint = -1; ); 114702 + return SQLITE_OK; 114703 +} 114704 + 114705 +/* 114706 +** When called, *ppPoslist must point to the byte immediately following the 114707 +** end of a position-list. i.e. ( (*ppPoslist)[-1]==POS_END ). This function 114708 +** moves *ppPoslist so that it instead points to the first byte of the 114709 +** same position list. 114710 +*/ 114711 +static void fts3ReversePoslist(char *pStart, char **ppPoslist){ 114712 + char *p = &(*ppPoslist)[-2]; 114713 + char c; 114714 + 114715 + while( p>pStart && (c=*p--)==0 ); 114716 + while( p>pStart && (*p & 0x80) | c ){ 114717 + c = *p--; 114718 + } 114719 + if( p>pStart ){ p = &p[2]; } 114720 + while( *p++&0x80 ); 114721 + *ppPoslist = p; 114093 114722 } 114094 114723 114095 114724 /* 114096 114725 ** Helper function used by the implementation of the overloaded snippet(), 114097 114726 ** offsets() and optimize() SQL functions. 114098 114727 ** 114099 114728 ** If the value passed as the third argument is a blob of size ................................................................................ 114317 114946 ); 114318 114947 fts3DbExec(&rc, db, 114319 114948 "ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';", 114320 114949 p->zDb, p->zName, zName 114321 114950 ); 114322 114951 return rc; 114323 114952 } 114953 + 114954 +static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ 114955 + UNUSED_PARAMETER(iSavepoint); 114956 + assert( ((Fts3Table *)pVtab)->inTransaction ); 114957 + assert( ((Fts3Table *)pVtab)->mxSavepoint < iSavepoint ); 114958 + TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint ); 114959 + return fts3SyncMethod(pVtab); 114960 +} 114961 +static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){ 114962 + TESTONLY( Fts3Table *p = (Fts3Table*)pVtab ); 114963 + UNUSED_PARAMETER(iSavepoint); 114964 + UNUSED_PARAMETER(pVtab); 114965 + assert( p->inTransaction ); 114966 + assert( p->mxSavepoint >= iSavepoint ); 114967 + TESTONLY( p->mxSavepoint = iSavepoint-1 ); 114968 + return SQLITE_OK; 114969 +} 114970 +static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ 114971 + Fts3Table *p = (Fts3Table*)pVtab; 114972 + UNUSED_PARAMETER(iSavepoint); 114973 + assert( p->inTransaction ); 114974 + assert( p->mxSavepoint >= iSavepoint ); 114975 + TESTONLY( p->mxSavepoint = iSavepoint ); 114976 + sqlite3Fts3PendingTermsClear(p); 114977 + return SQLITE_OK; 114978 +} 114324 114979 114325 114980 static const sqlite3_module fts3Module = { 114326 - /* iVersion */ 0, 114981 + /* iVersion */ 2, 114327 114982 /* xCreate */ fts3CreateMethod, 114328 114983 /* xConnect */ fts3ConnectMethod, 114329 114984 /* xBestIndex */ fts3BestIndexMethod, 114330 114985 /* xDisconnect */ fts3DisconnectMethod, 114331 114986 /* xDestroy */ fts3DestroyMethod, 114332 114987 /* xOpen */ fts3OpenMethod, 114333 114988 /* xClose */ fts3CloseMethod, ................................................................................ 114339 114994 /* xUpdate */ fts3UpdateMethod, 114340 114995 /* xBegin */ fts3BeginMethod, 114341 114996 /* xSync */ fts3SyncMethod, 114342 114997 /* xCommit */ fts3CommitMethod, 114343 114998 /* xRollback */ fts3RollbackMethod, 114344 114999 /* xFindFunction */ fts3FindFunctionMethod, 114345 115000 /* xRename */ fts3RenameMethod, 115001 + /* xSavepoint */ fts3SavepointMethod, 115002 + /* xRelease */ fts3ReleaseMethod, 115003 + /* xRollbackTo */ fts3RollbackToMethod, 114346 115004 }; 114347 115005 114348 115006 /* 114349 115007 ** This function is registered as the module destructor (called when an 114350 115008 ** FTS3 enabled database connection is closed). It frees the memory 114351 115009 ** allocated for the tokenizer hash table. 114352 115010 */ ................................................................................ 114384 115042 const sqlite3_tokenizer_module *pSimple = 0; 114385 115043 const sqlite3_tokenizer_module *pPorter = 0; 114386 115044 114387 115045 #ifdef SQLITE_ENABLE_ICU 114388 115046 const sqlite3_tokenizer_module *pIcu = 0; 114389 115047 sqlite3Fts3IcuTokenizerModule(&pIcu); 114390 115048 #endif 115049 + 115050 +#ifdef SQLITE_TEST 115051 + rc = sqlite3Fts3InitTerm(db); 115052 + if( rc!=SQLITE_OK ) return rc; 115053 +#endif 114391 115054 114392 115055 rc = sqlite3Fts3InitAux(db); 114393 115056 if( rc!=SQLITE_OK ) return rc; 114394 115057 114395 115058 sqlite3Fts3SimpleTokenizerModule(&pSimple); 114396 115059 sqlite3Fts3PorterTokenizerModule(&pPorter); 114397 115060 ................................................................................ 114460 115123 const sqlite3_api_routines *pApi 114461 115124 ){ 114462 115125 SQLITE_EXTENSION_INIT2(pApi) 114463 115126 return sqlite3Fts3Init(db); 114464 115127 } 114465 115128 #endif 114466 115129 115130 + 115131 +/* 115132 +** Allocate an Fts3MultiSegReader for each token in the expression headed 115133 +** by pExpr. 115134 +** 115135 +** An Fts3SegReader object is a cursor that can seek or scan a range of 115136 +** entries within a single segment b-tree. An Fts3MultiSegReader uses multiple 115137 +** Fts3SegReader objects internally to provide an interface to seek or scan 115138 +** within the union of all segments of a b-tree. Hence the name. 115139 +** 115140 +** If the allocated Fts3MultiSegReader just seeks to a single entry in a 115141 +** segment b-tree (if the term is not a prefix or it is a prefix for which 115142 +** there exists prefix b-tree of the right length) then it may be traversed 115143 +** and merged incrementally. Otherwise, it has to be merged into an in-memory 115144 +** doclist and then traversed. 115145 +*/ 115146 +static void fts3EvalAllocateReaders( 115147 + Fts3Cursor *pCsr, 115148 + Fts3Expr *pExpr, 115149 + int *pnToken, /* OUT: Total number of tokens in phrase. */ 115150 + int *pnOr, /* OUT: Total number of OR nodes in expr. */ 115151 + int *pRc 115152 +){ 115153 + if( pExpr && SQLITE_OK==*pRc ){ 115154 + if( pExpr->eType==FTSQUERY_PHRASE ){ 115155 + int i; 115156 + int nToken = pExpr->pPhrase->nToken; 115157 + *pnToken += nToken; 115158 + for(i=0; i<nToken; i++){ 115159 + Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i]; 115160 + int rc = sqlite3Fts3TermSegReaderCursor(pCsr, 115161 + pToken->z, pToken->n, pToken->isPrefix, &pToken->pSegcsr 115162 + ); 115163 + if( rc!=SQLITE_OK ){ 115164 + *pRc = rc; 115165 + return; 115166 + } 115167 + } 115168 + assert( pExpr->pPhrase->iDoclistToken==0 ); 115169 + pExpr->pPhrase->iDoclistToken = -1; 115170 + }else{ 115171 + *pnOr += (pExpr->eType==FTSQUERY_OR); 115172 + fts3EvalAllocateReaders(pCsr, pExpr->pLeft, pnToken, pnOr, pRc); 115173 + fts3EvalAllocateReaders(pCsr, pExpr->pRight, pnToken, pnOr, pRc); 115174 + } 115175 + } 115176 +} 115177 + 115178 +static void fts3EvalPhraseMergeToken( 115179 + Fts3Table *pTab, 115180 + Fts3Phrase *p, 115181 + int iToken, 115182 + char *pList, 115183 + int nList 115184 +){ 115185 + assert( iToken!=p->iDoclistToken ); 115186 + 115187 + if( pList==0 ){ 115188 + sqlite3_free(p->doclist.aAll); 115189 + p->doclist.aAll = 0; 115190 + p->doclist.nAll = 0; 115191 + } 115192 + 115193 + else if( p->iDoclistToken<0 ){ 115194 + p->doclist.aAll = pList; 115195 + p->doclist.nAll = nList; 115196 + } 115197 + 115198 + else if( p->doclist.aAll==0 ){ 115199 + sqlite3_free(pList); 115200 + } 115201 + 115202 + else { 115203 + char *pLeft; 115204 + char *pRight; 115205 + int nLeft; 115206 + int nRight; 115207 + int nDiff; 115208 + 115209 + if( p->iDoclistToken<iToken ){ 115210 + pLeft = p->doclist.aAll; 115211 + nLeft = p->doclist.nAll; 115212 + pRight = pList; 115213 + nRight = nList; 115214 + nDiff = iToken - p->iDoclistToken; 115215 + }else{ 115216 + pRight = p->doclist.aAll; 115217 + nRight = p->doclist.nAll; 115218 + pLeft = pList; 115219 + nLeft = nList; 115220 + nDiff = p->iDoclistToken - iToken; 115221 + } 115222 + 115223 + fts3DoclistPhraseMerge(pTab->bDescIdx, nDiff, pLeft, nLeft, pRight,&nRight); 115224 + sqlite3_free(pLeft); 115225 + p->doclist.aAll = pRight; 115226 + p->doclist.nAll = nRight; 115227 + } 115228 + 115229 + if( iToken>p->iDoclistToken ) p->iDoclistToken = iToken; 115230 +} 115231 + 115232 +static int fts3EvalPhraseLoad( 115233 + Fts3Cursor *pCsr, 115234 + Fts3Phrase *p 115235 +){ 115236 + Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; 115237 + int iToken; 115238 + int rc = SQLITE_OK; 115239 + 115240 + for(iToken=0; rc==SQLITE_OK && iToken<p->nToken; iToken++){ 115241 + Fts3PhraseToken *pToken = &p->aToken[iToken]; 115242 + assert( pToken->pDeferred==0 || pToken->pSegcsr==0 ); 115243 + 115244 + if( pToken->pSegcsr ){ 115245 + int nThis = 0; 115246 + char *pThis = 0; 115247 + rc = fts3TermSelect(pTab, pToken, p->iColumn, 1, &nThis, &pThis); 115248 + if( rc==SQLITE_OK ){ 115249 + fts3EvalPhraseMergeToken(pTab, p, iToken, pThis, nThis); 115250 + } 115251 + } 115252 + assert( pToken->pSegcsr==0 ); 115253 + } 115254 + 115255 + return rc; 115256 +} 115257 + 115258 +static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){ 115259 + int iToken; 115260 + int rc = SQLITE_OK; 115261 + 115262 + int nMaxUndeferred = pPhrase->iDoclistToken; 115263 + char *aPoslist = 0; 115264 + int nPoslist = 0; 115265 + int iPrev = -1; 115266 + 115267 + assert( pPhrase->doclist.bFreeList==0 ); 115268 + 115269 + for(iToken=0; rc==SQLITE_OK && iToken<pPhrase->nToken; iToken++){ 115270 + Fts3PhraseToken *pToken = &pPhrase->aToken[iToken]; 115271 + Fts3DeferredToken *pDeferred = pToken->pDeferred; 115272 + 115273 + if( pDeferred ){ 115274 + char *pList; 115275 + int nList; 115276 + rc = sqlite3Fts3DeferredTokenList(pDeferred, &pList, &nList); 115277 + if( rc!=SQLITE_OK ) return rc; 115278 + 115279 + if( pList==0 ){ 115280 + sqlite3_free(aPoslist); 115281 + pPhrase->doclist.pList = 0; 115282 + pPhrase->doclist.nList = 0; 115283 + return SQLITE_OK; 115284 + 115285 + }else if( aPoslist==0 ){ 115286 + aPoslist = pList; 115287 + nPoslist = nList; 115288 + 115289 + }else{ 115290 + char *aOut = pList; 115291 + char *p1 = aPoslist; 115292 + char *p2 = aOut; 115293 + 115294 + assert( iPrev>=0 ); 115295 + fts3PoslistPhraseMerge(&aOut, iToken-iPrev, 0, 1, &p1, &p2); 115296 + sqlite3_free(aPoslist); 115297 + aPoslist = pList; 115298 + nPoslist = aOut - aPoslist; 115299 + if( nPoslist==0 ){ 115300 + sqlite3_free(aPoslist); 115301 + pPhrase->doclist.pList = 0; 115302 + pPhrase->doclist.nList = 0; 115303 + return SQLITE_OK; 115304 + } 115305 + } 115306 + iPrev = iToken; 115307 + } 115308 + } 115309 + 115310 + if( iPrev>=0 ){ 115311 + if( nMaxUndeferred<0 ){ 115312 + pPhrase->doclist.pList = aPoslist; 115313 + pPhrase->doclist.nList = nPoslist; 115314 + pPhrase->doclist.iDocid = pCsr->iPrevId; 115315 + pPhrase->doclist.bFreeList = 1; 115316 + }else{ 115317 + int nDistance; 115318 + char *p1; 115319 + char *p2; 115320 + char *aOut; 115321 + 115322 + if( nMaxUndeferred>iPrev ){ 115323 + p1 = aPoslist; 115324 + p2 = pPhrase->doclist.pList; 115325 + nDistance = nMaxUndeferred - iPrev; 115326 + }else{ 115327 + p1 = pPhrase->doclist.pList; 115328 + p2 = aPoslist; 115329 + nDistance = iPrev - nMaxUndeferred; 115330 + } 115331 + 115332 + aOut = (char *)sqlite3_malloc(nPoslist+8); 115333 + if( !aOut ){ 115334 + sqlite3_free(aPoslist); 115335 + return SQLITE_NOMEM; 115336 + } 115337 + 115338 + pPhrase->doclist.pList = aOut; 115339 + if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){ 115340 + pPhrase->doclist.bFreeList = 1; 115341 + pPhrase->doclist.nList = (aOut - pPhrase->doclist.pList); 115342 + }else{ 115343 + sqlite3_free(aOut); 115344 + pPhrase->doclist.pList = 0; 115345 + pPhrase->doclist.nList = 0; 115346 + } 115347 + sqlite3_free(aPoslist); 115348 + } 115349 + } 115350 + 115351 + return SQLITE_OK; 115352 +} 115353 + 115354 +/* 115355 +** This function is called for each Fts3Phrase in a full-text query 115356 +** expression to initialize the mechanism for returning rows. Once this 115357 +** function has been called successfully on an Fts3Phrase, it may be 115358 +** used with fts3EvalPhraseNext() to iterate through the matching docids. 115359 +*/ 115360 +static int fts3EvalPhraseStart(Fts3Cursor *pCsr, int bOptOk, Fts3Phrase *p){ 115361 + int rc; 115362 + Fts3PhraseToken *pFirst = &p->aToken[0]; 115363 + Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; 115364 + 115365 + if( pCsr->bDesc==pTab->bDescIdx 115366 + && bOptOk==1 115367 + && p->nToken==1 115368 + && pFirst->pSegcsr 115369 + && pFirst->pSegcsr->bLookup 115370 + ){ 115371 + /* Use the incremental approach. */ 115372 + int iCol = (p->iColumn >= pTab->nColumn ? -1 : p->iColumn); 115373 + rc = sqlite3Fts3MsrIncrStart( 115374 + pTab, pFirst->pSegcsr, iCol, pFirst->z, pFirst->n); 115375 + p->bIncr = 1; 115376 + 115377 + }else{ 115378 + /* Load the full doclist for the phrase into memory. */ 115379 + rc = fts3EvalPhraseLoad(pCsr, p); 115380 + p->bIncr = 0; 115381 + } 115382 + 115383 + assert( rc!=SQLITE_OK || p->nToken<1 || p->aToken[0].pSegcsr==0 || p->bIncr ); 115384 + return rc; 115385 +} 115386 + 115387 +/* 115388 +** This function is used to iterate backwards (from the end to start) 115389 +** through doclists. 115390 +*/ 115391 +SQLITE_PRIVATE void sqlite3Fts3DoclistPrev( 115392 + int bDescIdx, /* True if the doclist is desc */ 115393 + char *aDoclist, /* Pointer to entire doclist */ 115394 + int nDoclist, /* Length of aDoclist in bytes */ 115395 + char **ppIter, /* IN/OUT: Iterator pointer */ 115396 + sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */ 115397 + int *pnList, /* IN/OUT: List length pointer */ 115398 + u8 *pbEof /* OUT: End-of-file flag */ 115399 +){ 115400 + char *p = *ppIter; 115401 + 115402 + assert( nDoclist>0 ); 115403 + assert( *pbEof==0 ); 115404 + assert( p || *piDocid==0 ); 115405 + assert( !p || (p>aDoclist && p<&aDoclist[nDoclist]) ); 115406 + 115407 + if( p==0 ){ 115408 + sqlite3_int64 iDocid = 0; 115409 + char *pNext = 0; 115410 + char *pDocid = aDoclist; 115411 + char *pEnd = &aDoclist[nDoclist]; 115412 + int iMul = 1; 115413 + 115414 + while( pDocid<pEnd ){ 115415 + sqlite3_int64 iDelta; 115416 + pDocid += sqlite3Fts3GetVarint(pDocid, &iDelta); 115417 + iDocid += (iMul * iDelta); 115418 + pNext = pDocid; 115419 + fts3PoslistCopy(0, &pDocid); 115420 + while( pDocid<pEnd && *pDocid==0 ) pDocid++; 115421 + iMul = (bDescIdx ? -1 : 1); 115422 + } 115423 + 115424 + *pnList = pEnd - pNext; 115425 + *ppIter = pNext; 115426 + *piDocid = iDocid; 115427 + }else{ 115428 + int iMul = (bDescIdx ? -1 : 1); 115429 + sqlite3_int64 iDelta; 115430 + fts3GetReverseVarint(&p, aDoclist, &iDelta); 115431 + *piDocid -= (iMul * iDelta); 115432 + 115433 + if( p==aDoclist ){ 115434 + *pbEof = 1; 115435 + }else{ 115436 + char *pSave = p; 115437 + fts3ReversePoslist(aDoclist, &p); 115438 + *pnList = (pSave - p); 115439 + } 115440 + *ppIter = p; 115441 + } 115442 +} 115443 + 115444 +/* 115445 +** Attempt to move the phrase iterator to point to the next matching docid. 115446 +** If an error occurs, return an SQLite error code. Otherwise, return 115447 +** SQLITE_OK. 115448 +** 115449 +** If there is no "next" entry and no error occurs, then *pbEof is set to 115450 +** 1 before returning. Otherwise, if no error occurs and the iterator is 115451 +** successfully advanced, *pbEof is set to 0. 115452 +*/ 115453 +static int fts3EvalPhraseNext( 115454 + Fts3Cursor *pCsr, 115455 + Fts3Phrase *p, 115456 + u8 *pbEof 115457 +){ 115458 + int rc = SQLITE_OK; 115459 + Fts3Doclist *pDL = &p->doclist; 115460 + Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; 115461 + 115462 + if( p->bIncr ){ 115463 + assert( p->nToken==1 ); 115464 + assert( pDL->pNextDocid==0 ); 115465 + rc = sqlite3Fts3MsrIncrNext(pTab, p->aToken[0].pSegcsr, 115466 + &pDL->iDocid, &pDL->pList, &pDL->nList 115467 + ); 115468 + if( rc==SQLITE_OK && !pDL->pList ){ 115469 + *pbEof = 1; 115470 + } 115471 + }else if( pCsr->bDesc!=pTab->bDescIdx && pDL->nAll ){ 115472 + sqlite3Fts3DoclistPrev(pTab->bDescIdx, pDL->aAll, pDL->nAll, 115473 + &pDL->pNextDocid, &pDL->iDocid, &pDL->nList, pbEof 115474 + ); 115475 + pDL->pList = pDL->pNextDocid; 115476 + }else{ 115477 + char *pIter; /* Used to iterate through aAll */ 115478 + char *pEnd = &pDL->aAll[pDL->nAll]; /* 1 byte past end of aAll */ 115479 + if( pDL->pNextDocid ){ 115480 + pIter = pDL->pNextDocid; 115481 + }else{ 115482 + pIter = pDL->aAll; 115483 + } 115484 + 115485 + if( pIter>=pEnd ){ 115486 + /* We have already reached the end of this doclist. EOF. */ 115487 + *pbEof = 1; 115488 + }else{ 115489 + sqlite3_int64 iDelta; 115490 + pIter += sqlite3Fts3GetVarint(pIter, &iDelta); 115491 + if( pTab->bDescIdx==0 || pDL->pNextDocid==0 ){ 115492 + pDL->iDocid += iDelta; 115493 + }else{ 115494 + pDL->iDocid -= iDelta; 115495 + } 115496 + pDL->pList = pIter; 115497 + fts3PoslistCopy(0, &pIter); 115498 + pDL->nList = (pIter - pDL->pList); 115499 + 115500 + /* pIter now points just past the 0x00 that terminates the position- 115501 + ** list for document pDL->iDocid. However, if this position-list was 115502 + ** edited in place by fts3EvalNearTrim2(), then pIter may not actually 115503 + ** point to the start of the next docid value. The following line deals 115504 + ** with this case by advancing pIter past the zero-padding added by 115505 + ** fts3EvalNearTrim2(). */ 115506 + while( pIter<pEnd && *pIter==0 ) pIter++; 115507 + 115508 + pDL->pNextDocid = pIter; 115509 + assert( pIter>=&pDL->aAll[pDL->nAll] || *pIter ); 115510 + *pbEof = 0; 115511 + } 115512 + } 115513 + 115514 + return rc; 115515 +} 115516 + 115517 +static void fts3EvalStartReaders( 115518 + Fts3Cursor *pCsr, 115519 + Fts3Expr *pExpr, 115520 + int bOptOk, 115521 + int *pRc 115522 +){ 115523 + if( pExpr && SQLITE_OK==*pRc ){ 115524 + if( pExpr->eType==FTSQUERY_PHRASE ){ 115525 + int i; 115526 + int nToken = pExpr->pPhrase->nToken; 115527 + for(i=0; i<nToken; i++){ 115528 + if( pExpr->pPhrase->aToken[i].pDeferred==0 ) break; 115529 + } 115530 + pExpr->bDeferred = (i==nToken); 115531 + *pRc = fts3EvalPhraseStart(pCsr, bOptOk, pExpr->pPhrase); 115532 + }else{ 115533 + fts3EvalStartReaders(pCsr, pExpr->pLeft, bOptOk, pRc); 115534 + fts3EvalStartReaders(pCsr, pExpr->pRight, bOptOk, pRc); 115535 + pExpr->bDeferred = (pExpr->pLeft->bDeferred && pExpr->pRight->bDeferred); 115536 + } 115537 + } 115538 +} 115539 + 115540 +typedef struct Fts3TokenAndCost Fts3TokenAndCost; 115541 +struct Fts3TokenAndCost { 115542 + Fts3Phrase *pPhrase; /* The phrase the token belongs to */ 115543 + int iToken; /* Position of token in phrase */ 115544 + Fts3PhraseToken *pToken; /* The token itself */ 115545 + Fts3Expr *pRoot; 115546 + int nOvfl; 115547 + int iCol; /* The column the token must match */ 115548 +}; 115549 + 115550 +static void fts3EvalTokenCosts( 115551 + Fts3Cursor *pCsr, 115552 + Fts3Expr *pRoot, 115553 + Fts3Expr *pExpr, 115554 + Fts3TokenAndCost **ppTC, 115555 + Fts3Expr ***ppOr, 115556 + int *pRc 115557 +){ 115558 + if( *pRc==SQLITE_OK && pExpr ){ 115559 + if( pExpr->eType==FTSQUERY_PHRASE ){ 115560 + Fts3Phrase *pPhrase = pExpr->pPhrase; 115561 + int i; 115562 + for(i=0; *pRc==SQLITE_OK && i<pPhrase->nToken; i++){ 115563 + Fts3TokenAndCost *pTC = (*ppTC)++; 115564 + pTC->pPhrase = pPhrase; 115565 + pTC->iToken = i; 115566 + pTC->pRoot = pRoot; 115567 + pTC->pToken = &pPhrase->aToken[i]; 115568 + pTC->iCol = pPhrase->iColumn; 115569 + *pRc = sqlite3Fts3MsrOvfl(pCsr, pTC->pToken->pSegcsr, &pTC->nOvfl); 115570 + } 115571 + }else if( pExpr->eType!=FTSQUERY_NOT ){ 115572 + if( pExpr->eType==FTSQUERY_OR ){ 115573 + pRoot = pExpr->pLeft; 115574 + **ppOr = pRoot; 115575 + (*ppOr)++; 115576 + } 115577 + fts3EvalTokenCosts(pCsr, pRoot, pExpr->pLeft, ppTC, ppOr, pRc); 115578 + if( pExpr->eType==FTSQUERY_OR ){ 115579 + pRoot = pExpr->pRight; 115580 + **ppOr = pRoot; 115581 + (*ppOr)++; 115582 + } 115583 + fts3EvalTokenCosts(pCsr, pRoot, pExpr->pRight, ppTC, ppOr, pRc); 115584 + } 115585 + } 115586 +} 115587 + 115588 +static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){ 115589 + if( pCsr->nRowAvg==0 ){ 115590 + /* The average document size, which is required to calculate the cost 115591 + ** of each doclist, has not yet been determined. Read the required 115592 + ** data from the %_stat table to calculate it. 115593 + ** 115594 + ** Entry 0 of the %_stat table is a blob containing (nCol+1) FTS3 115595 + ** varints, where nCol is the number of columns in the FTS3 table. 115596 + ** The first varint is the number of documents currently stored in 115597 + ** the table. The following nCol varints contain the total amount of 115598 + ** data stored in all rows of each column of the table, from left 115599 + ** to right. 115600 + */ 115601 + int rc; 115602 + Fts3Table *p = (Fts3Table*)pCsr->base.pVtab; 115603 + sqlite3_stmt *pStmt; 115604 + sqlite3_int64 nDoc = 0; 115605 + sqlite3_int64 nByte = 0; 115606 + const char *pEnd; 115607 + const char *a; 115608 + 115609 + rc = sqlite3Fts3SelectDoctotal(p, &pStmt); 115610 + if( rc!=SQLITE_OK ) return rc; 115611 + a = sqlite3_column_blob(pStmt, 0); 115612 + assert( a ); 115613 + 115614 + pEnd = &a[sqlite3_column_bytes(pStmt, 0)]; 115615 + a += sqlite3Fts3GetVarint(a, &nDoc); 115616 + while( a<pEnd ){ 115617 + a += sqlite3Fts3GetVarint(a, &nByte); 115618 + } 115619 + if( nDoc==0 || nByte==0 ){ 115620 + sqlite3_reset(pStmt); 115621 + return SQLITE_CORRUPT_VTAB; 115622 + } 115623 + 115624 + pCsr->nDoc = nDoc; 115625 + pCsr->nRowAvg = (int)(((nByte / nDoc) + p->nPgsz) / p->nPgsz); 115626 + assert( pCsr->nRowAvg>0 ); 115627 + rc = sqlite3_reset(pStmt); 115628 + if( rc!=SQLITE_OK ) return rc; 115629 + } 115630 + 115631 + *pnPage = pCsr->nRowAvg; 115632 + return SQLITE_OK; 115633 +} 115634 + 115635 +static int fts3EvalSelectDeferred( 115636 + Fts3Cursor *pCsr, 115637 + Fts3Expr *pRoot, 115638 + Fts3TokenAndCost *aTC, 115639 + int nTC 115640 +){ 115641 + int nDocSize = 0; 115642 + int nDocEst = 0; 115643 + int rc = SQLITE_OK; 115644 + Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; 115645 + int ii; 115646 + 115647 + int nOvfl = 0; 115648 + int nTerm = 0; 115649 + 115650 + for(ii=0; ii<nTC; ii++){ 115651 + if( aTC[ii].pRoot==pRoot ){ 115652 + nOvfl += aTC[ii].nOvfl; 115653 + nTerm++; 115654 + } 115655 + } 115656 + if( nOvfl==0 || nTerm<2 ) return SQLITE_OK; 115657 + 115658 + rc = fts3EvalAverageDocsize(pCsr, &nDocSize); 115659 + 115660 + for(ii=0; ii<nTerm && rc==SQLITE_OK; ii++){ 115661 + int jj; 115662 + Fts3TokenAndCost *pTC = 0; 115663 + 115664 + for(jj=0; jj<nTC; jj++){ 115665 + if( aTC[jj].pToken && aTC[jj].pRoot==pRoot 115666 + && (!pTC || aTC[jj].nOvfl<pTC->nOvfl) 115667 + ){ 115668 + pTC = &aTC[jj]; 115669 + } 115670 + } 115671 + assert( pTC ); 115672 + 115673 + /* At this point pTC points to the cheapest remaining token. */ 115674 + if( ii==0 ){ 115675 + if( pTC->nOvfl ){ 115676 + nDocEst = (pTC->nOvfl * pTab->nPgsz + pTab->nPgsz) / 10; 115677 + }else{ 115678 + Fts3PhraseToken *pToken = pTC->pToken; 115679 + int nList = 0; 115680 + char *pList = 0; 115681 + rc = fts3TermSelect(pTab, pToken, pTC->iCol, 1, &nList, &pList); 115682 + assert( rc==SQLITE_OK || pList==0 ); 115683 + 115684 + if( rc==SQLITE_OK ){ 115685 + nDocEst = fts3DoclistCountDocids(1, pList, nList); 115686 + fts3EvalPhraseMergeToken(pTab, pTC->pPhrase, pTC->iToken,pList,nList); 115687 + } 115688 + } 115689 + }else{ 115690 + if( pTC->nOvfl>=(nDocEst*nDocSize) ){ 115691 + Fts3PhraseToken *pToken = pTC->pToken; 115692 + rc = sqlite3Fts3DeferToken(pCsr, pToken, pTC->iCol); 115693 + fts3SegReaderCursorFree(pToken->pSegcsr); 115694 + pToken->pSegcsr = 0; 115695 + } 115696 + nDocEst = 1 + (nDocEst/4); 115697 + } 115698 + pTC->pToken = 0; 115699 + } 115700 + 115701 + return rc; 115702 +} 115703 + 115704 +SQLITE_PRIVATE int sqlite3Fts3EvalStart(Fts3Cursor *pCsr, Fts3Expr *pExpr, int bOptOk){ 115705 + Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; 115706 + int rc = SQLITE_OK; 115707 + int nToken = 0; 115708 + int nOr = 0; 115709 + 115710 + /* Allocate a MultiSegReader for each token in the expression. */ 115711 + fts3EvalAllocateReaders(pCsr, pExpr, &nToken, &nOr, &rc); 115712 + 115713 + /* Call fts3EvalPhraseStart() on all phrases in the expression. TODO: 115714 + ** This call will eventually also be responsible for determining which 115715 + ** tokens are 'deferred' until the document text is loaded into memory. 115716 + ** 115717 + ** Each token in each phrase is dealt with using one of the following 115718 + ** three strategies: 115719 + ** 115720 + ** 1. Entire doclist loaded into memory as part of the 115721 + ** fts3EvalStartReaders() call. 115722 + ** 115723 + ** 2. Doclist loaded into memory incrementally, as part of each 115724 + ** sqlite3Fts3EvalNext() call. 115725 + ** 115726 + ** 3. Token doclist is never loaded. Instead, documents are loaded into 115727 + ** memory and scanned for the token as part of the sqlite3Fts3EvalNext() 115728 + ** call. This is known as a "deferred" token. 115729 + */ 115730 + 115731 + /* If bOptOk is true, check if there are any tokens that should be deferred. 115732 + */ 115733 + if( rc==SQLITE_OK && bOptOk && nToken>1 && pTab->bHasStat ){ 115734 + Fts3TokenAndCost *aTC; 115735 + Fts3Expr **apOr; 115736 + aTC = (Fts3TokenAndCost *)sqlite3_malloc( 115737 + sizeof(Fts3TokenAndCost) * nToken 115738 + + sizeof(Fts3Expr *) * nOr * 2 115739 + ); 115740 + apOr = (Fts3Expr **)&aTC[nToken]; 115741 + 115742 + if( !aTC ){ 115743 + rc = SQLITE_NOMEM; 115744 + }else{ 115745 + int ii; 115746 + Fts3TokenAndCost *pTC = aTC; 115747 + Fts3Expr **ppOr = apOr; 115748 + 115749 + fts3EvalTokenCosts(pCsr, 0, pExpr, &pTC, &ppOr, &rc); 115750 + nToken = pTC-aTC; 115751 + nOr = ppOr-apOr; 115752 + 115753 + if( rc==SQLITE_OK ){ 115754 + rc = fts3EvalSelectDeferred(pCsr, 0, aTC, nToken); 115755 + for(ii=0; rc==SQLITE_OK && ii<nOr; ii++){ 115756 + rc = fts3EvalSelectDeferred(pCsr, apOr[ii], aTC, nToken); 115757 + } 115758 + } 115759 + 115760 + sqlite3_free(aTC); 115761 + } 115762 + } 115763 + 115764 + fts3EvalStartReaders(pCsr, pExpr, bOptOk, &rc); 115765 + return rc; 115766 +} 115767 + 115768 +static void fts3EvalZeroPoslist(Fts3Phrase *pPhrase){ 115769 + if( pPhrase->doclist.bFreeList ){ 115770 + sqlite3_free(pPhrase->doclist.pList); 115771 + } 115772 + pPhrase->doclist.pList = 0; 115773 + pPhrase->doclist.nList = 0; 115774 + pPhrase->doclist.bFreeList = 0; 115775 +} 115776 + 115777 +static int fts3EvalNearTrim2( 115778 + int nNear, 115779 + char *aTmp, /* Temporary space to use */ 115780 + char **paPoslist, /* IN/OUT: Position list */ 115781 + int *pnToken, /* IN/OUT: Tokens in phrase of *paPoslist */ 115782 + Fts3Phrase *pPhrase /* The phrase object to trim the doclist of */ 115783 +){ 115784 + int nParam1 = nNear + pPhrase->nToken; 115785 + int nParam2 = nNear + *pnToken; 115786 + int nNew; 115787 + char *p2; 115788 + char *pOut; 115789 + int res; 115790 + 115791 + assert( pPhrase->doclist.pList ); 115792 + 115793 + p2 = pOut = pPhrase->doclist.pList; 115794 + res = fts3PoslistNearMerge( 115795 + &pOut, aTmp, nParam1, nParam2, paPoslist, &p2 115796 + ); 115797 + if( res ){ 115798 + nNew = (pOut - pPhrase->doclist.pList) - 1; 115799 + assert( pPhrase->doclist.pList[nNew]=='\0' ); 115800 + assert( nNew<=pPhrase->doclist.nList && nNew>0 ); 115801 + memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew); 115802 + pPhrase->doclist.nList = nNew; 115803 + *paPoslist = pPhrase->doclist.pList; 115804 + *pnToken = pPhrase->nToken; 115805 + } 115806 + 115807 + return res; 115808 +} 115809 + 115810 +static int fts3EvalNearTest(Fts3Expr *pExpr, int *pRc){ 115811 + int res = 1; 115812 + 115813 + /* The following block runs if pExpr is the root of a NEAR query. 115814 + ** For example, the query: 115815 + ** 115816 + ** "w" NEAR "x" NEAR "y" NEAR "z" 115817 + ** 115818 + ** which is represented in tree form as: 115819 + ** 115820 + ** | 115821 + ** +--NEAR--+ <-- root of NEAR query 115822 + ** | | 115823 + ** +--NEAR--+ "z" 115824 + ** | | 115825 + ** +--NEAR--+ "y" 115826 + ** | | 115827 + ** "w" "x" 115828 + ** 115829 + ** The right-hand child of a NEAR node is always a phrase. The 115830 + ** left-hand child may be either a phrase or a NEAR node. There are 115831 + ** no exceptions to this. 115832 + */ 115833 + if( *pRc==SQLITE_OK 115834 + && pExpr->eType==FTSQUERY_NEAR 115835 + && pExpr->bEof==0 115836 + && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR) 115837 + ){ 115838 + Fts3Expr *p; 115839 + int nTmp = 0; /* Bytes of temp space */ 115840 + char *aTmp; /* Temp space for PoslistNearMerge() */ 115841 + 115842 + /* Allocate temporary working space. */ 115843 + for(p=pExpr; p->pLeft; p=p->pLeft){ 115844 + nTmp += p->pRight->pPhrase->doclist.nList; 115845 + } 115846 + nTmp += p->pPhrase->doclist.nList; 115847 + aTmp = sqlite3_malloc(nTmp*2); 115848 + if( !aTmp ){ 115849 + *pRc = SQLITE_NOMEM; 115850 + res = 0; 115851 + }else{ 115852 + char *aPoslist = p->pPhrase->doclist.pList; 115853 + int nToken = p->pPhrase->nToken; 115854 + 115855 + for(p=p->pParent;res && p && p->eType==FTSQUERY_NEAR; p=p->pParent){ 115856 + Fts3Phrase *pPhrase = p->pRight->pPhrase; 115857 + int nNear = p->nNear; 115858 + res = fts3EvalNearTrim2(nNear, aTmp, &aPoslist, &nToken, pPhrase); 115859 + } 115860 + 115861 + aPoslist = pExpr->pRight->pPhrase->doclist.pList; 115862 + nToken = pExpr->pRight->pPhrase->nToken; 115863 + for(p=pExpr->pLeft; p && res; p=p->pLeft){ 115864 + int nNear = p->pParent->nNear; 115865 + Fts3Phrase *pPhrase = ( 115866 + p->eType==FTSQUERY_NEAR ? p->pRight->pPhrase : p->pPhrase 115867 + ); 115868 + res = fts3EvalNearTrim2(nNear, aTmp, &aPoslist, &nToken, pPhrase); 115869 + } 115870 + } 115871 + 115872 + sqlite3_free(aTmp); 115873 + } 115874 + 115875 + return res; 115876 +} 115877 + 115878 +/* 115879 +** This macro is used by the fts3EvalNext() function. The two arguments are 115880 +** 64-bit docid values. If the current query is "ORDER BY docid ASC", then 115881 +** the macro returns (i1 - i2). Or if it is "ORDER BY docid DE