|
2013-03-07
| ||
| 21:14 | • Closed ticket [3567020edf]: Can't properly read UTF string with \0 symbol from a database. plus 2 other changes artifact: 0ff1a26fbc user: mistachkin | |
|
2013-03-06
| ||
| 23:06 | • Pending ticket [3567020edf]. artifact: 432f96cdbb user: mistachkin | |
| 23:05 | Properly handle embedded NUL characters in parameter and column values. Fix for [3567020edf]. check-in: c77dd425d4 user: mistachkin tags: trunk | |
| 22:14 | Preliminary changes to fix ticket [3567020edf]. Missing tests. check-in: bea6b5d82d user: mistachkin tags: tkt-3567020edf | |
| 22:02 | • Ticket [3567020edf] Can't properly read UTF string with \0 symbol from a database. status still Open with 3 other changes artifact: 4ced996a6a user: mistachkin | |
| 21:20 | • Ticket [3567020edf]: 3 changes artifact: 308163d17a user: mistachkin | |
| 21:16 | • Ticket [3567020edf]: 5 changes artifact: 918cf49600 user: mistachkin | |
| 10:07 | • New ticket [3567020edf]. artifact: 924351e33f user: anonymous | |
| Ticket Hash: | 3567020edf12d438cb7cf757b774ff3a04dc381e | ||
| Title: | Can't properly read UTF string with \0 symbol from a database. | ||
| Status: | Closed | Type: | Code_Defect |
| Severity: | Important | Priority: | NextRelease |
| Subsystem: | Native_Assembly | Resolution: | Fixed |
| Last Modified: |
2013-03-07 21:14:01 13.11 years ago |
Created: |
2013-03-06 10:07:42 13.11 years ago |
| Version Found In: | 1.0.84 | ||
| User Comments: | ||||
anonymous added on 2013-03-06 10:07:42:
I have an UTF string with \0 character (string s = "beforeNull\0afterNull";) and text field in a sqlite table.
When I tried to insert the string into table text field and then read it from the database I noticed that resulting string value was truncated after \0 character ("beforeNull"). Database contains whole string - (before \0 and after \0 parts) - it is been detected by opening database in text editor so problem is no read side.
OBSERVED: First UTF symbol \0 is not allowed to be used in UTF texts that are stored in sqlite tables.
EXPECTED: Strings that are being read from sqlite table can contain any UTF8(16) symbols including \0 symbol. Read operation doesn't truncate information from the database.
I tried to debug the issue and investigated that SQLite3.GetText() function is being used to grab text value. Inside there is a code to determine length of a resulting string
//////////////////////////
int len.
UnsafeNativeMethods.sqlite3_column_text_interop(stmt._sqlite_stmt, index, out len);
//////////////////////////
Unfortunately len return value is a string length till first \0 character.
Just for experiment I invoked another code to get length of the string
//////////////////////////
len = UnsafeNativeMethods.sqlite3_column_bytes(stmt._sqlite_stmt, index);
//////////////////////////
that returns original whole string length, and resulting UTF8ToString(ptr, len) contains all string sybmols with \0 in the middle.
( I can't say it working solution - that is just to illustrate problem)
See code snippet to reproduce the issue below.
///////////////////////////////////
public static void IssueWith0Character()
{
const string sql = "DROP TABLE IF EXISTS SomeTable;" +
"CREATE TABLE SomeTable (SomeField TEXT not null);"
+ "INSERT INTO SomeTable (SomeField) Values ( :value )";
var csb = new SQLiteConnectionStringBuilder
{DataSource = "stringWithNull.db", Version = 3};
// string with '0' character
const string stringWithNull = "beforeNull\0afterNull";
using (var c = new SQLiteConnection(csb.ConnectionString))
{
c.Open();
using (var cmd = c.CreateCommand())
{
var p = new SQLiteParameter(":value", DbType.String) {Value = stringWithNull};
cmd.CommandText = sql;
cmd.Parameters.Add(p);
cmd.ExecuteNonQuery();
}
using (var cmd = c.CreateCommand())
{
cmd.CommandText = "SELECT SomeField FROM SomeTable;";
var restoredValue = (string) cmd.ExecuteScalar();
Debug.Assert(stringWithNull == restoredValue);
}
}
}
///////////////////////////////////
mistachkin added on 2013-03-06 21:20:10:
The root cause appears to be improper use of strlen()/wcslen() by the sqlite3_column_text_interop() and sqlite3_column_text16_interop() functions present in the interop assembly. mistachkin added on 2013-03-06 22:02:47:
The same issue exists in the sqlite3_value_text_interop() and sqlite3_value_text16_interop() functions in the interop assembly. The methods SQLite3.GetText, SQLite3.GetDateTime, SQLite3.GetParamValueText, SQLite3_UTF16.GetText, and SQLite3_UTF16.GetParamValueText also have similar issues. mistachkin added on 2013-03-06 23:06:25:
Fixed on trunk via check-in [c77dd425d4]. | ||||