Morph MORPH
SIGN IN SIGN UP
dhruvhead / sqlite UNCLAIMED
0 0 244 C

COMMITS

/ test/fts4unicode.test
autosetup
January 4, 2021
D
Add extra test for handling of embedded nul characters in the fts4 unicode61 tokenizer.
dan committed
31afee9
May 30, 2017
D
Omit a test of codepoint 0x202F (non-break narrow space) from the fts3 ICU
dan committed
d26d2c7
June 15, 2015
D
Adjust ICU tests to account for recent changes in the official
drh committed
07d694c
December 19, 2013
D
Make sure errors encountered while initializing extensions such as FTS4
drh committed
f6b1a8e
October 12, 2013
M
Fix Unicode character encoding issues on Windows.
mistachkin committed
549bc3d
October 11, 2013
M
Fix test numbering.
mistachkin committed
cbc53fe
September 18, 2013
D
Test that the unicode61 tokenchars= and separators= options work with the fts3tokenize virtual table.
dan committed
6284d02
September 13, 2013
D
Add tests for the fts4 unicode61 tokenchars and separators options.
dan committed
f1d2670
August 30, 2013
D
Add a test for fts4 unicode61 option remove_diacritics=0.
dan committed
4339808
June 5, 2013
D
Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit fixes this so that all private use codepoints are considered alphanumeric. In other words, it fixes the handling of codepoints 0xE000, 0xF8FF, 0xF0000, 0xFFFFD, 0x100000 and 0x10FFFD.
dan committed
f2c9229
January 26, 2013
D
Add a single test case to fts4unicode.test to verify that title-case
drh committed
7c37e2f
June 19, 2012
D
Add tests to check that the "unicode61" and "icu" tokenizers both identify white-space codepoints outside the ASCII range.
dan committed
3aaa4cd
June 7, 2012
D
Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer.
dan committed
25cdf46
June 6, 2012
D
Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".
dan committed
754d3ad
May 26, 2012
D
If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.
dan committed
7946c53
D
Add coverage tests for fts3_unicode.c.
dan committed
7a79673
D
Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.
dan committed
ab322bd
May 25, 2012
D
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
dan committed
3d403c7