dan
|
de8aa17eb5
Fix a "applying zero offset to null pointer" usan error in the fts5 trigram tokenizer.
|
1 month ago |
dan
|
0cd2ffffb7
Fix the fts5 trigram tokenizer so that it handles non-nul-terminated strings.
|
1 month ago |
drh
|
4924847cb9
Avoid the possibility of buffer overrun in the READ_UTF8 macro by using
|
2 months ago |
dan
|
b651084713
Add tests to restore coverage of fts5_tokenizer.c.
|
4 months ago |
dan
|
29d5e43ad4
Update the porter tokenizer to use locales.
|
4 months ago |
drh
|
5af9fd53f4
Revision to check-in [d9f726ade6b258f8] so that OOM and other unrelated failures
|
4 months ago |
drh
|
e9b919d550
Improved robustness of parsing of tokenize= arguments in FTS5.
|
4 months ago |
dan
|
32ca0dbcdf
Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram".
|
7 months ago |
dan
|
7b0fd0c564
Fix a problem with amalgamation builds on this branch.
|
1 year ago |
dan
|
e186fe20f5
Add the "remove_diacritics" option to the fts5 trigram tokenizer.
|
1 year ago |
drh
|
d36f588f31
Fix harmless compiler warnings about unused function parameters.
|
4 years ago |
dan
|
95dca8d0cf
FTS5 does not handle tokens that contain embedded nul characters. Prevent the trigram tokenizer from returning such tokens. Fix for [2ba5930b2].
|
4 years ago |
dan
|
ccf578d435
Add tests for the trigram tokenizer. Fix minor issues.
|
4 years ago |
dan
|
33a99fad08
Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer.
|
4 years ago |
drh
|
3b574e4ea9
Use the 64-bit memory allocator interfaces in extensions, whenever possible.
|
5 years ago |
drh
|
2d77d80a65
Use 64-bit math to compute the sizes of memory allocations in extensions.
|
6 years ago |
drh
|
f9231c34eb
Fix harmless compiler warnings.
|
6 years ago |
dan
|
eefc72d12f
Avoid an undefined left-shift operation in fts5 caused by malformed utf-8
|
6 years ago |
dan
|
b163b57212
Fix problems in fts5 found by ASAN.
|
6 years ago |
dan
|
e89feee5c3
Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5
|
6 years ago |
dan
|
b80bb6ce88
Add the "categories" option to the unicode61 tokenizer in fts5.
|
6 years ago |
dan
|
22e8356368
Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code.
|
8 years ago |
dan
|
e9eb1593f5
Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode.
|
9 years ago |
dan
|
3e6a141130
Fix some harmless gcc compiler warnings. Mostly in fts5, but also two in the core code.
|
9 years ago |
mistachkin
|
b9becaa268
Fix even more harmless compiler warnings.
|
9 years ago |
mistachkin
|
cdabd7bd50
Fix harmless compiler warnings.
|
9 years ago |
dan
|
9c671b741c
Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same.
|
9 years ago |
dan
|
ee0c0a8de3
Another change to the fts5 tokenizer API.
|
9 years ago |
dan
|
57e0add3f9
Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document.
|
9 years ago |
dan
|
79e2347fdf
Fix a bug in the fts5 porter tokenizer preventing it from passing xCreate() arguments through to its parent tokenizer.
|
9 years ago |