Commit History

Author SHA1 Message Date
  dan de8aa17eb5 Fix a "applying zero offset to null pointer" usan error in the fts5 trigram tokenizer. 1 month ago
  dan 0cd2ffffb7 Fix the fts5 trigram tokenizer so that it handles non-nul-terminated strings. 1 month ago
  drh 4924847cb9 Avoid the possibility of buffer overrun in the READ_UTF8 macro by using 2 months ago
  dan b651084713 Add tests to restore coverage of fts5_tokenizer.c. 4 months ago
  dan 29d5e43ad4 Update the porter tokenizer to use locales. 4 months ago
  drh 5af9fd53f4 Revision to check-in [d9f726ade6b258f8] so that OOM and other unrelated failures 4 months ago
  drh e9b919d550 Improved robustness of parsing of tokenize= arguments in FTS5. 4 months ago
  dan 32ca0dbcdf Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram". 7 months ago
  dan 7b0fd0c564 Fix a problem with amalgamation builds on this branch. 1 year ago
  dan e186fe20f5 Add the "remove_diacritics" option to the fts5 trigram tokenizer. 1 year ago
  drh d36f588f31 Fix harmless compiler warnings about unused function parameters. 4 years ago
  dan 95dca8d0cf FTS5 does not handle tokens that contain embedded nul characters. Prevent the trigram tokenizer from returning such tokens. Fix for [2ba5930b2]. 4 years ago
  dan ccf578d435 Add tests for the trigram tokenizer. Fix minor issues. 4 years ago
  dan 33a99fad08 Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer. 4 years ago
  drh 3b574e4ea9 Use the 64-bit memory allocator interfaces in extensions, whenever possible. 5 years ago
  drh 2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions. 6 years ago
  drh f9231c34eb Fix harmless compiler warnings. 6 years ago
  dan eefc72d12f Avoid an undefined left-shift operation in fts5 caused by malformed utf-8 6 years ago
  dan b163b57212 Fix problems in fts5 found by ASAN. 6 years ago
  dan e89feee5c3 Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5 6 years ago
  dan b80bb6ce88 Add the "categories" option to the unicode61 tokenizer in fts5. 6 years ago
  dan 22e8356368 Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code. 8 years ago
  dan e9eb1593f5 Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode. 9 years ago
  dan 3e6a141130 Fix some harmless gcc compiler warnings. Mostly in fts5, but also two in the core code. 9 years ago
  mistachkin b9becaa268 Fix even more harmless compiler warnings. 9 years ago
  mistachkin cdabd7bd50 Fix harmless compiler warnings. 9 years ago
  dan 9c671b741c Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same. 9 years ago
  dan ee0c0a8de3 Another change to the fts5 tokenizer API. 9 years ago
  dan 57e0add3f9 Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document. 9 years ago
  dan 79e2347fdf Fix a bug in the fts5 porter tokenizer preventing it from passing xCreate() arguments through to its parent tokenizer. 9 years ago