guile-sqlite3/guile-sqlite3

The following script should within a few seconds segfault. Assuming it's the same issue that I've been chasing, the segfault occurs during sqlite3_prepare_v2 and I believe it's triggered when Guile's garbage collection runs shortly prior.

I've been seeing this occur within the Guix Build Coordinator.

(use-modules (ice-9 threads)
             (rnrs bytevectors)
             (sqlite3))

(setvbuf (current-output-port) 'none)
(setvbuf (current-error-port) 'none)

(add-hook! after-gc-hook
           (lambda _
             (display "\n==== GC ====\n")))

;; Bait the garbage collector
(call-with-new-thread
 (lambda ()
   (while #t
     (make-bytevector 512 0))))

(define (open-db)
  (define file
    "prepare-segfault.db")

  (define flags
    (list SQLITE_OPEN_READWRITE
          SQLITE_OPEN_CREATE
          SQLITE_OPEN_NOMUTEX))

  (when (file-exists? file)
    (delete-file file))

  (sqlite-open file (apply logior flags)))

(define (work)
  (let ((db (open-db)))
    (sqlite-exec
     db
     "CREATE TABLE builds (uuid varchar, derivation_name varchar);")
    (while #t
      (sqlite-prepare
       db
       (string-append "SELECT uuid FROM builds"
                      " WHERE derivation_name = :derivation"))
      (display "."))))

(work)

I haven't got a fix yet, but I think I've tracked down the cause. If I comment out the (sqlite-finalize stmt) bit within (pump-stmt-guardian), I can no longer get a segfault.

Additionally, if I change SQLITE_OPEN_NOMUTEX to SQLITE_OPEN_FULLMUTEX, I can no longer get a segfault.

So, my takeaway so far is that the default behavior of the Guile bindings with respect to finalizing statements (which aren't explicitly finalized) is that this is incompatible with the anything other than operating Sqlite in the Serialized threading mode (which is what's activated by using SQLITE_OPEN_FULLMUTEX). This is probably because the Guile GC triggered statement finalizing is taking place in the GC thread, not the application thread that should be the sole thread interacting with SQLite.

From an application perspective, I guess I can avoid this situation by explicitly finalizing prepared statements, in the right thread, which avoids it being done from the GC thread which will potentially cause segfaults.

From a library perspective, I think this is something that could be handled better though. Memory leaking through not finalizing prepared statements might be preferable to segfaults. It's also possible to ask SQLite what the threading mode is, so maybe that could be used to avoid risky behavior if anything other than Serialized is in use. Depending on how safe the bindings are meant to be, this potentially relates to the issue that they're not thread aware at all, so there's nothing to stop you opening SQLite connections in a way that they should only be used in a single thread, and using that connection in multiple threads.

Anyway, I'm glad now that I might be able to stop my app segfaulting now :)

Hi @cbaines! Your are right: Guile has a finalization thread, so one must assume that objects can be finalized anytime concurrently with application activity. Thus, SQLite has to be thread-safe.

I agree that as a library, guile-sqlite3 should prevent that from happening.

Then again, we would need to look at the backtrace of segfaults: it might be that finalizers run early when in fact the underlying statement objects are still in use somewhere.

#17 Segmentation fault in sqlite-prepare