#297 If mariadb is restarted, queuedaemons fail

For example, take this notice (published shortly after a mariadb restart): https://gnusocial.net/notice/14505811

The notice is not federated to other servers, only published locally. Restarting queuedaemons fixes (or, to be more precise, mitigates) the issue.

GS debug logs:

2023-05-01 19:12:15 LOG_ERR: [gnusocial.net:queuedaemon.php:2532] Error during Notice->getProfileTags() for id==14505811: [Profile:157720] DB_DataObject error []: MDB2 Error: unknown error 2023-05-01 19:12:15 LOG_ERR: [gnusocial.net:queuedaemon.php:2532] Distrib queue exception saving notice 14505811: [Reply] DB_DataObject error []: MDB2 Error: unknown error #0 /var/www/gnusocial/extlib/DB/DataObject.php(1627): Memcached_DataObject->raiseError('MDB2 Error: unk...') #1 /var/www/gnusocial/classes/Memcached_DataObject.php(762): DB_DataObject->_query('SELECT 0 /*M! +...') #2 /var/www/gnusocial/extlib/DB/DataObject.php(3135): Memcached_DataObject->_query('SELECT 0 /*M! +...') #3 /var/www/gnusocial/classes/Memcached_DataObject.php(103): DB_DataObject->query('SELECT 0 /*M! +...') #4 /var/www/gnusocial/classes/Managed_DataObject.php(91): Memcached_DataObject::multiGetClass('Reply', 'notice_id', Array, true, false) #5 /var/www/gnusocial/classes/Notice.php(1867): Managed_DataObject::multiGet('notice_id', Array) #6 /var/www/gnusocial/classes/Notice.php(1905): Notice->getReplies() #7 /var/www/gnusocial/lib/notices/distribqueuehandler.php(81): Notice->sendReplyNotifications() #8 /var/www/gnusocial/plugins/RedisQueue/classes/RedisQueueManager.php(58): DistribQueueHandler->handle(Object(Notice)) #9 /var/www/gnusocial/lib/util/iomaster.php(286): RedisQueueManager->poll() #10 /var/www/gnusocial/lib/util/iomaster.php(160): IoMaster->poll() #11 /var/www/gnusocial/scripts/queuedaemon.php(112): IoMaster->service() #12 /var/www/gnusocial/lib/util/spawningdaemon.php(188): QueueDaemon->runThread() #13 /var/www/gnusocial/lib/util/spawningdaemon.php(82): SpawningDaemon->initAndRunChild(1) #14 /var/www/gnusocial/lib/util/daemon.php(205): SpawningDaemon->run() #15 /var/www/gnusocial/scripts/queuedaemon.php(186): Daemon->runOnce() #16 {main}

2023-05-01 19:12:15 LOG_ERR: [gnusocial.net:queuedaemon.php:2532] Distrib queue exception saving notice 14505811: [Profile:157720] DB_DataObject error []: MDB2 Error: unknown error #0 /var/www/gnusocial/extlib/DB/DataObject.php(1627): Memcached_DataObject->raiseError('MDB2 Error: unk...') #1 /var/www/gnusocial/classes/Memcached_DataObject.php(762): DB_DataObject->_query('SELECT * \n FROM...') #2 /var/www/gnusocial/extlib/DB/DataObject.php(1192): Memcached_DataObject->_query('SELECT * \n FROM...') #3 /var/www/gnusocial/classes/GS_DataObject.php(134): DB_DataObject->find(1) #4 /var/www/gnusocial/extlib/DB/DataObject.php(904): GS_DataObject->find(1) #5 /var/www/gnusocial/classes/GS_DataObject.php(108): DB_DataObject->get('id', 157720) #6 /var/www/gnusocial/classes/Memcached_DataObject.php(49): GS_DataObject->get('id', 157720) #7 /var/www/gnusocial/classes/Managed_DataObject.php(50): Memcached_DataObject::getClassKV('Profile', 'id', 157720) #8 /var/www/gnusocial/classes/Notice.php(151): Managed_DataObject::getKV('id', 157720) #9 /var/www/gnusocial/classes/Notice.php(2897): Notice->getProfile() #10 /var/www/gnusocial/classes/Notice.php(2883): Notice->_inScope(NULL) #11 /var/www/gnusocial/plugins/OStatus/OStatusPlugin.php(142): Notice->inScope(NULL) #12 /var/www/gnusocial/lib/util/event.php(89): OStatusPlugin->onStartEnqueueNotice(Object(Notice), Array) #13 /var/www/gnusocial/lib/util/util.php(1691): Event::handle('StartEnqueueNot...', Array) #14 /var/www/gnusocial/lib/notices/distribqueuehandler.php(100): common_enqueue_notice(Object(Notice)) #15 /var/www/gnusocial/plugins/RedisQueue/classes/RedisQueueManager.php(58): DistribQueueHandler->handle(Object(Notice)) #16 /var/www/gnusocial/lib/util/iomaster.php(286): RedisQueueManager->poll() #17 /var/www/gnusocial/lib/util/iomaster.php(160): IoMaster->poll() #18 /var/www/gnusocial/scripts/queuedaemon.php(112): IoMaster->service() #19 /var/www/gnusocial/lib/util/spawningdaemon.php(188): QueueDaemon->runThread() #20 /var/www/gnusocial/lib/util/spawningdaemon.php(82): SpawningDaemon->initAndRunChild(1) #21 /var/www/gnusocial/lib/util/daemon.php(205): SpawningDaemon->run() #22 /var/www/gnusocial/scripts/queuedaemon.php(186): Daemon->runOnce() #23 {main}

Ok, not a solution, but it seems to work: including this line in /etc/systemd/system/mysqld.service:

ExecStartPost=/bin/sh /var/www/gnusocial/scripts/stopdaemons.sh

If you have monit installed and configured to boot queuedaemons, the previous line stops them, and monit reboots them within minutes.