#210 Delete unreachable profiles

Open
opened 4 years ago by postblue · 8 comments

After 5y+ running herds.eu I have lots of accounts that are no more active/reachable (quitter.se, et al.) and cluttering the install. It might be a good idea to make a script to clean those? Like:

  1. deleting profiles/previews/avatars/remote medias of unreachable instances
  2. unless a user has interacted with a notice (reply, fav, repeat), in that case save the notice and related medias
  3. remove instance/feeds of followers/followed

Moreover, I'm getting a good bunch of ERROR: unsubscribe request failed when running Ostatus/scripts/gcfeeds --delete, like:

[...]
https://mastodon.social/users/Petit_louis.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mastodon.social/users/Linguisticae.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/muchos.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/rezo.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mamot.fr/users/emmanuelc.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/etienne_cdl.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/Linuxine.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mamot.fr/users/jacqueschapman.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/e_jambon.atom (unsubscribe) ERROR: unsubscribe request failed.
https://social.targaryen.house/users/marsxyz.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mamot.fr/users/gibr.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/Asriel.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mastodon.pirateparty.be/users/HgO.atom (unsubscribe) ERROR: unsubscribe request failed.
https://octodon.social/users/Siphonay.atom (unsubscribe) ERROR: unsubscribe request failed.
https://framapiaf.org/users/Clailou.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mamot.fr/users/pascalou.atom (unsubscribe) ERROR: unsubscribe request failed.
https://mamot.fr/users/marseye.atom (unsubscribe) ERROR: unsubscribe request failed.
https://scholar.social/users/omanreagan.atom (unsubscribe) ERROR: unsubscribe request failed.
https://social.tchncs.de/users/Blort.atom (unsubscribe) ERROR: unsubscribe request failed.

Deleted 0 inactive feeds.
After 5y+ running herds.eu I have lots of accounts that are no more active/reachable (quitter.se, et al.) and cluttering the install. It might be a good idea to make a script to clean those? Like: 1. deleting profiles/previews/avatars/remote medias of unreachable instances 2. unless a user has interacted with a notice (reply, fav, repeat), in that case save the notice and related medias 3. remove instance/feeds of followers/followed Moreover, I'm getting a good bunch of ERROR: unsubscribe request failed when running Ostatus/scripts/gcfeeds --delete, like: ``` [...] https://mastodon.social/users/Petit_louis.atom (unsubscribe) ERROR: unsubscribe request failed. https://mastodon.social/users/Linguisticae.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/muchos.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/rezo.atom (unsubscribe) ERROR: unsubscribe request failed. https://mamot.fr/users/emmanuelc.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/etienne_cdl.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/Linuxine.atom (unsubscribe) ERROR: unsubscribe request failed. https://mamot.fr/users/jacqueschapman.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/e_jambon.atom (unsubscribe) ERROR: unsubscribe request failed. https://social.targaryen.house/users/marsxyz.atom (unsubscribe) ERROR: unsubscribe request failed. https://mamot.fr/users/gibr.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/Asriel.atom (unsubscribe) ERROR: unsubscribe request failed. https://mastodon.pirateparty.be/users/HgO.atom (unsubscribe) ERROR: unsubscribe request failed. https://octodon.social/users/Siphonay.atom (unsubscribe) ERROR: unsubscribe request failed. https://framapiaf.org/users/Clailou.atom (unsubscribe) ERROR: unsubscribe request failed. https://mamot.fr/users/pascalou.atom (unsubscribe) ERROR: unsubscribe request failed. https://mamot.fr/users/marseye.atom (unsubscribe) ERROR: unsubscribe request failed. https://scholar.social/users/omanreagan.atom (unsubscribe) ERROR: unsubscribe request failed. https://social.tchncs.de/users/Blort.atom (unsubscribe) ERROR: unsubscribe request failed. Deleted 0 inactive feeds. ```
Guillaume commented 4 years ago
Poster

Getting a lot of errors when updating ostatus profiles (30k+ profiles), like:

Updating OStatus profile 'http://quitter.se/user/122134' ... Could not reach profile page http://quitter.se/user/122134.
Getting a lot of errors when updating ostatus profiles (30k+ profiles), like: ``` Updating OStatus profile 'http://quitter.se/user/122134' ... Could not reach profile page http://quitter.se/user/122134. ```
aab commented 4 years ago

+1 on this. Deleting those profiles would save some space

+1 on this. Deleting those profiles would save some space
Diogo Cordeiro commented 3 years ago
Owner

ActivityPub's update_activitypub_profiles.php script deletes no-longer existing remote profiles. Don't know if OStatus has something like that...

ActivityPub's `update_activitypub_profiles.php` script deletes no-longer existing remote profiles. Don't know if OStatus has something like that...
aab commented 3 years ago

Things get worse when trying to delete an account with a lot of interactions: being unable to connect to dead nodes makes impossible the deletion. Update_activitypub_profiles.php runs here every week, so i guess it is not doing its job correctly...

Things get worse when trying to delete an account with a lot of interactions: being unable to connect to dead nodes makes impossible the deletion. Update_activitypub_profiles.php runs here every week, so i guess it is not doing its job correctly...
aab commented 3 years ago

An (auto) correction to my last post: as noted by @diogo, update_activitypub_profiles.php script deletes no-longer existing remote profiles. That is true, but only if the remote node is working.

An (auto) correction to my last post: as noted by @diogo, update_activitypub_profiles.php script deletes no-longer existing remote profiles. That is true, but only if the remote node is working.
Diogo Cordeiro commented 3 years ago
Owner

Well, of course, but how would you tell apart a dead instance from one that is having temporary issues? At most we could try to come up with a script that allows to delete local data from a given instance. But in general, this seems like a bad idea. You can't delete a profile and leave their previous posts.

Well, of course, but how would you tell apart a dead instance from one that is having temporary issues? At most we could try to come up with a script that allows to delete local data from a given instance. But in general, this seems like a bad idea. You can't delete a profile and leave their previous posts.
aab commented 3 years ago

You are right, @diogo, that's something tricky. Maybe implementing that script and using it for nodes that you know for sure aren't coming back (i know some of them).

You are right, @diogo, that's something tricky. Maybe implementing that script and using it for nodes that you know for sure aren't coming back (i know some of them).
Diogo Cordeiro commented 3 years ago
Owner

TODO (TFN):

  • After the exponential queue retries, stop attempting potentially dead nodes and prompt the admin for action in the /panel
  • Review admin tools regarding dead actors and its local cached presence
TODO (TFN): * After the exponential queue retries, stop attempting potentially dead nodes and prompt the admin for action in the /panel * Review admin tools regarding dead actors and its local cached presence
Sign in to join this conversation.
No Milestone
No assignee
3 Participants
Loading...
Cancel
Save
There is no content yet.