Maintenance starts today at 18:00 EEST
PostgreSQL and MariaDB will get replica on my NAS, so I have to enable read-only mode. It would take maybe 30 minutes to transfer DB files.
Maintenance starts today at 18:00 EEST
PostgreSQL and MariaDB will get replica on my NAS, so I have to enable read-only mode. It would take maybe 30 minutes to transfer DB files.
Some quite time ago someone sent me a message about minecraft server not working. Pelican (previously Pterodactyl) panel with its “docker-runner” software was down for some reason. I saw an issue with db, but I was too distracted by the work, so I forgot about it for some time. Secondly, one of our users sent me message with media uploading issue, I tried to fix it, but self-hosted Minio S3 was returning Access Denied all the time. I was too distracted by the work, so I planned to do it after my work is done (unfortunately I forgot a quite about it and it was night, so yeah).
Today I tried to fix Minio, but it kept sending me errors like “Permission denied”. I tried to chmod the directory and that’s the moment, where I spotted the issue - filesystem was in “read-mode”. I tried to check btrfs logs and it said something like 20k+ write errors and 600+ read errors and “forced readonly”. Tried to check mdadm - everything is good, but when I tried to check SMART on every disk, that’s what happened - /dev/nvme3n1 keeps spamming Input/Output error. After reboot the disk is gone, raid was RAID0, so it’s unrecoverable.
What happened and what happens next?I still don’t know if it’s a disk or the nvme x4 controller (small chance, but whatever), but if it is disk failure - congratulations, we’ve lost at least 1 year of media in pl.m0e.space (maybe I could find old backups when I migrated from GMhost or from docker, when it was in local-mode, but it was a long time ago), pelican panel with minecraft (i still have backups, but for some reason I don’t have the newest backups for current minecraft server), wiki server and it’s data about our minecraft server and maybe some other stuff that I don’t remember.
If it recovers somehow - we can restore all of this, but that’s a very small chance, unfortunately.
Anyway I’ll restore Minio S3 on HDD and make replica or smth like that (remotely or locally - we’ll see, I’ll probably won’t sleep today)
ConclusionsDO THE FUCKING BACKUPS, don’t wait tomorrow or weekends - DO IT NOW. Now I’ll spend some cash to make at least 2 backup storages for all of the servers + increase storage size for hogyoku (Git server, Status page and XMPP)
Also i fucking hate myself
SearX is very slow and i use it on ungoogled-chromium browser on my “Counter Strike 2 Windows 11”. Recently i found new promising search engine that is fast and more memory-efficient: 4get. I decided to migrate our slow SearX to 4get. It’s available at https://search.m0e.space
unrelated sectionAlso I replaced old info on the index page. Check our Minecraft server!
Finally, after almost 3 weeks of downtime, pl.m0e.space is back up and running! Your timeline may be empty and out of date, but wait a couple of hours and it will fill in by itself as the federation is now working properly.
This doesn’t change the fact that we will be migrating to another backend, but it won’t happen until we implement the core pleroma features in GTS.
Also: ALL YOUR DATA, INCLUDING POSTS, REPLIES, etc., WILL BE MIGRATED TO GOTOSOCIAL WHEN WE DO THE MIGRATION. You will not need to create an account again (only possibly set a password via email if gotosocial uses a different cryptographic method to encrypt your password in the database). If I cannot guarantee this, then GoToPleroma (current development name) will be a dead project.
Soapbox replaced Mangane (temporarily, probably) due to broken frontend settings. See mangane.social.m0e.space (clear site data may be required).
mangane.social.m0e.space domain may be changed to soapbox.social.m0e.space. I’ll inform you here before these changes.
Due to dead server and lost data (temporarily) we now have three big news:
m0e.space website is updatedNow this website is Hugo blog with custom theme, where you can get all necessary info and news. You can get RSS feed here. You don’t need to join telegram group, but all news will be crossposted on telegram and fediverse account that will be created after some time.
Akkoma is dead. (temporarily)I’ll bring it back alive when server will be online, we have alternative instance: GoToSocial, so go ahead and sign up. You can skip thinking about writing reason text and tell that you was a member of Akkoma instance and I’ll probably approve you as soon as I open your signup letter. Make sure to check account settings after registration, it has most of the features that Akkoma had and also brings new cool features, such as page style!
I talked about it on Fediverse (link is dead because pl.m0e.space is dead lol) and I really want to migrate to more stable and convenient (at least for me) ActivityPub backend such as GoToSocial. It’s more lightweight, it doesn’t have strange shit with admin panel (AdminFE fuck you), it works for me out-of-box without spending a couple of days to understand, why Akkoma can’t be upgraded again with it’s fucking Elixir dependencies, etc.
BUT! I want to migrate all features that I love in Akkoma (PleromaFE, emoji react, Deepl translate, etc.), so I decided to make my own fork that’ll cover all this convenient things.
social.m0e.space will become stable polygon for Akkoma-GoToSocial migration after original Akkoma instance revival, lolibuttson.me (this domain costs me 0$ so i don’t care about name) - unstable version with quick fixes and goofy errors (also it’s an anarchy server, so you can do anything you want except breaking Finnish laws, so I’;ll ban you if you do that).