Skip to content

Servers

  • in Servers
  • 4 min read

Files, Git, and ZFS

ZFS

Yesterday I found two old 512GB laptop hard drives and I put them together into my modded minecraft server because some friends told me i should use ZFS and well I had two okayish drives so I put them in a mirror for redundancy.

I also got some scripts put together to get alerted for my ZFS pool's health and the S.M.A.R.T status of my drives juuusssssttt incase because they are old, and I want to replace them soon with some proper 4TB NAS drives

The scripts can be found here for the ZFS one and over here for the S.M.A.R.T one.

output of the monitors

The scripts have comments for the proper way to use them but using --all will make it always send to the webhook and without that flag it will only send on errors.

Git Server

Since i now have a decent reliable storage situation (aisde from fires or other major issues) and finding Forgejo which is a fork of Gitea which I did not want to set up due to usage of AI.

It is hosted over at git.evalyngoemer.com

If you or a friend needs an account Contact Me and if your projects are good enough and suitable to be on my instance you may have an account.

My projects are also moved off GitHub to Codeberg which has the same repos as my instance for my projects so people can still be able to contribute until forgejo gets federation done.

File Server

I also set up a Copyparty server on that server, and it has a public read only share for me to share a few files but there isnt much to see there if you weren’t sent a link but its worth mentioning.

Uptime Monitoring Part 2: Episode 2: DIY Monitoring Tool

Well well well. Here we are again. Today i woke up to ~130 notifications from my uptume kuma instance.

EVERY SINGLE GOD DAMN ONE WAS A FALSE POSITIVE

WHAT. THE. FUCK.

So. I've had enough with this BS and made my own uptime monitor tool basicUptime in golang

I added are bare minimum features i need and a basic UI you can see here. Made nginx cache the backend to make it preform decently, and it works a bit better than Uptime Kuma at least and I may add support for instances asking others about status to see if it should send a ping.

For now though It's perfectly fine and il demo it without pings just in case it's gonna false positive too and hopefully it works a lot better

I will add support for Minecraft Java server monitoring though at some point.

  • in Servers
  • 5 min read

Uptime Monitoring Part 3: Tokyo Drift (Finale)

Okay. So. I'm gonna level with you. The previous "fix" helped but there were false positives sadly.

So I caved and a few days ago i got some stock notifications for servers and I got a new server in the family located in Bern, Switzerland with 1GB ram 1/4th of an 7950x3d (burstable) 25mbps networking (burstable to 10gbps)

Obligatory fastfetch of the VM
fastfetch program output on the VM in switzerland

I got Uptime Kuma setup on there with PM2 & nginx and got it all loaded to monitor my servers.

And you would think this would work. aaaaaandd drum roll please. Did it work? NOPE.

As it turns out optime monitoring is a pain in the ass like anything on computers and is hard to make right.

Uptime Kuma keeps dropping requests from my discord bot that has a perfectly valid user agent of EvalynBot/UptimeMonitor/1.1 and nothing should be triggering it to drop. I had even implemented retries and yet it still fails

John ULTRAKILL

FAILURE AFTER FAILURE AFTER FAILURE
AFTER FAILURE AFTER FAILURE AFTER
FAILURE AFTER FAILURE AFTER FAILURE
AFTER FAILURE AFTER FAILURE

THE RESULTS REFUSE TO ALTER

AGAIN AND AGAIN AND AGAIN AND AGAIN AND
AGAIN AND AGAIN AND AGAIN AND AGAIN AND
AGAIN AND AGAIN AND AGAIN AND AGAIN AND
AGAIN AND AGAIN AND AGAIN AND AGAIN

MY FAITH BEGINS TO FALTER

So I give up I’ll just give it a bunch of retries and widen windows of things so it won't bother me or make the website say its down when it shouldn’t.

If it actually is down it's going to alert me anyway so it should be fine for a smaller scale personal deployment, and maybe it's just a bug with the software.

Maybe I should write my own? Maybe.


TLDR:
Uptime monitoring sucks and I got a proper dedicated VPS for it now.
Learn from my lessons and try to keep sane when doing this crap.

  • in Servers
  • 3 min read

Uptime Monitoring Part 2: Electric Boogaloo (False Positives)

So when setting up my uptime monitor I was noticing there were false positives happening about every 8 to 12 hours. I am on shared hosting so that may have been why, but it was strange.

I ended up asking some peeps on the discord and made a ticket, and eventually I got the owner saying that it was possibly due to an nproc limit being hit. I couldn’t see it in the monitor but there were several threads called libuv-worker and all together there were about 50 threads. The limit is 50, but this was really strange for having a basic uptime monitor running.

I look into this, and it turns out this is a thing node uses for async work but by default there is meant to be 4. I even look into the code and nothing should be changing it. But these servers are running on CPUs with 96c/192t cpus so something may have been funky.

So I looked more into and I can set UV_THREADPOOL_SIZE to a value and force it. I set it to 2 to be reasonable and... It worked!

But it seems after a bit it decided to spawn two extra threads and become 4 but it is still way more in line at ~20 threads and using less memory.

There should not be any more false positive on the status page hopefully. It's been about a day or two without any so and I have disabled retries so we should be all good!

  • in Servers
  • 2 min read

Uptime Monitoring

I have been meaning to set up some kind of monitoring for my services but my VPS provider (i would like this monitor to be actually up 24/7 XD) is always out of stock so I never got a server for this. But I had forgotten I have a shared hosting plan with them that can run NodeJS and Uptime Kuma is light enough so i was able to put it up on there!

It should be fineee as its only uses a small bit of ram and CPU pinging my servers and worst case il take it down for a bit and get a VPS. But it seems I have a limit of 2G of ram on the panel and 1GB is stated on the plan and I’m only using ~450mb so it should be all good and I get a neat uptime monitor that should have high availability.

Check here or the sidebar to see the status monitor
And check here for the path.net status monitor because they keep goin down XD

(note these sites will not auto redirect to https for whatever reason. Please switch to https if possible)
This has now been fixed