As others said, spin down the drives when they’re not in use. Make sure power saving is enabled on the drives and tune them to spin down after some appropriate amount of time. (hdparm lets you customize it on Linux)
Consider also sleeping the NAS when not in use. You can try using Wake-on-LAN to remotely wake it up when you need to use it. Saves on electricity and heat! You could also sleep it on a schedule, in case you need to be online for backups to run at particular times.
This is flat out wrong. If you’re the copyright owner, you’re not licensing the code to yourself. The AGPL is the license under which they’re making the open source version available to YOU. The version they run themselves is proprietary.
Look into DeltaChat
You need that SRE team you said you don’t have. :)
I’m sold on the sales pitch for it, but deflect.ca resolves to a Hetzner box in their Washington DC datacenter. I have a 112 ms ping to it from Toronto.
How do you sell a CDN when your own website is hosted out of the country at a PoP that’s that far away?
I totally understand. It sucks that there’s not really any options in between these two extremes.
If I can ramble a bit more - forget the Anycast bit. If you run your own DNS server(s), you can just configure them to respond based on the geographic location of the requester. PowerDNS is pretty easy to set up for this. You could run your own DNS just for the image domain. You basically run PowerDNS authoritative server, set up your zones and the geoip stuff, then slap dnsdist in front of it to be publicly exposed. dnsdist has anti-DDoS features and loadbalancing in it, in case you need it down the road.
Since it’s just for static images, you can have a higher TTL so you don’t need to worry about distributing the DNS servers. (ie. the DNS lookup might not be super fast since it could go across the country, but it doesn’t matter since that lookup is only going to happen every TTL period on each client, which can be high.)
One suggestion to consider for Lemmy.ca is to move your images and other easily-cacheable content to a different domain or subdomain, to give you more flexibility.
eg. If you serve your static assets off of lemmyimages.ca, then you can have only that behind a CDN, Cloudflare, or some other hosting with DDoS scrubbing. It gives you more flexibility to cope with various situations.
2tb a week isn’t much (6 mbps on average?). It’s pretty easy to set up nginx as a caching reverse proxy and spin that up on a couple of VPSes, but the annoying bit is you need to anycast your own IP address space in order for it to be functional as a CDN.
I’m not aware of any Canadian-owned CDNs either… OVH has one but they’re pretty crappy as a company. Beware of whitelabelled CDNs too, even some of the CDNs provided by big cloud hosting companies are actually whitelabelled from another company.
As someone who works in tech, I feel like the sales pitch for VPNs is snakeoil for the average person. All you’re doing is trading out your own ISP spying on you for another ISP who can spy on you. I know I have rights if my ISP spies on me in my own country, but if my traffic is all egressing via a foreign country, I may have zero privacy rights in that country.
If you need to hide your torrenting activity, just use a seedbox. If you need to hide from the NSA, none of this is going to help you. And TLS basically does the rest.
Same with DNS-over-HTTPS - the DNS server is where they’re going to be spying on you!
Wrapping up this thread, I really appreciate all the opinions and experiences everyone shared! Gave me lots of new perspectives to think about.
Yeah, this might be the way to go. OpenWRT supports hardware NAT with many of these ARM-based routers like many of the MediaTek-based ones, which gives them super high throughput at very low CPU usage. The efficiency blows x86 out of the water. The ability to migrate your OpenWRT config to new hardware (real or virtual) in the future means you kinda get the best of both worlds…
Do not use an SSD for cold storage - it will fail. SSDs need to be plugged in every once to refresh the charge in their NAND, otherwise they’ll lose the data.
This is not a theoretical thing - I’ve had a good Samsung 850 Pro drive fail while being off for 2 years.
Thanks, this is good data!
How fast is your internet?
Do a speed test and run htop… you’ll see CPU usage only on one core spiking. Not a big deal if your CPU can handle it, but the AMD GX-412TC in the APU2 I was using is too slow.
Even if the virtualized router is down, I’ll still have access to the physical server over the network until the DHCP lease expires. The switch does the work of delivering my packets on the LAN, not the router.
Thanks for the tip about the pfSense limit. After running pfSense for like 8 years, my opinion is that is flush with features but overall, it’s trash. Nobody, not even Netgate, understands how to configure limiters, queues, and QoS properly. The official documentation and all the guides on the internet are all contradictory and wrong. I did loads of testing and it worked somewhat, but never as well as it should have on paper (ie. I got ping spikes if I ran a bandwidth test simultaneously, which shouldn’t happen.) I don’t necessarily think OpenWRT is any better, but I know the Linux kernel has multithreaded PPPOE and I expect some modern basics like SQM to work properly in it.
The other thing to keep in mind is to pass through physical nics. Using just the vnics will potentially lead to security risks. That’s the reason I went back to physical fws.
I could throw an extra NIC in the server and pass it through, but what are the security risks of using the virtualized NICs? I’m just using virtio to share a dedicated bridge adapter with the router VM.
If you just use 2 nodes, you will need a q-device to make quorum if you have one of the nodes down
I could just use VRRP / keepalived instead, no?
I should try Proxmox, thanks for the suggestion. I set up ZFS recently on my NAS and I regret not learning it earlier. I can see how the snapshotting would make managing VMs easier!
That is pretty sweet. I have a second server I could use for an HA configuration of the router VM. I’ve been meaning to play around with live migrations (KVM) so this could be a cool use case for testing.
I’m late to the party here, but did you consider just paying for Mattermost? If it meets your needs, and your organization has 250 people, the cost for licensing is going to be relatively small compared to your IT budget (right?). They have “contact us” pricing, which means you can negotiate it.