• 6 Posts
  • 107 Comments
Joined 2 years ago
cake
Cake day: June 12th, 2023

help-circle








  • Your use-case and situation seems very close to mine except I specifically do not host communities.

    First of all, you can run as many services from single nginx as you want (or can handle), usually you do this by having each service on it’s own (sub)domain and routing it all to the same IP, nginx then proxies the requests to the corresponding service running locally on a given port (see nginx reverse proxy).

    I would definitely recommend docker images unless you have specific needs, afaik the ansible recipe installs and manages a docker compose project too (unless they also added official bare-bones ansible setup). Might be wrong here, I do docker and manage it myself, updating is usually a file edit and two commands away.

    About the VPS being enough - from my monitoring, every foreign subscribed community increases the load, with bigger/more active communities increasing it more.
    The main limiting resource for my setup is disk space, sometime ago I’ve calculated my database size is increasing about 1G per month with about 500 subscribed communities and that’s only the postgresql database size without any media. The stats from my s3 provider (you can host images locally too), hint that I am gaining 1-5GBs of media per month.

    I don’t have any metrics how much the amount of active users drains the server as my instance is intentionally small, but I can imagine that having 10-100-1000 active users at the same time would drastically increase the load of at least postgres as well as increase the bandwith.

    And about my setup for comparison, I am renting a dedicated server from Hetzner (AX41-NVMe) running a bunch of other services as well (minecraft server, factorio server, file sharing service, …) and as of the last 30 days my monitoring reports the “average” load average (same for all 1/5/15m) being around 1 core (out of 12 core processor, 6*2 smt).
    Memory is sitting at about 50% month average out of 64G.
    Though, most of the services are really under-utilized (minecraft) or don’t require much (factorio).

    Rule of thumb, if your users subscribe to a lot of outside communities expect at least increased disk space consumption, at worst also increased bandwidth and load.
    If any of your hosted communities get popular on the wider fediverse, definitely expect increased bandwith and load - more servers hitting your server with more data (upvotes, comments, edits…) means nginx, lemmy and postgres also need to process more.
    At baseline there will be a lot of a spiky but small chatter from other instances and the biggest resource drain will be postgres.

    I wouldn’t personally go into this with anything less then 4 vCPUs, 32G of RAM and non-shared/virtual storage (disk latency kills postgres performance).








  • Numbers from my instance, running for about a 1 year and with average ~2 MAU. According to some quick db queries there is currently 580 actively subscribed communities (it was probably a lot less before I used the subscribe bot to populate the All tab).

    SELECT pg_size_pretty( pg_database_size('lemmy') ): 17 GB

    Backblaze B2 (S3) reports average 22.5 GB stored. With everything capped to max 1 USD, I pay cents - no idea how backblaze does it but it’s really super cheap, except for some specific transactions done on the bucket afaik, which pictrs does not seem to do.

    According to my zabbix monitoring, two months ago (I don’t keep longer stats) the DB had only about 14G of data, so with this much communities I am getting about 1.5G per month (it’s probably a bit more as I was recently prunning stuff from some dead instances).

    Prometheus says whole lemmy service (I use traefik) is getting within about 5 req/s (1m average) though if I go lower it does spike a lot, up to 12 requests within a second then nothing for few.