[Architecture] Hub-Spoke model for federation? · Issue #3245 · LemmyNet/lemmy
github.com
Question One of the biggest issues I am currently seeing with Lemmy, is federation. Either... Federation between instances having issues. Federation is backed up. Federation / Syncing is not scalin...

https://github.com/LemmyNet/lemmy/issues/3245

I posted far more details on the issue then I am putting here-

But, just to bring some math in- with the current full-mesh federation model, assuming 10,000 instances-

That will require nearly 50 million connections.

Each comment. Each vote. Each post, will have to be sent 50 million seperate times.

In the purposed hub-spoke model, We can reduce that by over 99%, so that each post/vote/comment/etc, only has to be sent 10,000 times (plus n*(n-1)/2 times, where n = number of hub servers).

The current full mesh architecture will not scale. I predict, exponential growth will continue to occur.

Let’s work on a solution to this problem together.

(Also- as federation has been completely broken on this particular server for me- there is a good chance I will not be able to see, or reply to anything posted below… That is, also assuming this even posts correctly to this server.)

  • PlutoniumAcid@lemmy.worldEnglish
    1·
    2 years ago

    Is your logic faulty?

    Yes, there are 50M total connections between 10k nodes, but when one server sees one comment/updootmwhatever, then that server “only” needs to send it to the 10k peers.

    Those peers do not need to pass it on to all other peers; that would mean 50M messages total, but again, not from each peer.

  • deejay4am@lemmy.worldEnglish
    1·
    2 years ago

    Also this post from GitHub explaining why this entire premise is wrong:

    Great presentation and write up - but this is just not how any of this works. The whole thing is based on a premise that just isn’t true.

    Reposting my reply from lemmy:

    The way federation actually works:

    A user on lemmy.ml subscribes to a community on lemmy.world. Say, [email protected]

    Assume that this user is the first lemmy.ml user to do so - basically what happens is the lemmy.world community sees that a member of a never before seen instance just subscribed. [email protected] then adds lemmy.ml to its list of instances it needs to tell whenever something happens in the community.

    No matter how many users of lemmy.ml subscribe, this only happens once.

    Now when a user of sh.itjust.works upvotes a post on [email protected], the sh.itjust.works instance then tells [email protected] of this change. It accepts the change, then tells everyone on its list of instances that have subscribers on them.

    So essentially, sh.itjust.works talks to lemmy.world, lemmy.world tells everyone else. There is no “full mesh”. The instance hosting the community is the “hub”, everything else is a spoke.

    So if there’s 10,000 instances, and they all just so happen to have at least one subscriber to some community, each change will be sent out 9,999 times. Your “50 million” premise is just completely wrong and I’m not sure where it’s coming from.