Lazy Guilds

Note: This is an edit of Luna's Lazy Guilds page with emphasis on understanding and usage rather than implementation. If you'd like to understand how to get the members of a guild, this'll help.

Note: This is not a complete document on how lazy guilds work. More info has been added onto Luna's original docs.

Known history

Lazy guilds were first implemented because of known stresses caused by the Fortnite Discord guild. The existing tooling to handle big guilds were not enough for something as big as Fortnite (e.g Guild Sync).

Limitations of the old member list methods (Guild Sync, Request Guild Members) were already known to not scale well at large guilds. One of the first big examples was the original Blobs guild. The course of events goes as this:

  • An at-everyone ping happened in the guild, causing a big spike in internal server traffic as the notification subsystem fans it out to users.
  • Users open the server. Many. This part is important, because the client subscribes to the guild via (what that time was) guild sync.
  • The large influx of subscriptions also caused large subscriptions to the member list, and considering Blobs has many members, there's a very high chance of the first 1K members in the guild going online or offline. You were able to see the online counts changing quickly.
  • If enough subscriptions were done, the Guild genserver powering the Blobs guild would crash, and so all users would find the infamous "unavailable guild" icon on their clients.

(Keep in mind the above "line of events" was not in any way confirmed by Discord, and is a collection of "best guesses" by non-Discord-employees)

Lazy guilds were deployed on the guild and it was a success, now all the guilds have that feature enabled. The official client only uses Lazy Guild methods from now on. It is unknown when exactly this was done. The other guild presence fetch methods still exist and are functional for backwards-compatibility's sake are:

  • OP 8 Request Guild Members (client uses this when you search for members in the guild search bar)
  • OP 12 Guild Sync (client no longer uses this)

OP 14 "Lazy Request" (what to send)

This OP Code is undocumented. "Lazy Request" is an unofficial name taken by this documentation.

OP 14 is used by the client when wanting to load the member list of a guild.

When the client wants to load the member list, it describes its request in detail. That is what the channels field is for. It explicitly says which parts of the member list the client wants.

For example, in the official client, it preloads the first 100 members in the list, then requesting more as time goes by, so the only range being requested in its OP 14 is [0, 99].

Once a client requests a certain range, it is considered "subscribed" to that range and will receive respective GUILD_MEMBER_LIST_UPDATE events related to those ranges.

ASSUMPTION: typing field means the client wants to be subscribed to the ranges the currently-typing members are on.

ASSUMPTION: Ranges are positive and their general format is [100*n, 100*n+99]

  • Lazy Request, server sends back GUILD_MEMBER_LIST_UPDATE events [which are not limited to requested guild]
field type description
guild_id snowflake the guild id for the request
channels dictionary with channel id(s) (str) as keys {channel_id: member_range(s), ...}
members array of unknown unknown
activities boolean unknown
typing boolean unknown, check assumptions
threads boolean unknown

First, a request for lazy requesting is sent (example below):

{
    "op": 14,
    "d": {
        "guild_id": "708898615824875540",
        "typing": true,
        "threads": false,
        "activities": true,
        "members": [],
        "channels": {"708898615824875543": [[0, 99]]},
    },
}

If the guild is not a large guild, you can now continue onto requesting member lists (just run the same request without the tying, threads, activities, and members fields).
However, if the guild is a large guild, you'll receive a GUILD_CREATE response from the server. Once this occurs, you can then start requesting for the member list as usual. You can tell if the guild is large by checking whether the "unavailable" key in present for that guild in the READY response.

  • The [0,99] range is always requested for by the client. This can be checked by inspecting the ws communications while scrolling thru the member list.
  • other than the [0,99] range, 2 other ranges can be requested for. This requests for the 0,99 range:
{
    "op": 14,
    "d": {
        "guild_id": "708898615824875540",
        "channels": {"708898615824875543": [[0, 99]]},
    },
}

This requests for the 100,199 range:

{
    "op": 14,
    "d": {
        "guild_id": "708898615824875540",
        "channels": {"708898615824875543": [[0, 99], [100, 199]]},
    },
}

This requests for the 100,199 and the 200, 299 ranges:

{
    "op": 14,
    "d": {
        "guild_id": "708898615824875540",
        "channels": {"708898615824875543": [[0, 99], [100, 199], [200, 299]]},
    },
}

GUILD_MEMBER_LIST_UPDATE event (what is received)

This is the main event related to all lazy guild related work. It is sent by the Server to indicate updates to the member list, but only to the ranges the client specified in its OP 14.

Here's how the event looks like:

{
    "t": "GUILD_MEMBER_LIST_UPDATE",
    "s": s,
    "op": 0,
    "d": {
        "ops": [
            ...(operator objects)...
        ],
        "online_count": online_num (int),
        "member_count": member_num (int),
        "id": id (str),
        "guild_id": guild_id (str),
        "groups": [
            ...(group objects)...
        ],
    },
}
field type description
id string the list being updated, output of the list_id function
guild_id snowflake the guild id being referenced
ops list[Operator] operator objects
groups list[Group] group references

Operator Objects (SYNC, INVALIDATE, UPDATE, INSERT, DELETE)

Operator objects represent operations a client MUST act upon to achieve a synchronized member list with the server. The client MUST process each operator based on the given order by the GUILD_MEMBER_LIST_UPDATE event.

  • range is present if op is any of: "SYNC", "INVALIDATE"
  • items is present if op is equal to "SYNC"
  • index is present if op is any of: "INSERT", "DELETE", "UPDATE"
  • item is present if op is any of: "INSERT", "UPDATE"
field type description
op OperatorType operator type for the list
range list[int, int] range being operated upon
items list[SyncItem] the list of items related to the range given
index positive integer, includes 0 the item being acted upon
item SyncItem new item for the index

SYNC

If you just want the members, this is what to focus on. The SYNC response carries the member list for the requested ranges. This is an example response containing a SYNC operator:

{
    "range": [0, 99],
    "op": "SYNC",
    "items": [
        {"group": {"id": hoisted_role_id (str), "count": num (int)}},
        {"member": {userdata}},
        ...
    ]
}

At times, the "items" value can contain 0 items (just an empty array):

  1. This can happen if you request for a certain range mutliple times consecutively (there'll be no items if no members were updated).
  2. This can also happen if the requested range cannot be fetched. This happens for large guilds where the client cannot actually see the entire member list. This should not be confused with the INVALIDATE response, which happens when invalid ranges (outside of actual member ranges) are requested for.

INVALIDATE

This happens when an invalid range is requested for. For example, say a guild has 999 members and the [1000, 1999] range is requested for. example:

{
    "range": [1000, 1099],
    "op": "INVALIDATE",
    "items": []
}

UPDATE, INSERT, and DELETE

These operator objects describe changes to the 0,99 range (the lowest index value is 0 and the highest index value is 99).
To better understand indices, see the image below (index on the left, member list on the right):

Notice how the hoisted roles count as indices themselves.
Members are ordered alphanumerically by nickname (or username if they don't have a nickname). Roles are ordered by their order in the role settings.
update example:

{
    "op": "UPDATE",
    "item": {
        "member": {userdata}
    },
    "index": 90,
}

insert example:

{
    "op": "INSERT",
    "item": {
        "member": {userdata}
    },
    "index": 42,
}

delete example:

{'op': 'DELETE', 'index': 99}

Groups

Groups are hoisted roles (roles that are shown separately from everyone). A role can be turned into a hoisted role by checking "Display role members separately from online members" in the role settings:

field type description
id snowflake OR "online" OR "offline" group id
count integer the amount of members in that group

Implementation Notes (more info)

This section is non-normative. This is also aimed at server implementors wanting to implement Lazy Guilds. Clients only bother with the GUILD_MEMBER_LIST_UPDATE events

For SYNC events, server implementors can assume the index is an index in a list composed of, for every group in the list, by:

  • a group object
  • members in that group

With the given list, make slices out of the list based on the requested ranges by the client. This approach is known to work, but it is not known if the list generation method is the same used by Discord. Proceed with care.

For guidelines on when to update the list, here follows an informal list of state changes to a list. For a member:

  • A member moves from offline group to any other group (presence change).
  • A member moves from any group to the offline group (presence change).
  • A member moves from any group to any other group (role changes).
  • A member moves from a group G to the same G, while changing their position in the group (nickname changes).
  • A member moves from a group G to the same G, while not changing their position in the group ("simple" presence changes).

The fifth state change is the simplest to handle, it can be caused by a member going from online to idle while maintaining their roles. The others are more complex and cane grouped into two categories: simple presence update and complex presence update.

The "presence update" sentence is not always tied to a presence, it represents an update to member state, and while that includes roles, it also includes the member's nickname and current presence (status is enough). Complex presence updates always involve a member going from an old group to a new group. Simple presence updates only involve an update of the state at the same group.

When implementing complex presence updates, server implementors are recommended to use a single SYNC operator (given the correct information, e.g ranges) instead of calculating the correct order / appearance of INSERT / UPDATE / DELETE operators.

When implementing this server-side, it is valid behavior to send a SYNC operator to the client (given the correct range) instead of calculating the correct INSERT / UPDATE / DELETE operators.

The simplifying behavior is caused by unstability in the index property, It is currently unknown the governing rules about the property, esp. when it starts, or ends, or what enters the index or what doesn't, etc.

There are other related state changes to the list that are related to members but not related to the actual member state (or are, but aren't tied to a single member):

  • A role being created with hoisted property set to true (a new group).
  • A role having its position updated (group changes position).
  • A role having its hoisted property set to false (removal of a group).
  • A new member joining (creation of a member in a group, most probably auto-inserting them into the online group).
  • A member leaving (be it via a kick or a ban, they all have the same meaning to the member list).
  • A member updating its user information (such as avatar or username).