Merge e0f02743bd into ca9c376076

* Clarify Well-Known URIs

Signed-off-by: HarHarLinks <2803622+HarHarLinks@users.noreply.github.com>

* Fix section link

---------

Signed-off-by: HarHarLinks <2803622+HarHarLinks@users.noreply.github.com>

.github/workflows/main.yml

@@ -2,6 +2,7 @@ name: "Spec"
 
 env:
   HUGO_VERSION: 0.139.0
+  PYTHON_VERSION: 3.13
 
 on:
   push:
@@ -40,7 +41,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -59,7 +60,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -78,7 +79,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -120,7 +121,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -172,7 +173,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
       - name: "➕ Install towncrier"
         run: "pip install 'towncrier'"
       - name: "Generate changelog"

assets/scss/_styles_project.scss

@@ -316,13 +316,19 @@ Custom SCSS for the Matrix spec
   h2 {
     font-weight: $font-weight-bold;
     font-size: 1.3rem;
-    margin: 3rem 0 .5rem 0;
+    margin: 1.5rem 0 1rem 0;
   }
 
   h3 {
     font-weight: $font-weight-bold;
     font-size: 1.1rem;
-    margin: 1.5rem 0 .75rem 0;
+    margin: 1.5rem 0 1rem 0;
+
+  }
+
+  /* Reduce top margin of h3 if previous sibling is a h2 */
+  h2 + h3 {
+    margin-top: 1rem;
   }
 
   hr {
@@ -367,11 +373,6 @@ Custom SCSS for the Matrix spec
       }
     }
 
-    // add some space between two tables when they are right next to each other
-    & + table {
-        margin-top: 4rem;
-    }
-
     caption {
       caption-side: top;
       color: $dark;
@@ -443,6 +444,17 @@ Custom SCSS for the Matrix spec
     }
   }
 
+  /* Have consistent spacing around tables and examples */
+  table, .highlight {
+    margin-top: 0;
+    margin-bottom: 2rem;
+
+    /* We don't need the margin on the last child of the .rendered-data block */
+    &:last-child {
+      margin-bottom: 0;
+    } 
+  }
+
   pre {
     border: 0;
     border-left: solid 5px $secondary;

changelogs/client_server/newsfragments/2068.clarification

@@ -0,0 +1 @@
+Clarify behaviour when the `topic` key of a `m.room.topic` event is absent, null, or empty.

changelogs/client_server/newsfragments/2077.clarification

@@ -0,0 +1 @@
+Fix the example of the `GET /sync` endpoint and the `m.room.member` example used in several places.

changelogs/client_server/newsfragments/2083.clarification

@@ -0,0 +1,2 @@
+Clarify the format of third-party invites, including the fact that identity
+server public keys can be encoded using standard or URL-safe base64.

changelogs/client_server/newsfragments/2140.clarification

@@ -0,0 +1 @@
+Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks.

changelogs/identity_service/newsfragments/2083.clarification

@@ -0,0 +1 @@
+Clarify that public keys can be encoded using standard or URL-safe base64.

changelogs/internal/newsfragments/2081.clarification

@@ -0,0 +1 @@
+Adjust margins in rendered endpoints.

changelogs/internal/newsfragments/2088.clarification

@@ -0,0 +1 @@
+Replace Hugo shortcodes in OpenAPI output.

changelogs/server_server/newsfragments/2067.clarification

@@ -0,0 +1 @@
+Add a note to the invite endpoints that invites to local users may be received twice over federation if the homeserver is already in the room.

changelogs/server_server/newsfragments/2083.clarification

@@ -0,0 +1,2 @@
+Clarify the format of third-party invites, including the fact that identity
+server public keys can be encoded using standard or URL-safe base64.

changelogs/server_server/newsfragments/2140.clarification

@@ -0,0 +1 @@
+Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks.

content/client-server-api/_index.md

@@ -371,15 +371,23 @@ valid data was obtained, but no server is available to serve the client.
 No further guess should be attempted and the user should make a
 conscientious decision what to do next.
 
-### Well-known URI
+### Well-known URIs
+
+Matrix facilitates automatic discovery for the Client-Server API base URL and more via the
+[RFC 8615](https://datatracker.ietf.org/doc/html/rfc8615) "Well-Known URI" method.
+This method uses JSON files at a predetermined location on the root path `/.well-known/` to
+specify parameter values.
 
 {{% boxes/note %}}
+Diverging from the rest of the endpoints in the Client-Server spec, these files can not be provided
+on the base URL that the Client-Server API is reachable on, as it is yet to be discovered.
+Instead, they can be reached via HTTPS on the [server name](/appendices/#server-name)'s hostname as domain.
+
 Servers hosting the `.well-known` JSON file SHOULD offer CORS headers,
 as per the [CORS](#web-browser-clients) section in this specification.
 {{% /boxes/note %}}
 
-The `.well-known` method uses a JSON file at a predetermined location to
+The flow for auto-discovery is as follows:
-specify parameter values. The flow for this method is as follows:
 
 1.  Extract the [server name](/appendices/#server-name) from the user's Matrix ID by splitting the
     Matrix ID at the first colon.
@@ -415,10 +423,17 @@ specify parameter values. The flow for this method is as follows:
 
 {{% http-api spec="client-server" api="wellknown" %}}
 
-{{% http-api spec="client-server" api="versions" %}}
-
 {{% http-api spec="client-server" api="support" %}}
 
+### API Versions
+
+Upon connecting, the Matrix client and server need to negotiate which version of the specification
+they commonly support, as the API evolves over time. The server advertises its supported versions
+and optionally unstable features to the client, which can then go on to make requests to the
+endpoints it supports.
+
+{{% http-api spec="client-server" api="versions" %}}
+
 ## Client Authentication
 
 Most API endpoints require the user to identify themselves by presenting

content/client-server-api/modules/third_party_invites.md

@@ -5,8 +5,8 @@ This module adds in support for inviting new members to a room where
 their Matrix user ID is not known, instead addressing them by a third-party
 identifier such as an email address. There are two flows here; one
 if a Matrix user ID is known for the third-party identifier, and one if
-not. Either way, the client calls [`/invite`](#post_matrixclientv3roomsroomidinvite) with the details of the
+not. Either way, the client calls [`/invite`](#thirdparty_post_matrixclientv3roomsroomidinvite)
-third-party identifier.
+with the details of the third-party identifier.
 
 The homeserver asks the identity server whether a Matrix user ID is
 known for that identifier:
@@ -37,10 +37,12 @@ A client asks a server to invite a user by their third-party identifier.
 
 #### Server behaviour
 
-Upon receipt of an [`/invite`](#post_matrixclientv3roomsroomidinvite), the server is expected to look up the
+Upon receipt of an [`/invite`](#thirdparty_post_matrixclientv3roomsroomidinvite),
-third-party identifier with the provided identity server. If the lookup
+the server is expected to look up the third-party identifier with the provided
-yields a result for a Matrix User ID then the normal invite process can
+identity server by making a call to [`/_matrix/identity/v2/lookup`](/identity-service-api/#post_matrixidentityv2lookup).
-be initiated. This process ends up looking like this:
+If the lookup yields a result for a Matrix User ID then the normal [invite
+process](/server-server-api/#inviting-to-a-room) can be initiated. This process
+ends up looking like this:
 
 ```
     +---------+                         +-------------+                                    +-----------------+
@@ -66,10 +68,11 @@ be initiated. This process ends up looking like this:
         |                                     |                                                    |
 ```
 
-However, if the lookup does not yield a bound User ID, the homeserver
+However, if the lookup does not yield a bound User ID, the homeserver must store
-must store the invite on the identity server and emit a valid
+the invite on the identity server with a call to
-`m.room.third_party_invite` event to the room. This process ends up
+[`/_matrix/identity/v2/store-invite`](/identity-service-api/#post_matrixidentityv2store-invite)
-looking like this:
+and emit a valid [`m.room.third_party_invite`](#mroomthird_party_invite) event
+to the room. This process ends up looking like this:
 
 ```
     +---------+                         +-------------+                                               +-----------------+
@@ -101,15 +104,18 @@ looking like this:
         |                                     |                                                               |
 ```
 
-All homeservers MUST verify the signature in the event's
+The third-party user will then need to verify their identity, which results in a
-`content.third_party_invite.signed` object.
+request to [`/_matrix/federation/v1/3pid/onbind`](/server-server-api/#put_matrixfederationv13pidonbind)
+from the identity server to the homeserver that bound the third-party identifier
+to a user. The homeserver then exchanges the `m.room.third_party_invite` event
+in the room for a complete [`m.room.member`](#mroommember) event with
+`content.membership: invite` and a `content.third_party_invite` property for the
+user that has bound the third-party identifier. If the invitee is on a different
+homeserver than the inviting user, the invitee's homeserver makes a request to
+[`/_matrix/federation/v1/exchange_third_party_invite/{roomId}`](/server-server-api/#put_matrixfederationv1exchange_third_party_inviteroomid).
 
-The third-party user will then need to verify their identity, which
+All homeservers MUST verify the signature in the `m.room.member` event's
-results in a call from the identity server to the homeserver that bound
+`content.third_party_invite.signed` object.
-the third-party identifier to a user. The homeserver then exchanges the
-`m.room.third_party_invite` event in the room for a complete
-`m.room.member` event for `membership: invite` for the user that has
-bound the third-party identifier.
 
 If a homeserver is joining a room for the first time because of an
 `m.room.third_party_invite`, the server which is already participating
@@ -193,8 +199,8 @@ at any time - the completion is not shown in the diagram.
 
 H1 MUST verify the request from H3 to ensure the `signed` property is
 correct as well as the `key_validity_url` as still being valid. This is
-done by making a request to the [identity server
+done by making a request to the identity server's
-/isvalid](/identity-service-api/#get_matrixidentityv2pubkeyisvalid)
+[`/pubkey/isvalid`](/identity-service-api/#get_matrixidentityv2pubkeyisvalid)
 endpoint, using the provided URL rather than constructing a new one. The
 query string and response for the provided URL must match the Identity
 Service Specification.

content/server-server-api.md

@@ -119,7 +119,8 @@ to send. The process overall is as follows:
     server must present a valid certificate for the hostname.
 
 3.  If the hostname is not an IP literal, a regular HTTPS request is
-    made to `https://<hostname>/.well-known/matrix/server`, expecting
+    made to `https://<hostname>/.well-known/matrix/server` (according to 
+    [RFC 8615](https://datatracker.ietf.org/doc/html/rfc8615)), expecting
     the schema defined later in this section. 30x redirects should be
     followed, however redirection loops should be avoided. Responses
     (successful or otherwise) to the `/.well-known` endpoint should be
@@ -970,9 +971,8 @@ the event to other servers in the room.
 ## Third-party invites
 
 {{% boxes/note %}}
-More information about third-party invites is available in the
+More information about third-party invites is available in the Client-Server API
-[Client-Server API](/client-server-api) under
+under the [Third-party invites](/client-server-api/#third-party-invites) module.
-the Third-party Invites module.
 {{% /boxes/note %}}
 
 When a user wants to invite another user in a room but doesn't know the
@@ -985,38 +985,41 @@ API](/identity-service-api).
 
 ### Cases where an association exists for a third-party identifier
 
-If the third-party identifier is already bound to a Matrix ID, a lookup
+If the third-party identifier is already bound to a Matrix ID, a [lookup
-request on the identity server will return it. The invite is then
+request](/identity-service-api/#post_matrixidentityv2lookup) on the identity
-processed by the inviting homeserver as a standard `m.room.member`
+server will return it. The invite is then processed by the inviting homeserver
-invite event. This is the simplest case.
+as a [standard `m.room.member` invite event](#inviting-to-a-room). This is the
+simplest case.
 
 ### Cases where an association doesn't exist for a third-party identifier
 
 If the third-party identifier isn't bound to any Matrix ID, the inviting
-homeserver will request the identity server to store an invite for this
+homeserver will request the identity server to [store an invite](/identity-service-api/#invitation-storage)
-identifier and to deliver it to whoever binds it to its Matrix ID. It
+for this identifier and to deliver it to whoever binds it to its Matrix ID. It
-will also send an `m.room.third_party_invite` event in the room to
+will also send an [`m.room.third_party_invite`](/client-server-api/#mroomthird_party_invite)
-specify a display name, a token and public keys the identity server
+event in the room to specify a display name, a token and public keys the
-provided as a response to the invite storage request.
+identity server provided as a response to the invite storage request.
 
-When a third-party identifier with pending invites gets bound to a
+When a third-party identifier with pending invites gets bound to a Matrix ID,
-Matrix ID, the identity server will send a POST request to the ID's
+the identity server will send a request to the [`/3pid/onbind`](#put_matrixfederationv13pidonbind)
-homeserver as described in the [Invitation
+endpoint of the the ID's homeserver as described in the [Invitation
-Storage](/identity-service-api#invitation-storage)
+Storage](/identity-service-api#invitation-storage) section of the Identity
-section of the Identity Service API.
+Service API.
 
 The following process applies for each invite sent by the identity
 server:
 
-The invited homeserver will create an `m.room.member` invite event
+The invited homeserver will create an [`m.room.member`](/client-server-api/#mroommember)
-containing a special `third_party_invite` section containing the token
+invite event containing a special `third_party_invite` section containing the
-and a signed object, both provided by the identity server.
+token and a `signed` object, both provided by the identity server.
 
 If the invited homeserver is in the room the invite came from, it can
 auth the event and send it.
 
 However, if the invited homeserver isn't in the room the invite came
-from, it will need to request the room's homeserver to auth the event.
+from, it will need to request the inviting homeserver to auth the event
+at the [`/exchange_third_party_invite`](#put_matrixfederationv1exchange_third_party_inviteroomid)
+endpoint.
 
 {{% http-api spec="server-server" api="third_party_invite" %}}
 

data/api/client-server/support.yaml

@@ -22,9 +22,12 @@ paths:
       description: |-
         Gets server admin contact and support page of the domain.
 
-        Like the [well-known discovery URI](/client-server-api/#well-known-uri),
+        {{% boxes/note %}}
-        this should be accessed with the hostname of the homeserver by making a
+        Like the [well-known discovery URI](/client-server-api/#well-known-uris),
+        this endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
         GET request to `https://hostname/.well-known/matrix/support`.
+        {{% /boxes/note %}}
 
         Note that this endpoint is not necessarily handled by the homeserver.
         It may be served by another webserver, used for discovering support

data/api/client-server/sync.yaml

@@ -441,17 +441,57 @@ paths:
                           "state": {
                             "events": [
                               {
-                                "$ref": "../../event-schemas/examples/m.room.member.yaml"
+                                "content": {
+                                  "avatar_url": "mxc://example.org/SFHyPlCeYUSFFxlgbQYZmoEoe",
+                                  "displayname": "Example user",
+                                  "membership": "join"
+                                },
+                                "event_id": "$143273976499sgjks:example.org",
+                                "origin_server_ts": 1432735824653,
+                                "sender": "@example:example.org",
+                                "state_key": "@example:example.org",
+                                "type": "m.room.member",
+                                "unsigned": {
+                                  "age": 45603,
+                                  "membership": "join"
+                                }
                               }
                             ]
                           },
                           "timeline": {
                             "events": [
                               {
-                                "$ref": "../../event-schemas/examples/m.room.member.yaml"
+                                "content": {
+                                  "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF",
+                                  "displayname": "Alice Margatroid",
+                                  "membership": "join",
+                                  "reason": "Looking for support"
+                                },
+                                "event_id": "$143273582443PhrSn:example.org",
+                                "origin_server_ts": 1432735824653,
+                                "sender": "@alice:example.org",
+                                "state_key": "@alice:example.org",
+                                "type": "m.room.member",
+                                "unsigned": {
+                                  "age": 1234,
+                                  "membership": "join"
+                                }
                               },
                               {
-                                "$ref": "../../event-schemas/examples/m.room.message$m.text.yaml"
+                                "content": {
+                                  "body": "This is an example text message",
+                                  "format": "org.matrix.custom.html",
+                                  "formatted_body": "<b>This is an example text message</b>",
+                                  "msgtype": "m.text"
+                                },
+                                "event_id": "$143273582443PhrSn:example.org",
+                                "origin_server_ts": 1432735824653,
+                                "sender": "@example:example.org",
+                                "type": "m.room.message",
+                                "unsigned": {
+                                  "age": 1234,
+                                  "membership": "join"
+                                }
                               }
                             ],
                             "limited": true,

data/api/client-server/third_party_membership.yaml

@@ -57,9 +57,6 @@ paths:
         - A signature of the token, signed with the identity server's private key
 
         - The matrix user ID who invited them to the room
-
-        If a token is requested from the identity server, the homeserver will
-        append a `m.room.third_party_invite` event to the room.
       operationId: inviteBy3PID
       security:
         - accessTokenQuery: []
@@ -72,6 +69,8 @@ paths:
           example: "!d41d8cd:matrix.org"
           schema:
             type: string
+            format: mx-room-id
+            pattern: "^!"
       requestBody:
         content:
           application/json:
@@ -90,7 +89,9 @@ paths:
                   value: {}
         "403":
           description: |-
-            You do not have permission to invite the user to the room. A meaningful `errcode` and description error text will be returned. Example reasons for rejections are:
+            You do not have permission to invite the user to the room. A
+            meaningful `errcode` and description error text will be returned.
+            Example reasons for rejections are:
 
             - The invitee has been banned from the room.
             - The invitee is already a member of the room.

data/api/client-server/wellknown.yaml

@@ -26,6 +26,12 @@ paths:
         suitably namespaced for each application and reduces the risk of
         clashes.
 
+        {{% boxes/note %}}
+        This endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
+        GET request to `https://hostname/.well-known/matrix/client`.
+        {{% /boxes/note %}}
+
         Note that this endpoint is not necessarily handled by the homeserver,
         but by another webserver, to be used for discovering the homeserver URL.
       operationId: getWellknown

data/api/identity/v2_pubkey.yaml

@@ -43,7 +43,8 @@ paths:
                 properties:
                   public_key:
                     type: string
-                    description: Unpadded Base64 encoded public key.
+                    description: |-
+                      [Unpadded Base64](/appendices/#unpadded-base64)-encoded public key.
                 required:
                   - public_key
               examples:
@@ -74,7 +75,8 @@ paths:
         - in: query
           name: public_key
           required: true
-          description: The unpadded base64-encoded public key to check.
+          description: |-
+            The [unpadded Base64](/appendices/#unpadded-base64)-encoded public key to check.
           example: VXuGitF39UH5iRfvbIknlvlAVKgD1BsLDMvBf0pmp7c
           schema:
             type: string
@@ -105,7 +107,14 @@ paths:
         - in: query
           name: public_key
           required: true
-          description: The unpadded base64-encoded public key to check.
+          description: |-
+            The [unpadded Base64](/appendices/#unpadded-base64)-encoded public
+            key to check.
+
+            This MUST be the exact same encoded string returned in the response
+            of the [`/store-invite`](/identity-service-api/#post_matrixidentityv2store-invite)
+            endpoint, or found in the corresponding [`m.room.third_party_invite`](/client-server-api/#mroomthird_party_invite)
+            event, so it may use the standard or URL-safe alphabets.
           example: VXuGitF39UH5iRfvbIknlvlAVKgD1BsLDMvBf0pmp7c
           schema:
             type: string

data/api/identity/v2_store_invite.yaml

@@ -42,7 +42,7 @@ paths:
         (if present) from the request here.
 
         Also, the generated ephemeral public key will be listed as valid on
-        requests to `/_matrix/identity/v2/pubkey/ephemeral/isvalid`.
+        requests to [`/_matrix/identity/v2/pubkey/ephemeral/isvalid`](/identity-service-api/#get_matrixidentityv2pubkeyephemeralisvalid).
 
         Currently, invites may only be issued for 3pids of the `email` medium.
 
@@ -70,10 +70,14 @@ paths:
                 room_id:
                   type: string
                   description: The Matrix room ID to which the user is invited
+                  format: mx-room-id
+                  pattern: "^!"
                   example: "!something:example.org"
                 sender:
                   type: string
                   description: The Matrix user ID of the inviting user
+                  format: mx-user-id
+                  pattern: "^@"
                   example: "@bob:example.com"
                 room_alias:
                   type: string
@@ -81,12 +85,16 @@ paths:
                     The Matrix room alias for the room to which the user is
                     invited. This should be retrieved from the `m.room.canonical_alias`
                     state event.
+                  format: mx-room-alias
+                  pattern: "^#"
                   example: "#somewhere:example.org"
                 room_avatar_url:
                   type: string
                   description: |-
                     The Content URI for the room to which the user is invited. This should
                     be retrieved from the `m.room.avatar` state event.
+                  format: mx-mxc-uri
+                  pattern: "^mxc:\\/\\/"
                   example: mxc://example.org/s0meM3dia
                 room_join_rules:
                   type: string
@@ -108,6 +116,8 @@ paths:
                   type: string
                   description: The Content URI for the avatar of the user ID initiating the
                     invite.
+                  format: mx-mxc-uri
+                  pattern: "^mxc:\\/\\/"
                   example: mxc://example.org/an0th3rM3dia
                 room_type:
                   type: string
@@ -146,7 +156,7 @@ paths:
                         public_key:
                           type: string
                           description: |
-                            The public key, encoded using [unpadded Base64](/appendices/#unpadded-base64).
+                            The public key, encoded using standard or URL-safe [unpadded Base64](/appendices/#unpadded-base64).
                         key_validity_url:
                           type: string
                           description: |

data/api/server-server/invites-v1.yaml

@@ -20,7 +20,7 @@ paths:
     put:
       summary: Invites a remote user to a room
       description: |-
-        Invites a remote user to a room. Once the event has been  signed by both the inviting
+        Invites a remote user to a room. Once the event has been signed by both the inviting
         homeserver and the invited homeserver, it can be sent to all of the servers in the
         room by the inviting homeserver.
 
@@ -32,6 +32,10 @@ paths:
         [room version specification](/rooms) for precise event formats. **The request and response
         bodies here describe the common event fields in more detail and may be missing other
         required fields for a PDU.**
+
+        Also note that if the remote homeserver is already in the room, it will receive the
+        invite event twice; once through this endpoint, and again through a [federation
+        transaction](/server-server-api/#transactions).
       operationId: sendInviteV1
       security:
         - signedRequest: []

data/api/server-server/invites-v2.yaml

@@ -24,7 +24,7 @@ paths:
         This API is nearly identical to the v1 API with the exception of the request
         body being different, and the response format fixed.
 
-        Invites a remote user to a room. Once the event has been  signed by both the inviting
+        Invites a remote user to a room. Once the event has been signed by both the inviting
         homeserver and the invited homeserver, it can be sent to all of the servers in the
         room by the inviting homeserver.
 
@@ -36,6 +36,10 @@ paths:
         [room version specification](/rooms) for precise event formats. **The request and response
         bodies here describe the common event fields in more detail and may be missing other
         required fields for a PDU.**
+
+        Also note that if the remote homeserver is already in the room, it will receive the
+        invite event twice; once through this endpoint, and again through a [federation
+        transaction](/server-server-api/#transactions).
       operationId: sendInviteV2
       security:
         - signedRequest: []

data/api/server-server/third_party_invite.yaml

@@ -35,6 +35,8 @@ paths:
           example: "!abc123:matrix.org"
           schema:
             type: string
+            format: mx-room-id
+            pattern: "^!"
       requestBody:
         content:
           application/json:
@@ -50,16 +52,22 @@ paths:
                   description: |-
                     The room ID the event is for. Must match the ID given in
                     the path.
+                    format: mx-room-id
+                    pattern: "^!"
                   example: "!abc123:matrix.org"
                 sender:
                   type: string
                   description: |-
                     The user ID of the user who sent the original `m.room.third_party_invite`
                     event.
+                  format: mx-user-id
+                  pattern: "^@"
                   example: "@joe:matrix.org"
                 state_key:
                   type: string
                   description: The user ID of the invited user
+                  format: mx-user-id
+                  pattern: "^@"
                   example: "@someone:example.org"
                 content:
                   type: object
@@ -82,45 +90,7 @@ paths:
                             third-party identifier.
                           example: alice
                         signed:
-                          type: object
+                          $ref: ../../event-schemas/schema/components/signed_third_party_invite.yaml
-                          description: |-
-                            A block of content which has been signed, which servers can use to
-                            verify the event.
-                          title: Invite Signatures
-                          properties:
-                            signatures:
-                              type: object
-                              title: Signatures
-                              additionalProperties:
-                                type: object
-                                additionalProperties:
-                                  type: string
-                              description: |-
-                                The server signatures for this event.
-
-                                The signature is calculated using the process
-                                described at [Signing JSON](/appendices/#signing-json).
-                              example:
-                                magic.forest:
-                                  ed25519:3: fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg
-                            mxid:
-                              type: string
-                              description: The invited matrix user ID
-                              example: "@alice:localhost"
-                            token:
-                              type: string
-                              description: The token used to verify the event
-                              example: abc123
-                          required:
-                            - signatures
-                            - mxid
-                            - token
-                          example:
-                            mxid: "@alice:localhost"
-                            token: abc123
-                            signatures:
-                              magic.forest:
-                                ed25519:3: fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg
                       required:
                         - display_name
                         - signed
@@ -215,6 +185,8 @@ paths:
                 mxid:
                   type: string
                   description: The user that is now bound to the third-party identifier.
+                  format: mx-user-id
+                  pattern: "^@"
                   example: "@alice:matrix.org"
                 invites:
                   type: array
@@ -237,59 +209,23 @@ paths:
                       mxid:
                         type: string
                         description: The now-bound user ID that received the invite.
+                        format: mx-user-id
+                        pattern: "^@"
                         example: "@alice:matrix.org"
                       room_id:
                         type: string
                         description: The room ID the invite is valid for.
+                        format: mx-room-id
+                        pattern: "^!"
                         example: "!somewhere:example.org"
                       sender:
                         type: string
                         description: The user ID that sent the invite.
+                        format: mx-user-id
+                        pattern: "^@"
                         example: "@bob:matrix.org"
-                      # TODO (TravisR): Make this reusable when doing IS spec changes
-                      #   also make sure it isn't lying about anything, like the key version
                       signed:
-                        type: object
+                        $ref: ../../event-schemas/schema/components/signed_third_party_invite.yaml
-                        title: Identity Server Signatures
-                        description: |-
-                          Signature from the identity server using a long-term private
-                          key.
-                        properties:
-                          mxid:
-                            type: string
-                            description: |-
-                              The user ID that has been bound to the third-party
-                              identifier.
-                            example: "@alice:matrix.org"
-                          token:
-                            type: string
-                            # TODO: What is this actually?
-                            description: A token.
-                            example: Hello World
-                          signatures:
-                            type: object
-                            title: Identity Server Signature
-                            description: |-
-                              The signature from the identity server. The `string` key
-                              is the identity server's domain name, such as vector.im
-                            additionalProperties:
-                              type: object
-                              title: Identity Server Domain Signature
-                              description: The signature for the identity server.
-                              properties:
-                                ed25519:0:
-                                  type: string
-                                  description: The signature.
-                                  example: SomeSignatureGoesHere
-                              required:
-                                - ed25519:0
-                            example:
-                              vector.im:
-                                ed25519:0: SomeSignatureGoesHere
-                        required:
-                          - mxid
-                          - token
-                          - signatures
                     required:
                       - medium
                       - address

data/api/server-server/wellknown.yaml

@@ -24,6 +24,12 @@ paths:
         Gets information about the delegated server for server-server communication
         between Matrix homeservers. Servers should follow 30x redirects, carefully
         avoiding redirect loops, and use normal X.509 certificate validation.
+
+        {{% boxes/note %}}
+        This endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
+        GET request to `https://hostname/.well-known/matrix/server`.
+        {{% /boxes/note %}}
       operationId: getWellKnown
       responses:
         "200":

data/event-schemas/examples/m.room.member.yaml

@@ -1,6 +1,7 @@
 {
   "$ref": "core/state_event.json",
   "state_key": "@alice:example.org",
+  "sender": "@alice:example.org",
   "type": "m.room.member",
   "content": {
     "membership": "join",

data/event-schemas/schema/components/signed_third_party_invite.yaml

@@ -0,0 +1,45 @@
+title: SignedThirdPartyInvite
+description: |-
+  A block of content which has been signed by the identity server, which
+  homeservers can use to verify the event. Clients should ignore this.
+type: object
+properties:
+  mxid:
+    description: |-
+      The user ID that has been bound to the third-party identifier.
+    type: string
+    format: mx-user-id
+    pattern: "^@"
+    example: "@alice:example.org"
+  signatures:
+    title: IdentityServerSignatures
+    description: |-
+      The identity server signatures for this block. This is a map of identity
+      server name to signing key identifier to base64-encoded signature.
+
+      The signatures are calculated using the process described at
+      [Signing JSON](/appendices/#signing-json).
+    type: object
+    additionalProperties:
+      type: object
+      additionalProperties:
+        type: string
+    example: {
+      "magic.forest": {
+        "ed25519:3": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg"
+      }
+    }
+  token:
+    description: |-
+      The token generated by the identity server at the
+      [`/store_invite`](/identity-service-api/#post_matrixidentityv2store-invite)
+      endpoint.
+
+      It matches the `state_key` of the corresponding [`m.room.third_party_invite`](/client-server-api/#mroomthird_party_invite)
+      event.
+    type: string
+    example: "abc123"
+required:
+  - mxid
+  - signatures
+  - token

data/event-schemas/schema/m.room.member.yaml

@@ -2,17 +2,27 @@
 allOf:
   - $ref: core-event-schema/state_event.yaml
 description: |-
-  Adjusts the membership state for a user in a room. It is preferable to use the membership APIs (`/rooms/<room id>/invite` etc) when performing membership actions rather than adjusting the state directly as there are a restricted set of valid transformations. For example, user A cannot force user B to join a room, and trying to force this state change directly will fail.
+  Adjusts the membership state for a user in a room. It is preferable to use the membership APIs
+  (`/rooms/<room id>/invite` etc) when performing membership actions rather than adjusting the
+  state directly as there are a restricted set of valid transformations. For example, user A cannot
+  force user B to join a room, and trying to force this state change directly will fail.
 
   The following membership states are specified:
 
-  - `invite` - The user has been invited to join a room, but has not yet joined it. They may not participate in the room until they join.
+  - `invite` - The user has been invited to join a room, but has not yet joined it. They may not
-  - `join` - The user has joined the room (possibly after accepting an invite), and may participate in it.
+    participate in the room until they join.
-  - `leave` - The user was once joined to the room, but has since left (possibly by choice, or possibly by being kicked).
+  - `join` - The user has joined the room (possibly after accepting an invite), and may participate
-  - `ban` - The user has been banned from the room, and is no longer allowed to join it until they are un-banned from the room (by having their membership state set to a value other than `ban`).
+    in it.
-  - `knock` - The user has knocked on the room, requesting permission to participate. They may not participate in the room until they join.
+  - `leave` - The user was once joined to the room, but has since left (possibly by choice, or
+    possibly by being kicked).
+  - `ban` - The user has been banned from the room, and is no longer allowed to join it until they
+    are un-banned from the room (by having their membership state set to a value other than `ban`).
+  - `knock` - The user has knocked on the room, requesting permission to participate. They may not
+    participate in the room until they join.
 
-  The `third_party_invite` property will be set if this invite is an `invite` event and is the successor of an `m.room.third_party_invite` event, and absent otherwise.
+  The `third_party_invite` property will be set if this invite is an `invite` event and is the
+  successor of an [`m.room.third_party_invite`](/client-server-api/#mroomthird_party_invite) event,
+  and absent otherwise.
 
   This event may also include an `invite_room_state` key inside the event's `unsigned` data.
   If present, this contains an array of [stripped state events](/client-server-api/#stripped-state)
@@ -57,63 +67,54 @@ properties:
           - ban
         type: string
       is_direct:
-        description: Flag indicating if the room containing this event was created with the intention of being a direct chat. See [Direct Messaging](/client-server-api/#direct-messaging).
+        description: |-
+          Flag indicating if the room containing this event was created with the intention of being
+          a direct chat. See [Direct Messaging](/client-server-api/#direct-messaging).
         type: boolean
       join_authorised_via_users_server:
         x-addedInMatrixVersion: "1.2"
         type: string
         description: |-
-          Usually found on `join` events, this field is used to denote which homeserver (through representation of a user with sufficient power level)
+          Usually found on `join` events, this field is used to denote which homeserver (through
-          authorised the user's join. More information about this field can be found in the [Restricted Rooms Specification](/client-server-api/#restricted-rooms).
+          representation of a user with sufficient power level) authorised the user's join. More
+          information about this field can be found in the [Restricted Rooms Specification](/client-server-api/#restricted-rooms).
 
-          Client and server implementations should be aware of the [signing implications](/rooms/v8/#authorization-rules) of including this
+          Client and server implementations should be aware of the [signing implications](/rooms/v8/#authorization-rules)
-          field in further events: in particular, the event must be signed by the server which
+          of including this field in further events: in particular, the event must be signed by the
-          owns the user ID in the field. When copying the membership event's `content`
+          server which owns the user ID in the field. When copying the membership event's `content`
-          (for profile updates and similar) it is therefore encouraged to exclude this
+          (for profile updates and similar) it is therefore encouraged to exclude this field in the
-          field in the copy, as otherwise the event might fail event authorization.
+          copy, as otherwise the event might fail event authorization.
       reason:
         x-addedInMatrixVersion: "1.1"
         type: string
         description: |-
-          Optional user-supplied text for why their membership has changed. For kicks and bans, this is typically the reason for the kick or ban.
+          Optional user-supplied text for why their membership has changed. For kicks and bans,
-          For other membership changes, this is a way for the user to communicate their intent without having to send a message to the room, such
+          this is typically the reason for the kick or ban. For other membership changes, this is a
-          as in a case where Bob rejects an invite from Alice about an upcoming concert, but can't make it that day.
+          way for the user to communicate their intent without having to send a message to the
+          room, such as in a case where Bob rejects an invite from Alice about an upcoming concert,
+          but can't make it that day.
 
-          Clients are not recommended to show this reason to users when receiving an invite due to the potential for spam and abuse. Hiding the
+          Clients are not recommended to show this reason to users when receiving an invite due to
-          reason behind a button or other component is recommended.
+          the potential for spam and abuse. Hiding the reason behind a button or other component is
+          recommended.
       third_party_invite:
+        title: ThirdPartyInvite
+        description: |-
+          A third-party invite, if this `m.room.member` is the successor to an
+          [`m.room.third_party_invite`](/client-server-api/#mroomthird_party_invite)
+          event.
+        type: object
         properties:
           display_name:
-            description: A name which can be displayed to represent the user instead of their third-party identifier
+            description: |-
+              A name which can be displayed to represent the user instead of their
+              third-party identifier
             type: string
           signed:
-            description: 'A block of content which has been signed, which servers can use to verify the event. Clients should ignore this.'
+            $ref: components/signed_third_party_invite.yaml
-            properties:
-              mxid:
-                description: The invited matrix user ID. Must be equal to the user_id property of the event.
-                type: string
-              signatures:
-                description: 'A single signature from the verifying server, in the format specified by the Signing Events section of the server-server API.'
-                title: Signatures
-                type: object
-                additionalProperties:
-                  type: object
-                  additionalProperties:
-                    type: string
-              token:
-                description: The token property of the containing third_party_invite object.
-                type: string
-            required:
-              - mxid
-              - signatures
-              - token
-            title: signed
-            type: object
         required:
           - display_name
           - signed
-        title: Invite
-        type: object
     required:
       - membership
     title: EventContent

data/event-schemas/schema/m.room.third_party_invite.yaml

@@ -1,28 +1,56 @@
 ---
 allOf:
   - $ref: core-event-schema/state_event.yaml
-description: "Acts as an `m.room.member` invite event, where there isn't a target user_id to invite. This event contains a token and a public key whose private key must be used to sign the token. Any user who can present that signature may use this invitation to join the target room."
+description: |-
+  Acts as an `m.room.member` invite event, where there isn't a target user_id to
+  invite. This event contains a token and a public key whose private key must be
+  used to sign the token. Any user who can present that signature may use this
+  invitation to join the target room.
 properties:
   content:
     properties:
       display_name:
-        description: "A user-readable string which represents the user who has been invited. This should not contain the user's third-party ID, as otherwise when the invite is accepted it would leak the association between the matrix ID and the third-party ID."
+        description: |-
+          A user-readable string which represents the user who has been invited.
+          This should not contain the user's third-party ID, as otherwise when
+          the invite is accepted it would leak the association between the
+          matrix ID and the third-party ID.
         type: string
       key_validity_url:
-        description: "A URL which can be fetched, with querystring public_key=public_key, to validate whether the key has been revoked. The URL must return a JSON object containing a boolean property named 'valid'."
+        description: |-
+          A URL which can be fetched, with querystring public_key=public_key, to
+          validate whether the key has been revoked. The URL must return a JSON
+          object containing a boolean property named 'valid'.
         type: string
+        format: uri
       public_key:
-        description: A base64-encoded ed25519 key with which token must be signed (though a signature from any entry in public_keys is also sufficient). This exists for backwards compatibility.
+        description: |-
+          An Ed25519 key with which the token must be signed (though a signature
+          from any entry in `public_keys` is also sufficient).
+
+          The key is encoded using [Unpadded Base64](/appendices/#unpadded-base64),
+          using the standard or URL-safe alphabets.
+
+          This exists for backwards compatibility.
         type: string
       public_keys:
         description: Keys with which the token may be signed.
         items:
           properties:
             key_validity_url:
-              description: "An optional URL which can be fetched, with querystring public_key=public_key, to validate whether the key has been revoked. The URL must return a JSON object containing a boolean property named 'valid'. If this URL is absent, the key must be considered valid indefinitely."
+              description: |-
+                An optional URL which can be fetched, with querystring
+                `public_key=<public_key>`, to validate whether the key has been
+                revoked. The URL must return a JSON object containing a boolean
+                property named `valid`. If this URL is absent, the key must be
+                considered valid indefinitely.
               type: string
             public_key:
-              description: A base-64 encoded ed25519 key with which token may be signed.
+              description: |-
+                An Ed25519 key with which the token may be signed.
+
+                The key is encoded using [Unpadded Base64](/appendices/#unpadded-base64),
+                using the standard or URL-safe alphabets.
               type: string
           required:
             - public_key
@@ -35,11 +63,15 @@ properties:
       - public_key
     type: object
   state_key:
-    description: 'The token, of which a signature must be produced in order to join the room.'
+    description: |-
+      The token, of which a signature must be produced in order to join the
+      room.
     type: string
   type:
     enum:
       - m.room.third_party_invite
     type: string
-title: 'An invitation to a room issued to a third-party identifier, rather than a matrix user ID.'
+title: |-
+  An invitation to a room issued to a third-party identifier, rather than a
+  matrix user ID.
 type: object

data/event-schemas/schema/m.room.topic.yaml

@@ -1,7 +1,14 @@
 ---
 allOf:
   - $ref: core-event-schema/state_event.yaml
-description: 'A topic is a short message detailing what is currently being discussed in the room.  It can also be used as a way to display extra information about the room, which may not be suitable for the room name. The room topic can also be set when creating a room using `/createRoom` with the `topic` key.'
+description: |- 
+    A topic is a short message detailing what is currently being discussed in the room.
+    It can also be used as a way to display extra information about the room, which may not
+    be suitable for the room name.
+    The room topic can also be set when creating a room using `/createRoom` with the `topic` key.'
+    
+    If the `topic` property is absent, null, or empty then the topic is unset. In other words,
+    an empty `topic` property effectively resets the room to having no topic.
 properties:
   content:
     properties:

data/string-formats.yaml

@@ -51,6 +51,11 @@ mx-room-id:
   url: appendices#room-ids
   # regex: "^!"
 
+mx-room-alias:
+  title: Room Alias
+  url: appendices#room-aliases
+  # regex: "^#"
+
 mx-server-name:
   title: Server Name
   url: appendices#server-name

scripts/dump-openapi.py

@@ -32,6 +32,35 @@ import yaml
 scripts_dir = os.path.dirname(os.path.abspath(__file__))
 api_dir = os.path.join(os.path.dirname(scripts_dir), "data", "api")
 
+# Finds a Hugo shortcode in a string.
+#
+# A shortcode is defined as (newlines and whitespaces for presentation purpose):
+#
+# {{%
+#     <zero or more whitespaces>
+#     <name of shortcode>
+#     (optional <one or more whitespaces><list of parameters>)
+#     <zero or more whitespaces>
+# %}}
+#
+# With:
+#
+# * <name of shortcode>: any word character and `-` and `/`. `re.ASCII` is used to only match
+#   ASCII characters in the name.
+# * <list of parameters>: any character except `}`, must not start or end with a
+#   whitespace.
+shortcode_regex = re.compile(r"""\{\{\%                                   # {{%
+                                 \s*                                      # zero or more whitespaces
+                                 (?P<name>[\w/-]+)                        # name of shortcode
+                                 (?:\s+(?P<params>[^\s\}][^\}]+[^\s\}]))? # optional list of parameters
+                                 \s*                                      # zero or more whitespaces
+                                 \%\}\}                                   # %}}""", re.ASCII | re.VERBOSE)
+
+# Parses the parameters of a Hugo shortcode.
+#
+# For simplicity, this currently only supports the `key="value"` format.
+shortcode_params_regex = re.compile(r"(?P<key>\w+)=\"(?P<value>[^\"]+)\"", re.ASCII)
+
 def prefix_absolute_path_references(text, base_url):
     """Adds base_url to absolute-path references.
 
@@ -44,17 +73,90 @@ def prefix_absolute_path_references(text, base_url):
     """
     return text.replace("](/", "]({}/".format(base_url))
 
-def edit_links(node, base_url):
+def replace_match(match, replacement):
-    """Finds description nodes and makes any links in them absolute."""
+    """Replaces the regex match by the replacement in the text."""
+    return match.string[:match.start()] + replacement + match.string[match.end():]
+
+def replace_shortcode(shortcode):
+    """Replaces the shortcode by a Markdown fallback in the text.
+
+    The supported shortcodes are:
+
+    * boxes/note, boxes/rationale, boxes/warning
+    * added-in, changed-in
+    
+    All closing tags (`{{ /shortcode }}`) are replaced with the empty string.
+    """
+
+    if shortcode['name'].startswith("/"):
+        # This is the end of the shortcode, just remove it.
+        return replace_match(shortcode, "")
+
+    # Parse the parameters of the shortcode
+    params = {}
+    if shortcode['params']:
+        for param in shortcode_params_regex.finditer(shortcode['params']):
+            if param['key']:
+                params[param['key']] = param['value']
+
+    match shortcode['name']:
+        case "boxes/note":
+            return replace_match(shortcode, "**NOTE:** ")
+        case "boxes/rationale":
+            return replace_match(shortcode, "**RATIONALE:** ")
+        case "boxes/warning":
+            return replace_match(shortcode, "**WARNING:** ")
+        case "added-in":
+            version = params['v']
+            if not version:
+                raise ValueError("Missing parameter `v` for `added-in` shortcode")
+
+            return replace_match(shortcode, f"**[Added in `v{version}`]** ")
+        case "changed-in":
+            version = params['v']
+            if not version:
+                raise ValueError("Missing parameter `v` for `changed-in` shortcode")
+
+            return replace_match(shortcode, f"**[Changed in `v{version}`]** ")
+        case _:
+            raise ValueError("Unknown shortcode", shortcode['name'])
+
+
+def find_and_replace_shortcodes(text):
+    """Finds Hugo shortcodes and replaces them by a Markdown fallback.
+
+    The supported shortcodes are:
+
+    * boxes/note, boxes/rationale, boxes/warning
+    * added-in, changed-in
+    """
+    # We use a `while` loop with `search` instead of a `for` loop with
+    # `finditer`, because as soon as we start replacing text, the
+    # indices of the match are invalid.
+    while shortcode := shortcode_regex.search(text):
+        text = replace_shortcode(shortcode)
+
+    return text
+
+def edit_descriptions(node, base_url):
+    """Finds description nodes and apply fixes to them.
+
+    The fixes that are applied are:
+
+    * Make links absolute
+    * Replace Hugo shortcodes
+    """
     if isinstance(node, dict):
         for key in node:
             if isinstance(node[key], str):
                 node[key] = prefix_absolute_path_references(node[key], base_url)
+                node[key] = find_and_replace_shortcodes(node[key])
             else:
-                edit_links(node[key], base_url)
+                edit_descriptions(node[key], base_url)
     elif isinstance(node, list):
         for item in node:
-            edit_links(item, base_url)
+            edit_descriptions(item, base_url)
+
 
 parser = argparse.ArgumentParser(
     "dump-openapi.py - assemble the OpenAPI specs into a single JSON file"
@@ -164,7 +266,7 @@ for filename in os.listdir(selected_api_dir):
 if untagged != 0:
     print("{} untagged operations, you may want to look into fixing that.".format(untagged))
 
-edit_links(output, base_url)
+edit_descriptions(output, base_url)
 
 print("Generating %s" % output_file)