Merge a3dc964679 into 67743d5715

With the move of the config file, the command in CI did not work as
expected anymore.
I am unsure why Hugo actually ignored the missing config file in the
command…

To avoid this problem in the future and simplify the job, we use the
default config and add an environment variable for the status which will
always take precedence over the config.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>

.github/workflows/main.yml

@@ -2,6 +2,7 @@ name: "Spec"
 
 env:
   HUGO_VERSION: 0.139.0
+  PYTHON_VERSION: 3.13
 
 on:
   push:
@@ -40,7 +41,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -59,7 +60,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -78,7 +79,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -120,7 +121,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -172,7 +173,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
       - name: "➕ Install towncrier"
         run: "pip install 'towncrier'"
       - name: "Generate changelog"
@@ -283,10 +284,11 @@ jobs:
           npm i
           npm run get-proposals
       - name: "⚙️ hugo"
+        env:
+          HUGO_PARAMS_VERSION_STATUS: "historical"
         # Create a baseURL like `/v1.2` out of the `v1.2` tag
         run: |
-          echo -e '[params.version]\nstatus="historical"' > historical.toml
+          hugo --baseURL "/${GITHUB_REF/refs\/tags\//}" -d "spec"
-          hugo --config config.toml,historical.toml --baseURL "/${GITHUB_REF/refs\/tags\//}" -d "spec"
 
       - name: "📥 Spec definition download"
         uses: actions/download-artifact@v4

changelogs/application_service/newsfragments/2131.clarification

@@ -0,0 +1 @@
+Format Application Service schemas.

changelogs/client_server/newsfragments/2095.feature

@@ -0,0 +1 @@
+Add `m.topic` content block to enable rich text in `m.room.topic` events as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765).

changelogs/client_server/newsfragments/2101.clarification

@@ -0,0 +1 @@
+"Public" rooms in profile look-ups are defined through their join rule and history visibility.

changelogs/client_server/newsfragments/2102.clarification

@@ -0,0 +1 @@
+"Public" rooms in user directory queries are defined through their join rule and history visibility.

changelogs/client_server/newsfragments/2106.clarification

@@ -0,0 +1 @@
+"Public" rooms with respect to call invites are defined through their join rule.

changelogs/client_server/newsfragments/2107.clarification

@@ -0,0 +1 @@
+"Public" rooms have no specific meaning with respect to moderation policy lists.

changelogs/client_server/newsfragments/2108.clarification

@@ -0,0 +1 @@
+"Public" rooms with respect to presence are defined through their join rule.

changelogs/client_server/newsfragments/2140.clarification

@@ -0,0 +1 @@
+Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks.

changelogs/internal/newsfragments/2088.clarification

@@ -0,0 +1 @@
+Replace Hugo shortcodes in OpenAPI output.

changelogs/internal/newsfragments/2123.clarification

@@ -0,0 +1 @@
+Fix the historical info box when generating the historical spec in CI.

changelogs/server_server/newsfragments/2067.clarification

@@ -0,0 +1 @@
+Add a note to the invite endpoints that invites to local users may be received twice over federation if the homeserver is already in the room.

changelogs/server_server/newsfragments/2095.feature

@@ -0,0 +1 @@
+Add `m.topic` content block to enable rich text in `m.room.topic` events as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765).

changelogs/server_server/newsfragments/2100.clarification

@@ -0,0 +1 @@
+Clarify that auth event of `content.join_authorised_via_users_server` is only necessary for `m.room.member` with a `membership` of `join`.

changelogs/server_server/newsfragments/2140.clarification

@@ -0,0 +1 @@
+Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks.

content/client-server-api/_index.md

@@ -371,15 +371,23 @@ valid data was obtained, but no server is available to serve the client.
 No further guess should be attempted and the user should make a
 conscientious decision what to do next.
 
-### Well-known URI
+### Well-known URIs
+
+Matrix facilitates automatic discovery for the Client-Server API base URL and more via the
+[RFC 8615](https://datatracker.ietf.org/doc/html/rfc8615) "Well-Known URI" method.
+This method uses JSON files at a predetermined location on the root path `/.well-known/` to
+specify parameter values.
 
 {{% boxes/note %}}
+Diverging from the rest of the endpoints in the Client-Server spec, these files can not be provided
+on the base URL that the Client-Server API is reachable on, as it is yet to be discovered.
+Instead, they can be reached via HTTPS on the [server name](/appendices/#server-name)'s hostname as domain.
+
 Servers hosting the `.well-known` JSON file SHOULD offer CORS headers,
 as per the [CORS](#web-browser-clients) section in this specification.
 {{% /boxes/note %}}
 
-The `.well-known` method uses a JSON file at a predetermined location to
+The flow for auto-discovery is as follows:
-specify parameter values. The flow for this method is as follows:
 
 1.  Extract the [server name](/appendices/#server-name) from the user's Matrix ID by splitting the
     Matrix ID at the first colon.
@@ -415,10 +423,17 @@ specify parameter values. The flow for this method is as follows:
 
 {{% http-api spec="client-server" api="wellknown" %}}
 
-{{% http-api spec="client-server" api="versions" %}}
-
 {{% http-api spec="client-server" api="support" %}}
 
+### API Versions
+
+Upon connecting, the Matrix client and server need to negotiate which version of the specification
+they commonly support, as the API evolves over time. The server advertises its supported versions
+and optionally unstable features to the client, which can then go on to make requests to the
+endpoints it supports.
+
+{{% http-api spec="client-server" api="versions" %}}
+
 ## Client Authentication
 
 Most API endpoints require the user to identify themselves by presenting
@@ -2847,10 +2862,15 @@ re-invited.
 
 #### Server behaviour
 
-Homeservers MUST at a minimum allow profile look-up for:
+Homeservers MUST at a minimum allow profile look-up for users who are
+visible to the requester based on their membership in rooms known to the
+homeserver. This means:
 
 -   users that share a room with the requesting user
--   users that reside in public rooms known to the homeserver
+-   users who are joined to rooms known to the homeserver that have a
+    `public` [join rule](#mroomjoin_rules)
+-   users who are joined to rooms known to the homeserver that have a
+    `world_readable` [history visibility](#room-history-visibility)
 
 In all other cases, homeservers MAY deny profile look-up by responding with
 403 and an error code of `M_FORBIDDEN`.

content/client-server-api/modules/moderation_policies.md

@@ -18,8 +18,9 @@ the entity making the decisions on filtering is best positioned to
 interpret the rules how it sees fit.
 
 Moderation policy lists are stored as room state events. There are no
-restrictions on how the rooms can be configured (they could be public,
+restrictions on how the rooms can be configured in terms of
-private, encrypted, etc).
+[join rules](#mroomjoin_rules), [history visibility](#room-history-visibility),
+encryption, etc.
 
 There are currently 3 kinds of entities which can be affected by rules:
 `user`, `server`, and `room`. All 3 are described with

content/client-server-api/modules/presence.md

@@ -68,5 +68,7 @@ will cause the server to automatically set their presence to `online`.
 
 #### Security considerations
 
-Presence information is shared with all users who share a room with the
+Presence information is published to all users who share a room with the
-target user. In large public rooms this could be undesirable.
+target user. If the target user is a member of a room with a `public`
+[join rule](#mroomjoin_rules), any other user in the federation is
+able to gain access to the target user's presence. This could be undesirable.

content/client-server-api/modules/search.md

@@ -26,9 +26,10 @@ on certain keys of certain event types.
 
 The supported keys to search over are:
 
--   `content.body` in `m.room.message`
+-   `content.body` in [`m.room.message`](/client-server-api/#mroommessage)
--   `content.name` in `m.room.name`
+-   `content.name` in [`m.room.name`](/client-server-api/#mroomname)
--   `content.topic` in `m.room.topic`
+-   In [`m.room.topic`](/client-server-api/#mroomtopic), `content.topic`
+    as well as the `body` of the `text/plain` representation in `content['m.topic']`.
 
 The search will *not* include rooms that are end to end encrypted.
 

content/client-server-api/modules/voip_events.md

@@ -202,11 +202,13 @@ specific user, and should be set to the Matrix user ID of that user. Invites
 without an `invitee` field are defined to be intended for any member of the
 room other than the sender of the event.
 
-Clients should consider an incoming call if they see a non-expired invite event where the `invitee` field is either
+Clients should consider an incoming call if they see a non-expired invite event
-absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their
+where the `invitee` field is either absent or equal to their user's Matrix ID.
-user's trust relationship with the callers and/or where the call was placed. As a starting point, it is
+They should, however, evaluate whether or not to ring based on their user's trust
-suggested that clients ignore call invites from users in public rooms. It is strongly recommended that
+relationship with the callers and/or where the call was placed. As a starting
-when clients do not ring for an incoming call invite, they still display the call invite in the room and
+point, it is RECOMMENDED that clients ignore call invites in rooms with a
+[join rule](#mroomjoin_rules) of `public`. When clients suppress ringing for an
+incoming call invite, they SHOULD still display the call invite in the room and
 annotate that it was ignored.
 
 ##### Glare

content/server-server-api.md

@@ -119,7 +119,8 @@ to send. The process overall is as follows:
     server must present a valid certificate for the hostname.
 
 3.  If the hostname is not an IP literal, a regular HTTPS request is
-    made to `https://<hostname>/.well-known/matrix/server`, expecting
+    made to `https://<hostname>/.well-known/matrix/server` (according to 
+    [RFC 8615](https://datatracker.ietf.org/doc/html/rfc8615)), expecting
     the schema defined later in this section. 30x redirects should be
     followed, however redirection loops should be avoided. Responses
     (successful or otherwise) to the `/.well-known` endpoint should be
@@ -543,8 +544,8 @@ the following subset of the room state:
       `third_party_invite` property, the current
       `m.room.third_party_invite` event with `state_key` matching
       `content.third_party_invite.signed.token`, if any.
-    - If `content.join_authorised_via_users_server` is present,
+    - If `membership` is `join`, `content.join_authorised_via_users_server`
-      and the [room version supports restricted rooms](/rooms/#feature-matrix),
+      is present, and the [room version supports restricted rooms](/rooms/#feature-matrix),
       then the `m.room.member` event with `state_key` matching
       `content.join_authorised_via_users_server`.
 

data/api/application-service/definitions/location.yaml

@@ -29,4 +29,4 @@ properties:
     }
 required: ['alias', 'protocol', 'fields']
 title: Location
-type: object
+type: object

data/api/application-service/definitions/namespace_list.yaml

@@ -19,10 +19,13 @@ items:
   properties:
     regex:
       type: string
-      description: A POSIX regular expression defining which values this namespace includes.
+      description: >-
+        A POSIX regular expression defining which values this namespace includes.
     exclusive:
       type: boolean
-      description: A true or false value stating whether this application service has exclusive access to events within this namespace.
+      description: >-
+        A true or false value stating whether this application service has
+        exclusive access to events within this namespace.
   required:
     - regex
     - exclusive

data/api/application-service/definitions/protocol.yaml

@@ -16,7 +16,7 @@ allOf:
 - type: object
   properties:
     instances:
-      description: |-
+      description: >-
         A list of objects representing independent instances of configuration.
         For example, multiple networks on IRC if multiple are provided by the
         same application service.

data/api/application-service/definitions/protocol_base.yaml

@@ -15,7 +15,7 @@ title: Protocol
 type: object
 properties:
   user_fields:
-    description: |-
+    description: >-
       Fields which may be used to identify a third-party user. These should be
       ordered to suggest the way that entities may be grouped, where higher
       groupings are ordered first. For example, the name of a network should be
@@ -26,9 +26,9 @@ properties:
       description: Field used to identify a third-party user.
     example: ["network", "nickname"]
   location_fields:
-    description: |-
+    description: >-
-      Fields which may be used to identify a third-party location. These should be
+      Fields which may be used to identify a third-party location. These should
-      ordered to suggest the way that entities may be grouped, where higher
+      be ordered to suggest the way that entities may be grouped, where higher
       groupings are ordered first. For example, the name of a network should be
       searched before the name of a channel.
     type: array
@@ -41,10 +41,11 @@ properties:
     type: string
     example: "mxc://example.org/aBcDeFgH"
   field_types:
-    description: |-
+    description: >-
       The type definitions for the fields defined in `user_fields` and
-      `location_fields`. Each entry in those arrays MUST have an entry here.
+      `location_fields`. Each entry in those arrays MUST have an entry here. The
-      The `string` key for this object is the field name itself.
+      `string` key for this object is the field name itself.
+
 
       May be an empty object if no fields are defined.
     type: object
@@ -54,10 +55,11 @@ properties:
       type: object
       properties:
         regexp:
-          description: |-
+          description: >-
-            A regular expression for validation of a field's value. This may be relatively
+            A regular expression for validation of a field's value. This may be
-            coarse to verify the value as the application service providing this protocol
+            relatively coarse to verify the value as the application service
-            may apply additional validation or filtering.
+            providing this protocol may apply additional validation or
+            filtering.
           type: string
         placeholder:
           description: A placeholder serving as a valid example of the field value.

data/api/application-service/definitions/protocol_instance.yaml

@@ -20,9 +20,9 @@ properties:
     example: "Freenode"
   icon:
     type: string
-    description: |-
+    description: >-
-      An optional content URI representing the protocol. Overrides the one provided
+      An optional content URI representing the protocol. Overrides the one
-      at the higher level Protocol object.
+      provided at the higher level Protocol object.
     example: "mxc://example.org/JkLmNoPq"
   fields:
     type: object

data/api/application-service/definitions/registration.yaml

@@ -17,27 +17,37 @@ title: Registration
 properties:
   id:
     type: string
-    description: A unique, user-defined ID of the application service which will never change.
+    description: >-
+      A unique, user-defined ID of the application service which will never change.
   url:
     type: string
-    description: The URL for the application service. May include a path after the domain name. Optionally set to null if no traffic is required.
+    description: >-
+      The URL for the application service. May include a path after the domain
+      name. Optionally set to null if no traffic is required.
   as_token:
     type: string
-    description: A secret token that the application service will use to authenticate requests to the homeserver.
+    description: >-
+      A secret token that the application service will use to authenticate
+      requests to the homeserver.
   hs_token:
     type: string
-    description: A secret token that the homeserver will use authenticate requests to the application service.
+    description: >-
+      A secret token that the homeserver will use authenticate requests to the
+      application service.
   sender_localpart:
     type: string
-    description: |-
+    description: >-
-      The localpart of the user associated with the application service. Events will be sent to the AS if this user is the target of the event, or
+      The localpart of the user associated with the application service. Events
-      is a joined member of the room where the event occurred.
+      will be sent to the AS if this user is the target of the event, or is a
+      joined member of the room where the event occurred.
   receive_ephemeral:
     type: boolean
     x-addedInMatrixVersion: "1.13"
-    description: |-
+    description: >-
-      Whether the application service wants to [receive ephemeral data](/application-service-api/#pushing-ephemeral-data).
+      Whether the application service wants to [receive ephemeral
-      
+      data](/application-service-api/#pushing-ephemeral-data).
+
+
       Defaults to `false` if not present.
   namespaces:
     type: object
@@ -47,7 +57,7 @@ properties:
       users:
         allOf:
           - $ref: namespace_list.yaml
-          - description: |-
+          - description: >-
               A list of namespaces defining the user IDs that the application
               service is interested in, in addition to its `sender_localpart`.
               Events will be sent to the AS if a local user matching one of the
@@ -56,23 +66,26 @@ properties:
       rooms:
         allOf:
           - $ref: namespace_list.yaml
-          - description: |-
+          - description: >-
               A list of namespaces defining the room IDs that the application
               service is interested in. All events sent in a room with an ID
               which matches one of the namespaces will be sent to the AS.
       aliases:
         allOf:
           - $ref: namespace_list.yaml
-          - description: |-
+          - description: >-
-              A list of namespaces defining the room aliases that the application
+              A list of namespaces defining the room aliases that the
-              service is interested in. All events sent in a room with an alias
+              application service is interested in. All events sent in a room
-              which matches one of the namespaces will be sent to the AS.
+              with an alias which matches one of the namespaces will be sent to
+              the AS.
   rate_limited:
     type: boolean
-    description: Whether requests from masqueraded users are rate-limited. The sender is excluded.
+    description: >-
+      Whether requests from masqueraded users are rate-limited. The sender is excluded.
   protocols:
     type: array
-    description: The external protocols which the application service provides (e.g. IRC).
+    description: >-
+      The external protocols which the application service provides (e.g. IRC).
     items:
       type: string
 required:

data/api/application-service/definitions/security.yaml

@@ -15,4 +15,5 @@
 homeserverAccessToken:
   type: http
   scheme: bearer
-  description: The `Bearer` `hs_token` provided by the application service's registration.
+  description: >-
+    The `Bearer` `hs_token` provided by the application service's registration.

data/api/application-service/definitions/user.yaml

@@ -30,4 +30,4 @@ properties:
     }
 required: ['userid', 'protocol', 'fields']
 title: User
-type: object
+type: object

data/api/application-service/ping.yaml

@@ -20,13 +20,18 @@ paths:
     post:
       x-addedInMatrixVersion: "1.7"
       summary: Ping the application service
-      description: |-
+      description: >-
         This API is called by the homeserver to ensure that the connection works
         and the `hs_token` the homeserver has is correct.
 
+
         Currently this is only called by the homeserver as a direct result of
         the application service calling
-        [`POST /_matrix/client/v1/appservice/{appserviceId}/ping`](#post_matrixclientv1appserviceappserviceidping).
+        [`POST /_matrix/client/v1/appservice/{appserviceId}/ping`].
+
+
+        [`POST /_matrix/client/v1/appservice/{appserviceId}/ping`]:
+          <#post_matrixclientv1appserviceappserviceidping>
       operationId: ping
       security:
         - homeserverAccessToken: []
@@ -41,13 +46,14 @@ paths:
               properties:
                 transaction_id:
                   type: string
-                  description: |-
+                  description: >-
                     A transaction ID for the ping, copied directly from the
                     `POST /_matrix/client/v1/appservice/{appserviceId}/ping` call.
         description: Ping body with optional transaction ID.
       responses:
         "200":
-          description: The provided `hs_token` is valid and the ping request was successful.
+          description: >-
+            The provided `hs_token` is valid and the ping request was successful.
           content:
             application/json:
               schema:

data/api/application-service/protocols.yaml

@@ -20,10 +20,10 @@ paths:
     get:
       summary: Retrieve metadata about a specific protocol that the application
         service supports.
-      description: |-
+      description: >-
         This API is called by the homeserver when it wants to present clients
-        with specific information about the various third-party networks that
+        with specific information about the various third-party networks that an
-        an application service supports.
+        application service supports.
       operationId: getProtocolMetadata
       security:
         - homeserverAccessToken: []
@@ -43,9 +43,10 @@ paths:
               schema:
                 $ref: definitions/protocol.yaml
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -80,10 +81,10 @@ paths:
   "/thirdparty/user/{protocol}":
     get:
       summary: Retrieve the Matrix User ID of a corresponding third-party user.
-      description: |-
+      description: >-
-        This API is called by the homeserver in order to retrieve a Matrix
+        This API is called by the homeserver in order to retrieve a Matrix User
-        User ID linked to a user on the third-party network, given a set of
+        ID linked to a user on the third-party network, given a set of user
-        user parameters.
+        parameters.
       operationId: queryUserByProtocol
       security:
         - homeserverAccessToken: []
@@ -97,9 +98,9 @@ paths:
             type: string
         - in: query
           name: fields
-          description: |-
+          description: >-
-            One or more custom fields that are passed to the application
+            One or more custom fields that are passed to the application service
-            service to help identify the user.
+            to help identify the user.
           schema:
             type: object
             additionalProperties:
@@ -112,9 +113,10 @@ paths:
               schema:
                 $ref: definitions/user_batch.yaml
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -149,7 +151,8 @@ paths:
   "/thirdparty/location/{protocol}":
     get:
       summary: Retrieve Matrix-side portal rooms leading to a third-party location.
-      description: Retrieve a list of Matrix portal rooms that lead to the matched
+      description: >-
+        Retrieve a list of Matrix portal rooms that lead to the matched
         third-party location.
       operationId: queryLocationByProtocol
       security:
@@ -164,9 +167,9 @@ paths:
             type: string
         - in: query
           name: fields
-          description: |-
+          description: >-
-            One or more custom fields that are passed to the application
+            One or more custom fields that are passed to the application service
-            service to help identify the third-party location.
+            to help identify the third-party location.
           schema:
             type: object
             additionalProperties:
@@ -179,9 +182,10 @@ paths:
               schema:
                 $ref: definitions/location_batch.yaml
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -216,9 +220,8 @@ paths:
   /thirdparty/location:
     get:
       summary: Reverse-lookup third-party locations given a Matrix room alias.
-      description: |-
+      description: >-
-        Retrieve an array of third-party network locations from a Matrix room
+        Retrieve an array of third-party network locations from a Matrix room alias.
-        alias.
       operationId: queryLocationByAlias
       security:
         - homeserverAccessToken: []
@@ -236,9 +239,10 @@ paths:
               schema:
                 $ref: definitions/location_batch.yaml
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -291,9 +295,10 @@ paths:
               schema:
                 $ref: definitions/user_batch.yaml
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:

data/api/application-service/query_room.yaml

@@ -20,12 +20,12 @@ paths:
   "/rooms/{roomAlias}":
     get:
       summary: Query if a room alias should exist on the application service.
-      description: |-
+      description: >-
-        This endpoint is invoked by the homeserver on an application service to query
+        This endpoint is invoked by the homeserver on an application service to
-        the existence of a given room alias. The homeserver will only query room
+        query the existence of a given room alias. The homeserver will only
-        aliases inside the application service's `aliases` namespace. The
+        query room aliases inside the application service's `aliases` namespace.
-        homeserver will send this request when it receives a request to join a
+        The homeserver will send this request when it receives a request to join
-        room alias within the application service's namespace.
+        a room alias within the application service's namespace.
       operationId: queryRoomByAlias
       security:
         - homeserverAccessToken: []
@@ -39,7 +39,7 @@ paths:
             type: string
       responses:
         "200":
-          description: |-
+          description: >-
             The application service indicates that this room alias exists. The
             application service MUST have created a room and associated it with
             the queried room alias using the client-server API. Additional
@@ -53,9 +53,10 @@ paths:
                 response:
                   value: {}
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -77,9 +78,10 @@ paths:
                     "errcode": "COM.EXAMPLE.MYAPPSERVICE_FORBIDDEN"
                   }
         "404":
-          description: |-
+          description: >-
-            The application service indicates that this room alias does not exist.
+            The application service indicates that this room alias does not
-            Optional error information can be included in the body of this response.
+            exist. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:

data/api/application-service/query_user.yaml

@@ -20,12 +20,13 @@ paths:
   "/users/{userId}":
     get:
       summary: Query if a user should exist on the application service.
-      description: |-
+      description: >-
-        This endpoint is invoked by the homeserver on an application service to query
+        This endpoint is invoked by the homeserver on an application service to
-        the existence of a given user ID. The homeserver will only query user IDs
+        query the existence of a given user ID. The homeserver will only query
-        inside the application service's `users` namespace. The homeserver will
+        user IDs inside the application service's `users` namespace. The
-        send this request when it receives an event for an unknown user ID in
+        homeserver will send this request when it receives an event for an
-        the application service's namespace, such as a room invite.
+        unknown user ID in the application service's namespace, such as a room
+        invite.
       operationId: queryUserById
       security:
         - homeserverAccessToken: []
@@ -39,9 +40,9 @@ paths:
             type: string
       responses:
         "200":
-          description: |-
+          description: >-
-            The application service indicates that this user exists. The application
+            The application service indicates that this user exists. The
-            service MUST create the user using the client-server API.
+            application service MUST create the user using the client-server API.
           content:
             application/json:
               schema:
@@ -50,9 +51,10 @@ paths:
                 response:
                   value: {}
         "401":
-          description: |-
+          description: >-
-            The homeserver has not supplied credentials to the application service.
+            The homeserver has not supplied credentials to the application
-            Optional error information can be included in the body of this response.
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -74,9 +76,10 @@ paths:
                     "errcode": "COM.EXAMPLE.MYAPPSERVICE_FORBIDDEN"
                   }
         "404":
-          description: |-
+          description: >-
             The application service indicates that this user does not exist.
-            Optional error information can be included in the body of this response.
+            Optional error information can be included in the body of this
+            response.
           content:
             application/json:
               schema:

data/api/application-service/transactions.yaml

@@ -20,20 +20,21 @@ paths:
   "/transactions/{txnId}":
     put:
       summary: Send some events to the application service.
-      description: |-
+      description: >-
-        This API is called by the homeserver when it wants to push an event
+        This API is called by the homeserver when it wants to push an event (or
-        (or batch of events) to the application service.
+        batch of events) to the application service.
 
-        Note that the application service should distinguish state events
+
-        from message events via the presence of a `state_key`, rather than
+        Note that the application service should distinguish state events from
-        via the event type.
+        message events via the presence of a `state_key`, rather than via the
+        event type.
       operationId: sendTransaction
       security:
         - homeserverAccessToken: []
       parameters:
         - in: path
           name: txnId
-          description: |-
+          description: >-
             The transaction ID for this set of events. Homeservers generate
             these IDs and they are used to ensure idempotency of requests.
           required: true
@@ -73,15 +74,19 @@ paths:
                 ephemeral:
                   type: array
                   x-addedInMatrixVersion: "1.13"
-                  description: |-
+                  description: >-
-                    A list of ephemeral data, if the `receive_ephemeral` setting was enabled in the
+                    A list of ephemeral data, if the `receive_ephemeral` setting
+                    was enabled in the
                     [registration](/application-service-api/#registration) file.
 
-                    There are only three event types that can currently occur in this list: `m.presence`,
+
-                    `m.typing`, and `m.receipt`. Room-scoped ephemeral data (`m.typing` and
+                    There are only three event types that can currently occur in
-                    `m.receipt`) MUST include a `room_id` property to identify the room that they
+                    this list: `m.presence`, `m.typing`, and `m.receipt`.
+                    Room-scoped ephemeral data (`m.typing` and `m.receipt`) MUST
+                    include a `room_id` property to identify the room that they
                     were sent in.
 
+
                     This property can be omitted if it would be empty.
                   items:
                     $ref: ../../event-schemas/schema/core-event-schema/event.yaml

data/api/client-server/create_room.yaml

@@ -109,15 +109,17 @@ paths:
                 name:
                   type: string
                   description: |-
-                    If this is included, an `m.room.name` event will be sent
+                    If this is included, an [`m.room.name`](/client-server-api/#mroomname) event
-                    into the room to indicate the name of the room. See Room
+                    will be sent into the room to indicate the name for the room.
-                    Events for more information on `m.room.name`.
+                    This overwrites any [`m.room.name`](/client-server-api/#mroomname)
+                    event in `initial_state`.
                 topic:
                   type: string
                   description: |-
-                    If this is included, an `m.room.topic` event will be sent
+                    If this is included, an [`m.room.topic`](/client-server-api/#mroomtopic)
-                    into the room to indicate the topic for the room. See Room
+                    event with a `text/plain` mimetype will be sent into the room
-                    Events for more information on `m.room.topic`.
+                    to indicate the topic for the room. This overwrites any
+                    [`m.room.topic`](/client-server-api/#mroomtopic) event in `initial_state`.
                 invite:
                   type: array
                   description: |-

data/api/client-server/definitions/public_rooms_chunk.yaml

@@ -33,7 +33,9 @@ properties:
     example: "!abcdefg:example.org"
   topic:
     type: string
-    description: The topic of the room, if any.
+    description: |-
+      The plain text topic of the room. Omitted if no `text/plain` mimetype
+      exists in [`m.room.topic`](/client-server-api/#mroomtopic).
     example: "All things general"
   world_readable:
     type: boolean

data/api/client-server/support.yaml

@@ -22,9 +22,12 @@ paths:
       description: |-
         Gets server admin contact and support page of the domain.
 
-        Like the [well-known discovery URI](/client-server-api/#well-known-uri),
+        {{% boxes/note %}}
-        this should be accessed with the hostname of the homeserver by making a
+        Like the [well-known discovery URI](/client-server-api/#well-known-uris),
+        this endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
         GET request to `https://hostname/.well-known/matrix/support`.
+        {{% /boxes/note %}}
 
         Note that this endpoint is not necessarily handled by the homeserver.
         It may be served by another webserver, used for discovering support

data/api/client-server/users.yaml

@@ -20,10 +20,17 @@ paths:
     post:
       summary: Searches the user directory.
       description: |-
-        Performs a search for users. The homeserver may
+        Performs a search for users. The homeserver may determine which
-        determine which subset of users are searched, however the homeserver
+        subset of users are searched. However, the homeserver MUST at a
-        MUST at a minimum consider the users the requesting user shares a
+        minimum consider users who are visible to the requester based
-        room with and those who reside in public rooms (known to the homeserver).
+        on their membership in rooms known to the homeserver. This means:
+
+        -   users that share a room with the requesting user
+        -   users who are joined to rooms known to the homeserver that have a
+            `public` [join rule](#mroomjoin_rules)
+        -   users who are joined to rooms known to the homeserver that have a
+            `world_readable` [history visibility](#room-history-visibility)
+        
         The search MUST consider local users to the homeserver, and SHOULD
         query remote users as part of the search.
 

data/api/client-server/wellknown.yaml

@@ -26,6 +26,12 @@ paths:
         suitably namespaced for each application and reduces the risk of
         clashes.
 
+        {{% boxes/note %}}
+        This endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
+        GET request to `https://hostname/.well-known/matrix/client`.
+        {{% /boxes/note %}}
+
         Note that this endpoint is not necessarily handled by the homeserver,
         but by another webserver, to be used for discovering the homeserver URL.
       operationId: getWellknown

data/api/server-server/invites-v1.yaml

@@ -20,7 +20,7 @@ paths:
     put:
       summary: Invites a remote user to a room
       description: |-
-        Invites a remote user to a room. Once the event has been  signed by both the inviting
+        Invites a remote user to a room. Once the event has been signed by both the inviting
         homeserver and the invited homeserver, it can be sent to all of the servers in the
         room by the inviting homeserver.
 
@@ -32,6 +32,10 @@ paths:
         [room version specification](/rooms) for precise event formats. **The request and response
         bodies here describe the common event fields in more detail and may be missing other
         required fields for a PDU.**
+
+        Also note that if the remote homeserver is already in the room, it will receive the
+        invite event twice; once through this endpoint, and again through a [federation
+        transaction](/server-server-api/#transactions).
       operationId: sendInviteV1
       security:
         - signedRequest: []

data/api/server-server/invites-v2.yaml

@@ -24,7 +24,7 @@ paths:
         This API is nearly identical to the v1 API with the exception of the request
         body being different, and the response format fixed.
 
-        Invites a remote user to a room. Once the event has been  signed by both the inviting
+        Invites a remote user to a room. Once the event has been signed by both the inviting
         homeserver and the invited homeserver, it can be sent to all of the servers in the
         room by the inviting homeserver.
 
@@ -36,6 +36,10 @@ paths:
         [room version specification](/rooms) for precise event formats. **The request and response
         bodies here describe the common event fields in more detail and may be missing other
         required fields for a PDU.**
+
+        Also note that if the remote homeserver is already in the room, it will receive the
+        invite event twice; once through this endpoint, and again through a [federation
+        transaction](/server-server-api/#transactions).
       operationId: sendInviteV2
       security:
         - signedRequest: []

data/api/server-server/wellknown.yaml

@@ -24,6 +24,12 @@ paths:
         Gets information about the delegated server for server-server communication
         between Matrix homeservers. Servers should follow 30x redirects, carefully
         avoiding redirect loops, and use normal X.509 certificate validation.
+
+        {{% boxes/note %}}
+        This endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
+        GET request to `https://hostname/.well-known/matrix/server`.
+        {{% /boxes/note %}}
       operationId: getWellKnown
       responses:
         "200":

data/event-schemas/examples/m.room.topic.yaml

@@ -3,6 +3,14 @@
   "type": "m.room.topic",
   "state_key": "",
   "content": {
-    "topic": "A room topic"
+    "m.topic": {
+      "m.text": [ {
+        "mimetype": "text/html",
+        "body": "An <em>interesting</em> room topic"
+      }, {
+        "body": "An interesting room topic"
+      }]
+    },
+    "topic": "An interesting room topic"
   }
 }

data/event-schemas/schema/components/m_text_content_block.yaml

@@ -0,0 +1,28 @@
+type: array
+description: |-
+  An ordered array of textual representations in different mimetypes.
+
+  Senders SHOULD specify at least one representation and SHOULD always
+  include a plaintext representation.
+  
+  Receivers SHOULD use the first representation in the array that
+  they understand.
+title: TextContentBlock
+items:
+  type: object
+  title: TextualRepresentation
+  properties:
+    mimetype:
+      type: string
+      description: The mimetype. Defaults to `text/plain` if omitted.
+      example: "text/html"
+    body:
+      type: string
+      description: |-
+        The string content.
+
+        Clients SHOULD validate and sanitize the content as they do
+        for rich content associated with [`msgtype`](/client-server-api/#mroommessage-msgtypes)
+        of [`m.room.message`](/client-server-api/#mroommessage).
+  required:
+  - body

data/event-schemas/schema/m.room.topic.yaml

@@ -1,20 +1,41 @@
 ---
 allOf:
   - $ref: core-event-schema/state_event.yaml
-description: |- 
+description: |-
-    A topic is a short message detailing what is currently being discussed in the room.
+    A topic is a short message detailing what is currently being discussed
-    It can also be used as a way to display extra information about the room, which may not
+    in the room.  It can also be used as a way to display extra information
-    be suitable for the room name.
+    about the room, which may not be suitable for the room name. The room
-    The room topic can also be set when creating a room using `/createRoom` with the `topic` key.'
+    topic can also be set when creating a room using
-    
+    [`/createRoom`](client-server-api/#post_matrixclientv3createroom), either
+    with the `topic` key or by specifying a full event in `initial_state`.
+  
     If the `topic` property is absent, null, or empty then the topic is unset. In other words,
     an empty `topic` property effectively resets the room to having no topic.
+
+    In order to prevent formatting abuse in room topics, clients SHOULD
+    limit the length of topics during both entry and display, for instance,
+    by capping the number of displayed lines. Additionally, clients SHOULD
+    ignore things like headings and enumerations (or format them as regular
+    text).
 properties:
   content:
     properties:
       topic:
-        description: The topic text.
+        description: |-
+          The topic in plain text.
+
+          This SHOULD duplicate the content of the `text/plain`
+          representation in `m.topic` if any exists.
         type: string
+      m.topic:
+        type: object
+        title: TopicContentBlock
+        x-addedInMatrixVersion: "1.15"
+        description: |-
+          Textual representation of the room topic in different mimetypes.
+        properties:
+          m.text:
+            $ref: components/m_text_content_block.yaml
     required:
       - topic
     type: object

scripts/dump-openapi.py

@@ -32,6 +32,35 @@ import yaml
 scripts_dir = os.path.dirname(os.path.abspath(__file__))
 api_dir = os.path.join(os.path.dirname(scripts_dir), "data", "api")
 
+# Finds a Hugo shortcode in a string.
+#
+# A shortcode is defined as (newlines and whitespaces for presentation purpose):
+#
+# {{%
+#     <zero or more whitespaces>
+#     <name of shortcode>
+#     (optional <one or more whitespaces><list of parameters>)
+#     <zero or more whitespaces>
+# %}}
+#
+# With:
+#
+# * <name of shortcode>: any word character and `-` and `/`. `re.ASCII` is used to only match
+#   ASCII characters in the name.
+# * <list of parameters>: any character except `}`, must not start or end with a
+#   whitespace.
+shortcode_regex = re.compile(r"""\{\{\%                                   # {{%
+                                 \s*                                      # zero or more whitespaces
+                                 (?P<name>[\w/-]+)                        # name of shortcode
+                                 (?:\s+(?P<params>[^\s\}][^\}]+[^\s\}]))? # optional list of parameters
+                                 \s*                                      # zero or more whitespaces
+                                 \%\}\}                                   # %}}""", re.ASCII | re.VERBOSE)
+
+# Parses the parameters of a Hugo shortcode.
+#
+# For simplicity, this currently only supports the `key="value"` format.
+shortcode_params_regex = re.compile(r"(?P<key>\w+)=\"(?P<value>[^\"]+)\"", re.ASCII)
+
 def prefix_absolute_path_references(text, base_url):
     """Adds base_url to absolute-path references.
 
@@ -44,17 +73,90 @@ def prefix_absolute_path_references(text, base_url):
     """
     return text.replace("](/", "]({}/".format(base_url))
 
-def edit_links(node, base_url):
+def replace_match(match, replacement):
-    """Finds description nodes and makes any links in them absolute."""
+    """Replaces the regex match by the replacement in the text."""
+    return match.string[:match.start()] + replacement + match.string[match.end():]
+
+def replace_shortcode(shortcode):
+    """Replaces the shortcode by a Markdown fallback in the text.
+
+    The supported shortcodes are:
+
+    * boxes/note, boxes/rationale, boxes/warning
+    * added-in, changed-in
+    
+    All closing tags (`{{ /shortcode }}`) are replaced with the empty string.
+    """
+
+    if shortcode['name'].startswith("/"):
+        # This is the end of the shortcode, just remove it.
+        return replace_match(shortcode, "")
+
+    # Parse the parameters of the shortcode
+    params = {}
+    if shortcode['params']:
+        for param in shortcode_params_regex.finditer(shortcode['params']):
+            if param['key']:
+                params[param['key']] = param['value']
+
+    match shortcode['name']:
+        case "boxes/note":
+            return replace_match(shortcode, "**NOTE:** ")
+        case "boxes/rationale":
+            return replace_match(shortcode, "**RATIONALE:** ")
+        case "boxes/warning":
+            return replace_match(shortcode, "**WARNING:** ")
+        case "added-in":
+            version = params['v']
+            if not version:
+                raise ValueError("Missing parameter `v` for `added-in` shortcode")
+
+            return replace_match(shortcode, f"**[Added in `v{version}`]** ")
+        case "changed-in":
+            version = params['v']
+            if not version:
+                raise ValueError("Missing parameter `v` for `changed-in` shortcode")
+
+            return replace_match(shortcode, f"**[Changed in `v{version}`]** ")
+        case _:
+            raise ValueError("Unknown shortcode", shortcode['name'])
+
+
+def find_and_replace_shortcodes(text):
+    """Finds Hugo shortcodes and replaces them by a Markdown fallback.
+
+    The supported shortcodes are:
+
+    * boxes/note, boxes/rationale, boxes/warning
+    * added-in, changed-in
+    """
+    # We use a `while` loop with `search` instead of a `for` loop with
+    # `finditer`, because as soon as we start replacing text, the
+    # indices of the match are invalid.
+    while shortcode := shortcode_regex.search(text):
+        text = replace_shortcode(shortcode)
+
+    return text
+
+def edit_descriptions(node, base_url):
+    """Finds description nodes and apply fixes to them.
+
+    The fixes that are applied are:
+
+    * Make links absolute
+    * Replace Hugo shortcodes
+    """
     if isinstance(node, dict):
         for key in node:
             if isinstance(node[key], str):
                 node[key] = prefix_absolute_path_references(node[key], base_url)
+                node[key] = find_and_replace_shortcodes(node[key])
             else:
-                edit_links(node[key], base_url)
+                edit_descriptions(node[key], base_url)
     elif isinstance(node, list):
         for item in node:
-            edit_links(item, base_url)
+            edit_descriptions(item, base_url)
+
 
 parser = argparse.ArgumentParser(
     "dump-openapi.py - assemble the OpenAPI specs into a single JSON file"
@@ -164,7 +266,7 @@ for filename in os.listdir(selected_api_dir):
 if untagged != 0:
     print("{} untagged operations, you may want to look into fixing that.".format(untagged))
 
-edit_links(output, base_url)
+edit_descriptions(output, base_url)
 
 print("Generating %s" % output_file)