Merge a3dc964679 into 67743d5715

With the move of the config file, the command in CI did not work as
expected anymore.
I am unsure why Hugo actually ignored the missing config file in the
command…

To avoid this problem in the future and simplify the job, we use the
default config and add an environment variable for the status which will
always take precedence over the config.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>

.github/workflows/main.yml

@@ -2,6 +2,7 @@ name: "Spec"
 
 env:
   HUGO_VERSION: 0.139.0
+  PYTHON_VERSION: 3.13
 
 on:
   push:
@@ -40,7 +41,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -59,7 +60,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -78,7 +79,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -120,7 +121,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -172,7 +173,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
       - name: "➕ Install towncrier"
         run: "pip install 'towncrier'"
       - name: "Generate changelog"
@@ -283,10 +284,11 @@ jobs:
           npm i
           npm run get-proposals
       - name: "⚙️ hugo"
+        env:
+          HUGO_PARAMS_VERSION_STATUS: "historical"
         # Create a baseURL like `/v1.2` out of the `v1.2` tag
         run: |
-          echo -e '[params.version]\nstatus="historical"' > historical.toml
-          hugo --config config.toml,historical.toml --baseURL "/${GITHUB_REF/refs\/tags\//}" -d "spec"
+          hugo --baseURL "/${GITHUB_REF/refs\/tags\//}" -d "spec"
 
       - name: "📥 Spec definition download"
         uses: actions/download-artifact@v4

changelogs/application_service/newsfragments/2131.clarification

@@ -0,0 +1 @@
+Format Application Service schemas.

changelogs/client_server/newsfragments/2095.feature

@@ -0,0 +1 @@
+Add `m.topic` content block to enable rich text in `m.room.topic` events as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765).

changelogs/client_server/newsfragments/2101.clarification

@@ -0,0 +1 @@
+"Public" rooms in profile look-ups are defined through their join rule and history visibility.

changelogs/client_server/newsfragments/2102.clarification

@@ -0,0 +1 @@
+"Public" rooms in user directory queries are defined through their join rule and history visibility.

changelogs/client_server/newsfragments/2106.clarification

@@ -0,0 +1 @@
+"Public" rooms with respect to call invites are defined through their join rule.

changelogs/client_server/newsfragments/2107.clarification

@@ -0,0 +1 @@
+"Public" rooms have no specific meaning with respect to moderation policy lists.

changelogs/client_server/newsfragments/2108.clarification

@@ -0,0 +1 @@
+"Public" rooms with respect to presence are defined through their join rule.

changelogs/client_server/newsfragments/2140.clarification

@@ -0,0 +1 @@
+Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks.

changelogs/internal/newsfragments/2088.clarification

@@ -0,0 +1 @@
+Replace Hugo shortcodes in OpenAPI output.

changelogs/internal/newsfragments/2123.clarification

@@ -0,0 +1 @@
+Fix the historical info box when generating the historical spec in CI.

changelogs/server_server/newsfragments/2067.clarification

@@ -0,0 +1 @@
+Add a note to the invite endpoints that invites to local users may be received twice over federation if the homeserver is already in the room.

changelogs/server_server/newsfragments/2095.feature

@@ -0,0 +1 @@
+Add `m.topic` content block to enable rich text in `m.room.topic` events as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765).

changelogs/server_server/newsfragments/2100.clarification

@@ -0,0 +1 @@
+Clarify that auth event of `content.join_authorised_via_users_server` is only necessary for `m.room.member` with a `membership` of `join`.

changelogs/server_server/newsfragments/2140.clarification

@@ -0,0 +1 @@
+Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks.

content/client-server-api/_index.md

@@ -371,15 +371,23 @@ valid data was obtained, but no server is available to serve the client.
 No further guess should be attempted and the user should make a
 conscientious decision what to do next.
 
-### Well-known URI
+### Well-known URIs
+
+Matrix facilitates automatic discovery for the Client-Server API base URL and more via the
+[RFC 8615](https://datatracker.ietf.org/doc/html/rfc8615) "Well-Known URI" method.
+This method uses JSON files at a predetermined location on the root path `/.well-known/` to
+specify parameter values.
 
 {{% boxes/note %}}
+Diverging from the rest of the endpoints in the Client-Server spec, these files can not be provided
+on the base URL that the Client-Server API is reachable on, as it is yet to be discovered.
+Instead, they can be reached via HTTPS on the [server name](/appendices/#server-name)'s hostname as domain.
+
 Servers hosting the `.well-known` JSON file SHOULD offer CORS headers,
 as per the [CORS](#web-browser-clients) section in this specification.
 {{% /boxes/note %}}
 
-The `.well-known` method uses a JSON file at a predetermined location to
-specify parameter values. The flow for this method is as follows:
+The flow for auto-discovery is as follows:
 
 1.  Extract the [server name](/appendices/#server-name) from the user's Matrix ID by splitting the
     Matrix ID at the first colon.
@@ -415,10 +423,17 @@ specify parameter values. The flow for this method is as follows:
 
 {{% http-api spec="client-server" api="wellknown" %}}
 
-{{% http-api spec="client-server" api="versions" %}}
-
 {{% http-api spec="client-server" api="support" %}}
 
+### API Versions
+
+Upon connecting, the Matrix client and server need to negotiate which version of the specification
+they commonly support, as the API evolves over time. The server advertises its supported versions
+and optionally unstable features to the client, which can then go on to make requests to the
+endpoints it supports.
+
+{{% http-api spec="client-server" api="versions" %}}
+
 ## Client Authentication
 
 Most API endpoints require the user to identify themselves by presenting
@@ -2847,10 +2862,15 @@ re-invited.
 
 #### Server behaviour
 
-Homeservers MUST at a minimum allow profile look-up for:
+Homeservers MUST at a minimum allow profile look-up for users who are
+visible to the requester based on their membership in rooms known to the
+homeserver. This means:
 
 -   users that share a room with the requesting user
--   users that reside in public rooms known to the homeserver
+-   users who are joined to rooms known to the homeserver that have a
+    `public` [join rule](#mroomjoin_rules)
+-   users who are joined to rooms known to the homeserver that have a
+    `world_readable` [history visibility](#room-history-visibility)
 
 In all other cases, homeservers MAY deny profile look-up by responding with
 403 and an error code of `M_FORBIDDEN`.

content/client-server-api/modules/moderation_policies.md

@@ -18,8 +18,9 @@ the entity making the decisions on filtering is best positioned to
 interpret the rules how it sees fit.
 
 Moderation policy lists are stored as room state events. There are no
-restrictions on how the rooms can be configured (they could be public,
-private, encrypted, etc).
+restrictions on how the rooms can be configured in terms of
+[join rules](#mroomjoin_rules), [history visibility](#room-history-visibility),
+encryption, etc.
 
 There are currently 3 kinds of entities which can be affected by rules:
 `user`, `server`, and `room`. All 3 are described with

content/client-server-api/modules/presence.md

@@ -68,5 +68,7 @@ will cause the server to automatically set their presence to `online`.
 
 #### Security considerations
 
-Presence information is shared with all users who share a room with the
-target user. In large public rooms this could be undesirable.
+Presence information is published to all users who share a room with the
+target user. If the target user is a member of a room with a `public`
+[join rule](#mroomjoin_rules), any other user in the federation is
+able to gain access to the target user's presence. This could be undesirable.

content/client-server-api/modules/search.md

@@ -26,9 +26,10 @@ on certain keys of certain event types.
 
 The supported keys to search over are:
 
--   `content.body` in `m.room.message`
--   `content.name` in `m.room.name`
--   `content.topic` in `m.room.topic`
+-   `content.body` in [`m.room.message`](/client-server-api/#mroommessage)
+-   `content.name` in [`m.room.name`](/client-server-api/#mroomname)
+-   In [`m.room.topic`](/client-server-api/#mroomtopic), `content.topic`
+    as well as the `body` of the `text/plain` representation in `content['m.topic']`.
 
 The search will *not* include rooms that are end to end encrypted.
 

content/client-server-api/modules/voip_events.md

@@ -202,11 +202,13 @@ specific user, and should be set to the Matrix user ID of that user. Invites
 without an `invitee` field are defined to be intended for any member of the
 room other than the sender of the event.
 
-Clients should consider an incoming call if they see a non-expired invite event where the `invitee` field is either
-absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their
-user's trust relationship with the callers and/or where the call was placed. As a starting point, it is
-suggested that clients ignore call invites from users in public rooms. It is strongly recommended that
-when clients do not ring for an incoming call invite, they still display the call invite in the room and
+Clients should consider an incoming call if they see a non-expired invite event
+where the `invitee` field is either absent or equal to their user's Matrix ID.
+They should, however, evaluate whether or not to ring based on their user's trust
+relationship with the callers and/or where the call was placed. As a starting
+point, it is RECOMMENDED that clients ignore call invites in rooms with a
+[join rule](#mroomjoin_rules) of `public`. When clients suppress ringing for an
+incoming call invite, they SHOULD still display the call invite in the room and
 annotate that it was ignored.
 
 ##### Glare

content/server-server-api.md

@@ -119,7 +119,8 @@ to send. The process overall is as follows:
     server must present a valid certificate for the hostname.
 
 3.  If the hostname is not an IP literal, a regular HTTPS request is
-    made to `https://<hostname>/.well-known/matrix/server`, expecting
+    made to `https://<hostname>/.well-known/matrix/server` (according to 
+    [RFC 8615](https://datatracker.ietf.org/doc/html/rfc8615)), expecting
     the schema defined later in this section. 30x redirects should be
     followed, however redirection loops should be avoided. Responses
     (successful or otherwise) to the `/.well-known` endpoint should be
@@ -543,8 +544,8 @@ the following subset of the room state:
       `third_party_invite` property, the current
       `m.room.third_party_invite` event with `state_key` matching
       `content.third_party_invite.signed.token`, if any.
-    - If `content.join_authorised_via_users_server` is present,
-      and the [room version supports restricted rooms](/rooms/#feature-matrix),
+    - If `membership` is `join`, `content.join_authorised_via_users_server`
+      is present, and the [room version supports restricted rooms](/rooms/#feature-matrix),
       then the `m.room.member` event with `state_key` matching
       `content.join_authorised_via_users_server`.
 

data/api/application-service/definitions/location.yaml

@@ -29,4 +29,4 @@ properties:
     }
 required: ['alias', 'protocol', 'fields']
 title: Location
-type: object
+type: object

data/api/application-service/definitions/namespace_list.yaml

@@ -19,10 +19,13 @@ items:
   properties:
     regex:
       type: string
-      description: A POSIX regular expression defining which values this namespace includes.
+      description: >-
+        A POSIX regular expression defining which values this namespace includes.
     exclusive:
       type: boolean
-      description: A true or false value stating whether this application service has exclusive access to events within this namespace.
+      description: >-
+        A true or false value stating whether this application service has
+        exclusive access to events within this namespace.
   required:
     - regex
     - exclusive

data/api/application-service/definitions/protocol.yaml

@@ -16,7 +16,7 @@ allOf:
 - type: object
   properties:
     instances:
-      description: |-
+      description: >-
         A list of objects representing independent instances of configuration.
         For example, multiple networks on IRC if multiple are provided by the
         same application service.

data/api/application-service/definitions/protocol_base.yaml

@@ -15,7 +15,7 @@ title: Protocol
 type: object
 properties:
   user_fields:
-    description: |-
+    description: >-
       Fields which may be used to identify a third-party user. These should be
       ordered to suggest the way that entities may be grouped, where higher
       groupings are ordered first. For example, the name of a network should be
@@ -26,9 +26,9 @@ properties:
       description: Field used to identify a third-party user.
     example: ["network", "nickname"]
   location_fields:
-    description: |-
-      Fields which may be used to identify a third-party location. These should be
-      ordered to suggest the way that entities may be grouped, where higher
+    description: >-
+      Fields which may be used to identify a third-party location. These should
+      be ordered to suggest the way that entities may be grouped, where higher
       groupings are ordered first. For example, the name of a network should be
       searched before the name of a channel.
     type: array
@@ -41,10 +41,11 @@ properties:
     type: string
     example: "mxc://example.org/aBcDeFgH"
   field_types:
-    description: |-
+    description: >-
       The type definitions for the fields defined in `user_fields` and
-      `location_fields`. Each entry in those arrays MUST have an entry here.
-      The `string` key for this object is the field name itself.
+      `location_fields`. Each entry in those arrays MUST have an entry here. The
+      `string` key for this object is the field name itself.
+
 
       May be an empty object if no fields are defined.
     type: object
@@ -54,10 +55,11 @@ properties:
       type: object
       properties:
         regexp:
-          description: |-
-            A regular expression for validation of a field's value. This may be relatively
-            coarse to verify the value as the application service providing this protocol
-            may apply additional validation or filtering.
+          description: >-
+            A regular expression for validation of a field's value. This may be
+            relatively coarse to verify the value as the application service
+            providing this protocol may apply additional validation or
+            filtering.
           type: string
         placeholder:
           description: A placeholder serving as a valid example of the field value.

data/api/application-service/definitions/protocol_instance.yaml

@@ -20,9 +20,9 @@ properties:
     example: "Freenode"
   icon:
     type: string
-    description: |-
-      An optional content URI representing the protocol. Overrides the one provided
-      at the higher level Protocol object.
+    description: >-
+      An optional content URI representing the protocol. Overrides the one
+      provided at the higher level Protocol object.
     example: "mxc://example.org/JkLmNoPq"
   fields:
     type: object

data/api/application-service/definitions/registration.yaml

@@ -17,27 +17,37 @@ title: Registration
 properties:
   id:
     type: string
-    description: A unique, user-defined ID of the application service which will never change.
+    description: >-
+      A unique, user-defined ID of the application service which will never change.
   url:
     type: string
-    description: The URL for the application service. May include a path after the domain name. Optionally set to null if no traffic is required.
+    description: >-
+      The URL for the application service. May include a path after the domain
+      name. Optionally set to null if no traffic is required.
   as_token:
     type: string
-    description: A secret token that the application service will use to authenticate requests to the homeserver.
+    description: >-
+      A secret token that the application service will use to authenticate
+      requests to the homeserver.
   hs_token:
     type: string
-    description: A secret token that the homeserver will use authenticate requests to the application service.
+    description: >-
+      A secret token that the homeserver will use authenticate requests to the
+      application service.
   sender_localpart:
     type: string
-    description: |-
-      The localpart of the user associated with the application service. Events will be sent to the AS if this user is the target of the event, or
-      is a joined member of the room where the event occurred.
+    description: >-
+      The localpart of the user associated with the application service. Events
+      will be sent to the AS if this user is the target of the event, or is a
+      joined member of the room where the event occurred.
   receive_ephemeral:
     type: boolean
     x-addedInMatrixVersion: "1.13"
-    description: |-
-      Whether the application service wants to [receive ephemeral data](/application-service-api/#pushing-ephemeral-data).
-      
+    description: >-
+      Whether the application service wants to [receive ephemeral
+      data](/application-service-api/#pushing-ephemeral-data).
+
+
       Defaults to `false` if not present.
   namespaces:
     type: object
@@ -47,7 +57,7 @@ properties:
       users:
         allOf:
           - $ref: namespace_list.yaml
-          - description: |-
+          - description: >-
               A list of namespaces defining the user IDs that the application
               service is interested in, in addition to its `sender_localpart`.
               Events will be sent to the AS if a local user matching one of the
@@ -56,23 +66,26 @@ properties:
       rooms:
         allOf:
           - $ref: namespace_list.yaml
-          - description: |-
+          - description: >-
               A list of namespaces defining the room IDs that the application
               service is interested in. All events sent in a room with an ID
               which matches one of the namespaces will be sent to the AS.
       aliases:
         allOf:
           - $ref: namespace_list.yaml
-          - description: |-
-              A list of namespaces defining the room aliases that the application
-              service is interested in. All events sent in a room with an alias
-              which matches one of the namespaces will be sent to the AS.
+          - description: >-
+              A list of namespaces defining the room aliases that the
+              application service is interested in. All events sent in a room
+              with an alias which matches one of the namespaces will be sent to
+              the AS.
   rate_limited:
     type: boolean
-    description: Whether requests from masqueraded users are rate-limited. The sender is excluded.
+    description: >-
+      Whether requests from masqueraded users are rate-limited. The sender is excluded.
   protocols:
     type: array
-    description: The external protocols which the application service provides (e.g. IRC).
+    description: >-
+      The external protocols which the application service provides (e.g. IRC).
     items:
       type: string
 required:

data/api/application-service/definitions/security.yaml

@@ -15,4 +15,5 @@
 homeserverAccessToken:
   type: http
   scheme: bearer
-  description: The `Bearer` `hs_token` provided by the application service's registration.
+  description: >-
+    The `Bearer` `hs_token` provided by the application service's registration.

data/api/application-service/definitions/user.yaml

@@ -30,4 +30,4 @@ properties:
     }
 required: ['userid', 'protocol', 'fields']
 title: User
-type: object
+type: object

data/api/application-service/ping.yaml

@@ -20,13 +20,18 @@ paths:
     post:
       x-addedInMatrixVersion: "1.7"
       summary: Ping the application service
-      description: |-
+      description: >-
         This API is called by the homeserver to ensure that the connection works
         and the `hs_token` the homeserver has is correct.
 
+
         Currently this is only called by the homeserver as a direct result of
         the application service calling
-        [`POST /_matrix/client/v1/appservice/{appserviceId}/ping`](#post_matrixclientv1appserviceappserviceidping).
+        [`POST /_matrix/client/v1/appservice/{appserviceId}/ping`].
+
+
+        [`POST /_matrix/client/v1/appservice/{appserviceId}/ping`]:
+          <#post_matrixclientv1appserviceappserviceidping>
       operationId: ping
       security:
         - homeserverAccessToken: []
@@ -41,13 +46,14 @@ paths:
               properties:
                 transaction_id:
                   type: string
-                  description: |-
+                  description: >-
                     A transaction ID for the ping, copied directly from the
                     `POST /_matrix/client/v1/appservice/{appserviceId}/ping` call.
         description: Ping body with optional transaction ID.
       responses:
         "200":
-          description: The provided `hs_token` is valid and the ping request was successful.
+          description: >-
+            The provided `hs_token` is valid and the ping request was successful.
           content:
             application/json:
               schema:

data/api/application-service/protocols.yaml

@@ -20,10 +20,10 @@ paths:
     get:
       summary: Retrieve metadata about a specific protocol that the application
         service supports.
-      description: |-
+      description: >-
         This API is called by the homeserver when it wants to present clients
-        with specific information about the various third-party networks that
-        an application service supports.
+        with specific information about the various third-party networks that an
+        application service supports.
       operationId: getProtocolMetadata
       security:
         - homeserverAccessToken: []
@@ -43,9 +43,10 @@ paths:
               schema:
                 $ref: definitions/protocol.yaml
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -80,10 +81,10 @@ paths:
   "/thirdparty/user/{protocol}":
     get:
       summary: Retrieve the Matrix User ID of a corresponding third-party user.
-      description: |-
-        This API is called by the homeserver in order to retrieve a Matrix
-        User ID linked to a user on the third-party network, given a set of
-        user parameters.
+      description: >-
+        This API is called by the homeserver in order to retrieve a Matrix User
+        ID linked to a user on the third-party network, given a set of user
+        parameters.
       operationId: queryUserByProtocol
       security:
         - homeserverAccessToken: []
@@ -97,9 +98,9 @@ paths:
             type: string
         - in: query
           name: fields
-          description: |-
-            One or more custom fields that are passed to the application
-            service to help identify the user.
+          description: >-
+            One or more custom fields that are passed to the application service
+            to help identify the user.
           schema:
             type: object
             additionalProperties:
@@ -112,9 +113,10 @@ paths:
               schema:
                 $ref: definitions/user_batch.yaml
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -149,7 +151,8 @@ paths:
   "/thirdparty/location/{protocol}":
     get:
       summary: Retrieve Matrix-side portal rooms leading to a third-party location.
-      description: Retrieve a list of Matrix portal rooms that lead to the matched
+      description: >-
+        Retrieve a list of Matrix portal rooms that lead to the matched
         third-party location.
       operationId: queryLocationByProtocol
       security:
@@ -164,9 +167,9 @@ paths:
             type: string
         - in: query
           name: fields
-          description: |-
-            One or more custom fields that are passed to the application
-            service to help identify the third-party location.
+          description: >-
+            One or more custom fields that are passed to the application service
+            to help identify the third-party location.
           schema:
             type: object
             additionalProperties:
@@ -179,9 +182,10 @@ paths:
               schema:
                 $ref: definitions/location_batch.yaml
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -216,9 +220,8 @@ paths:
   /thirdparty/location:
     get:
       summary: Reverse-lookup third-party locations given a Matrix room alias.
-      description: |-
-        Retrieve an array of third-party network locations from a Matrix room
-        alias.
+      description: >-
+        Retrieve an array of third-party network locations from a Matrix room alias.
       operationId: queryLocationByAlias
       security:
         - homeserverAccessToken: []
@@ -236,9 +239,10 @@ paths:
               schema:
                 $ref: definitions/location_batch.yaml
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -291,9 +295,10 @@ paths:
               schema:
                 $ref: definitions/user_batch.yaml
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:

data/api/application-service/query_room.yaml

@@ -20,12 +20,12 @@ paths:
   "/rooms/{roomAlias}":
     get:
       summary: Query if a room alias should exist on the application service.
-      description: |-
-        This endpoint is invoked by the homeserver on an application service to query
-        the existence of a given room alias. The homeserver will only query room
-        aliases inside the application service's `aliases` namespace. The
-        homeserver will send this request when it receives a request to join a
-        room alias within the application service's namespace.
+      description: >-
+        This endpoint is invoked by the homeserver on an application service to
+        query the existence of a given room alias. The homeserver will only
+        query room aliases inside the application service's `aliases` namespace.
+        The homeserver will send this request when it receives a request to join
+        a room alias within the application service's namespace.
       operationId: queryRoomByAlias
       security:
         - homeserverAccessToken: []
@@ -39,7 +39,7 @@ paths:
             type: string
       responses:
         "200":
-          description: |-
+          description: >-
             The application service indicates that this room alias exists. The
             application service MUST have created a room and associated it with
             the queried room alias using the client-server API. Additional
@@ -53,9 +53,10 @@ paths:
                 response:
                   value: {}
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -77,9 +78,10 @@ paths:
                     "errcode": "COM.EXAMPLE.MYAPPSERVICE_FORBIDDEN"
                   }
         "404":
-          description: |-
-            The application service indicates that this room alias does not exist.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The application service indicates that this room alias does not
+            exist. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:

data/api/application-service/query_user.yaml

@@ -20,12 +20,13 @@ paths:
   "/users/{userId}":
     get:
       summary: Query if a user should exist on the application service.
-      description: |-
-        This endpoint is invoked by the homeserver on an application service to query
-        the existence of a given user ID. The homeserver will only query user IDs
-        inside the application service's `users` namespace. The homeserver will
-        send this request when it receives an event for an unknown user ID in
-        the application service's namespace, such as a room invite.
+      description: >-
+        This endpoint is invoked by the homeserver on an application service to
+        query the existence of a given user ID. The homeserver will only query
+        user IDs inside the application service's `users` namespace. The
+        homeserver will send this request when it receives an event for an
+        unknown user ID in the application service's namespace, such as a room
+        invite.
       operationId: queryUserById
       security:
         - homeserverAccessToken: []
@@ -39,9 +40,9 @@ paths:
             type: string
       responses:
         "200":
-          description: |-
-            The application service indicates that this user exists. The application
-            service MUST create the user using the client-server API.
+          description: >-
+            The application service indicates that this user exists. The
+            application service MUST create the user using the client-server API.
           content:
             application/json:
               schema:
@@ -50,9 +51,10 @@ paths:
                 response:
                   value: {}
         "401":
-          description: |-
-            The homeserver has not supplied credentials to the application service.
-            Optional error information can be included in the body of this response.
+          description: >-
+            The homeserver has not supplied credentials to the application
+            service. Optional error information can be included in the body of
+            this response.
           content:
             application/json:
               schema:
@@ -74,9 +76,10 @@ paths:
                     "errcode": "COM.EXAMPLE.MYAPPSERVICE_FORBIDDEN"
                   }
         "404":
-          description: |-
+          description: >-
             The application service indicates that this user does not exist.
-            Optional error information can be included in the body of this response.
+            Optional error information can be included in the body of this
+            response.
           content:
             application/json:
               schema:

data/api/application-service/transactions.yaml

@@ -20,20 +20,21 @@ paths:
   "/transactions/{txnId}":
     put:
       summary: Send some events to the application service.
-      description: |-
-        This API is called by the homeserver when it wants to push an event
-        (or batch of events) to the application service.
+      description: >-
+        This API is called by the homeserver when it wants to push an event (or
+        batch of events) to the application service.
 
-        Note that the application service should distinguish state events
-        from message events via the presence of a `state_key`, rather than
-        via the event type.
+
+        Note that the application service should distinguish state events from
+        message events via the presence of a `state_key`, rather than via the
+        event type.
       operationId: sendTransaction
       security:
         - homeserverAccessToken: []
       parameters:
         - in: path
           name: txnId
-          description: |-
+          description: >-
             The transaction ID for this set of events. Homeservers generate
             these IDs and they are used to ensure idempotency of requests.
           required: true
@@ -73,15 +74,19 @@ paths:
                 ephemeral:
                   type: array
                   x-addedInMatrixVersion: "1.13"
-                  description: |-
-                    A list of ephemeral data, if the `receive_ephemeral` setting was enabled in the
+                  description: >-
+                    A list of ephemeral data, if the `receive_ephemeral` setting
+                    was enabled in the
                     [registration](/application-service-api/#registration) file.
 
-                    There are only three event types that can currently occur in this list: `m.presence`,
-                    `m.typing`, and `m.receipt`. Room-scoped ephemeral data (`m.typing` and
-                    `m.receipt`) MUST include a `room_id` property to identify the room that they
+
+                    There are only three event types that can currently occur in
+                    this list: `m.presence`, `m.typing`, and `m.receipt`.
+                    Room-scoped ephemeral data (`m.typing` and `m.receipt`) MUST
+                    include a `room_id` property to identify the room that they
                     were sent in.
 
+
                     This property can be omitted if it would be empty.
                   items:
                     $ref: ../../event-schemas/schema/core-event-schema/event.yaml

data/api/client-server/create_room.yaml

@@ -109,15 +109,17 @@ paths:
                 name:
                   type: string
                   description: |-
-                    If this is included, an `m.room.name` event will be sent
-                    into the room to indicate the name of the room. See Room
-                    Events for more information on `m.room.name`.
+                    If this is included, an [`m.room.name`](/client-server-api/#mroomname) event
+                    will be sent into the room to indicate the name for the room.
+                    This overwrites any [`m.room.name`](/client-server-api/#mroomname)
+                    event in `initial_state`.
                 topic:
                   type: string
                   description: |-
-                    If this is included, an `m.room.topic` event will be sent
-                    into the room to indicate the topic for the room. See Room
-                    Events for more information on `m.room.topic`.
+                    If this is included, an [`m.room.topic`](/client-server-api/#mroomtopic)
+                    event with a `text/plain` mimetype will be sent into the room
+                    to indicate the topic for the room. This overwrites any
+                    [`m.room.topic`](/client-server-api/#mroomtopic) event in `initial_state`.
                 invite:
                   type: array
                   description: |-

data/api/client-server/definitions/public_rooms_chunk.yaml

@@ -33,7 +33,9 @@ properties:
     example: "!abcdefg:example.org"
   topic:
     type: string
-    description: The topic of the room, if any.
+    description: |-
+      The plain text topic of the room. Omitted if no `text/plain` mimetype
+      exists in [`m.room.topic`](/client-server-api/#mroomtopic).
     example: "All things general"
   world_readable:
     type: boolean

data/api/client-server/support.yaml

@@ -22,9 +22,12 @@ paths:
       description: |-
         Gets server admin contact and support page of the domain.
 
-        Like the [well-known discovery URI](/client-server-api/#well-known-uri),
-        this should be accessed with the hostname of the homeserver by making a
+        {{% boxes/note %}}
+        Like the [well-known discovery URI](/client-server-api/#well-known-uris),
+        this endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
         GET request to `https://hostname/.well-known/matrix/support`.
+        {{% /boxes/note %}}
 
         Note that this endpoint is not necessarily handled by the homeserver.
         It may be served by another webserver, used for discovering support

data/api/client-server/users.yaml

@@ -20,10 +20,17 @@ paths:
     post:
       summary: Searches the user directory.
       description: |-
-        Performs a search for users. The homeserver may
-        determine which subset of users are searched, however the homeserver
-        MUST at a minimum consider the users the requesting user shares a
-        room with and those who reside in public rooms (known to the homeserver).
+        Performs a search for users. The homeserver may determine which
+        subset of users are searched. However, the homeserver MUST at a
+        minimum consider users who are visible to the requester based
+        on their membership in rooms known to the homeserver. This means:
+
+        -   users that share a room with the requesting user
+        -   users who are joined to rooms known to the homeserver that have a
+            `public` [join rule](#mroomjoin_rules)
+        -   users who are joined to rooms known to the homeserver that have a
+            `world_readable` [history visibility](#room-history-visibility)
+        
         The search MUST consider local users to the homeserver, and SHOULD
         query remote users as part of the search.
 

data/api/client-server/wellknown.yaml

@@ -26,6 +26,12 @@ paths:
         suitably namespaced for each application and reduces the risk of
         clashes.
 
+        {{% boxes/note %}}
+        This endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
+        GET request to `https://hostname/.well-known/matrix/client`.
+        {{% /boxes/note %}}
+
         Note that this endpoint is not necessarily handled by the homeserver,
         but by another webserver, to be used for discovering the homeserver URL.
       operationId: getWellknown

data/api/server-server/invites-v1.yaml

@@ -20,7 +20,7 @@ paths:
     put:
       summary: Invites a remote user to a room
       description: |-
-        Invites a remote user to a room. Once the event has been  signed by both the inviting
+        Invites a remote user to a room. Once the event has been signed by both the inviting
         homeserver and the invited homeserver, it can be sent to all of the servers in the
         room by the inviting homeserver.
 
@@ -32,6 +32,10 @@ paths:
         [room version specification](/rooms) for precise event formats. **The request and response
         bodies here describe the common event fields in more detail and may be missing other
         required fields for a PDU.**
+
+        Also note that if the remote homeserver is already in the room, it will receive the
+        invite event twice; once through this endpoint, and again through a [federation
+        transaction](/server-server-api/#transactions).
       operationId: sendInviteV1
       security:
         - signedRequest: []

data/api/server-server/invites-v2.yaml

@@ -24,7 +24,7 @@ paths:
         This API is nearly identical to the v1 API with the exception of the request
         body being different, and the response format fixed.
 
-        Invites a remote user to a room. Once the event has been  signed by both the inviting
+        Invites a remote user to a room. Once the event has been signed by both the inviting
         homeserver and the invited homeserver, it can be sent to all of the servers in the
         room by the inviting homeserver.
 
@@ -36,6 +36,10 @@ paths:
         [room version specification](/rooms) for precise event formats. **The request and response
         bodies here describe the common event fields in more detail and may be missing other
         required fields for a PDU.**
+
+        Also note that if the remote homeserver is already in the room, it will receive the
+        invite event twice; once through this endpoint, and again through a [federation
+        transaction](/server-server-api/#transactions).
       operationId: sendInviteV2
       security:
         - signedRequest: []

data/api/server-server/wellknown.yaml

@@ -24,6 +24,12 @@ paths:
         Gets information about the delegated server for server-server communication
         between Matrix homeservers. Servers should follow 30x redirects, carefully
         avoiding redirect loops, and use normal X.509 certificate validation.
+
+        {{% boxes/note %}}
+        This endpoint should be accessed with the hostname of the homeserver's
+        [server name](/appendices/#server-name) by making a
+        GET request to `https://hostname/.well-known/matrix/server`.
+        {{% /boxes/note %}}
       operationId: getWellKnown
       responses:
         "200":

data/event-schemas/examples/m.room.topic.yaml

@@ -3,6 +3,14 @@
   "type": "m.room.topic",
   "state_key": "",
   "content": {
-    "topic": "A room topic"
+    "m.topic": {
+      "m.text": [ {
+        "mimetype": "text/html",
+        "body": "An <em>interesting</em> room topic"
+      }, {
+        "body": "An interesting room topic"
+      }]
+    },
+    "topic": "An interesting room topic"
   }
 }

data/event-schemas/schema/components/m_text_content_block.yaml

@@ -0,0 +1,28 @@
+type: array
+description: |-
+  An ordered array of textual representations in different mimetypes.
+
+  Senders SHOULD specify at least one representation and SHOULD always
+  include a plaintext representation.
+  
+  Receivers SHOULD use the first representation in the array that
+  they understand.
+title: TextContentBlock
+items:
+  type: object
+  title: TextualRepresentation
+  properties:
+    mimetype:
+      type: string
+      description: The mimetype. Defaults to `text/plain` if omitted.
+      example: "text/html"
+    body:
+      type: string
+      description: |-
+        The string content.
+
+        Clients SHOULD validate and sanitize the content as they do
+        for rich content associated with [`msgtype`](/client-server-api/#mroommessage-msgtypes)
+        of [`m.room.message`](/client-server-api/#mroommessage).
+  required:
+  - body

data/event-schemas/schema/m.room.topic.yaml

@@ -1,20 +1,41 @@
 ---
 allOf:
   - $ref: core-event-schema/state_event.yaml
-description: |- 
-    A topic is a short message detailing what is currently being discussed in the room.
-    It can also be used as a way to display extra information about the room, which may not
-    be suitable for the room name.
-    The room topic can also be set when creating a room using `/createRoom` with the `topic` key.'
-    
+description: |-
+    A topic is a short message detailing what is currently being discussed
+    in the room.  It can also be used as a way to display extra information
+    about the room, which may not be suitable for the room name. The room
+    topic can also be set when creating a room using
+    [`/createRoom`](client-server-api/#post_matrixclientv3createroom), either
+    with the `topic` key or by specifying a full event in `initial_state`.
+  
     If the `topic` property is absent, null, or empty then the topic is unset. In other words,
     an empty `topic` property effectively resets the room to having no topic.
+
+    In order to prevent formatting abuse in room topics, clients SHOULD
+    limit the length of topics during both entry and display, for instance,
+    by capping the number of displayed lines. Additionally, clients SHOULD
+    ignore things like headings and enumerations (or format them as regular
+    text).
 properties:
   content:
     properties:
       topic:
-        description: The topic text.
+        description: |-
+          The topic in plain text.
+
+          This SHOULD duplicate the content of the `text/plain`
+          representation in `m.topic` if any exists.
         type: string
+      m.topic:
+        type: object
+        title: TopicContentBlock
+        x-addedInMatrixVersion: "1.15"
+        description: |-
+          Textual representation of the room topic in different mimetypes.
+        properties:
+          m.text:
+            $ref: components/m_text_content_block.yaml
     required:
       - topic
     type: object

scripts/dump-openapi.py

@@ -32,6 +32,35 @@ import yaml
 scripts_dir = os.path.dirname(os.path.abspath(__file__))
 api_dir = os.path.join(os.path.dirname(scripts_dir), "data", "api")
 
+# Finds a Hugo shortcode in a string.
+#
+# A shortcode is defined as (newlines and whitespaces for presentation purpose):
+#
+# {{%
+#     <zero or more whitespaces>
+#     <name of shortcode>
+#     (optional <one or more whitespaces><list of parameters>)
+#     <zero or more whitespaces>
+# %}}
+#
+# With:
+#
+# * <name of shortcode>: any word character and `-` and `/`. `re.ASCII` is used to only match
+#   ASCII characters in the name.
+# * <list of parameters>: any character except `}`, must not start or end with a
+#   whitespace.
+shortcode_regex = re.compile(r"""\{\{\%                                   # {{%
+                                 \s*                                      # zero or more whitespaces
+                                 (?P<name>[\w/-]+)                        # name of shortcode
+                                 (?:\s+(?P<params>[^\s\}][^\}]+[^\s\}]))? # optional list of parameters
+                                 \s*                                      # zero or more whitespaces
+                                 \%\}\}                                   # %}}""", re.ASCII | re.VERBOSE)
+
+# Parses the parameters of a Hugo shortcode.
+#
+# For simplicity, this currently only supports the `key="value"` format.
+shortcode_params_regex = re.compile(r"(?P<key>\w+)=\"(?P<value>[^\"]+)\"", re.ASCII)
+
 def prefix_absolute_path_references(text, base_url):
     """Adds base_url to absolute-path references.
 
@@ -44,17 +73,90 @@ def prefix_absolute_path_references(text, base_url):
     """
     return text.replace("](/", "]({}/".format(base_url))
 
-def edit_links(node, base_url):
-    """Finds description nodes and makes any links in them absolute."""
+def replace_match(match, replacement):
+    """Replaces the regex match by the replacement in the text."""
+    return match.string[:match.start()] + replacement + match.string[match.end():]
+
+def replace_shortcode(shortcode):
+    """Replaces the shortcode by a Markdown fallback in the text.
+
+    The supported shortcodes are:
+
+    * boxes/note, boxes/rationale, boxes/warning
+    * added-in, changed-in
+    
+    All closing tags (`{{ /shortcode }}`) are replaced with the empty string.
+    """
+
+    if shortcode['name'].startswith("/"):
+        # This is the end of the shortcode, just remove it.
+        return replace_match(shortcode, "")
+
+    # Parse the parameters of the shortcode
+    params = {}
+    if shortcode['params']:
+        for param in shortcode_params_regex.finditer(shortcode['params']):
+            if param['key']:
+                params[param['key']] = param['value']
+
+    match shortcode['name']:
+        case "boxes/note":
+            return replace_match(shortcode, "**NOTE:** ")
+        case "boxes/rationale":
+            return replace_match(shortcode, "**RATIONALE:** ")
+        case "boxes/warning":
+            return replace_match(shortcode, "**WARNING:** ")
+        case "added-in":
+            version = params['v']
+            if not version:
+                raise ValueError("Missing parameter `v` for `added-in` shortcode")
+
+            return replace_match(shortcode, f"**[Added in `v{version}`]** ")
+        case "changed-in":
+            version = params['v']
+            if not version:
+                raise ValueError("Missing parameter `v` for `changed-in` shortcode")
+
+            return replace_match(shortcode, f"**[Changed in `v{version}`]** ")
+        case _:
+            raise ValueError("Unknown shortcode", shortcode['name'])
+
+
+def find_and_replace_shortcodes(text):
+    """Finds Hugo shortcodes and replaces them by a Markdown fallback.
+
+    The supported shortcodes are:
+
+    * boxes/note, boxes/rationale, boxes/warning
+    * added-in, changed-in
+    """
+    # We use a `while` loop with `search` instead of a `for` loop with
+    # `finditer`, because as soon as we start replacing text, the
+    # indices of the match are invalid.
+    while shortcode := shortcode_regex.search(text):
+        text = replace_shortcode(shortcode)
+
+    return text
+
+def edit_descriptions(node, base_url):
+    """Finds description nodes and apply fixes to them.
+
+    The fixes that are applied are:
+
+    * Make links absolute
+    * Replace Hugo shortcodes
+    """
     if isinstance(node, dict):
         for key in node:
             if isinstance(node[key], str):
                 node[key] = prefix_absolute_path_references(node[key], base_url)
+                node[key] = find_and_replace_shortcodes(node[key])
             else:
-                edit_links(node[key], base_url)
+                edit_descriptions(node[key], base_url)
     elif isinstance(node, list):
         for item in node:
-            edit_links(item, base_url)
+            edit_descriptions(item, base_url)
+
 
 parser = argparse.ArgumentParser(
     "dump-openapi.py - assemble the OpenAPI specs into a single JSON file"
@@ -164,7 +266,7 @@ for filename in os.listdir(selected_api_dir):
 if untagged != 0:
     print("{} untagged operations, you may want to look into fixing that.".format(untagged))
 
-edit_links(output, base_url)
+edit_descriptions(output, base_url)
 
 print("Generating %s" % output_file)