Merge c1894e09f0 into 2c734c3c5b

Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
Co-authored-by: Kim Brose <2803622+HarHarLinks@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

changelogs/client_server/newsfragments/2104.clarification

@@ -0,0 +1 @@
+Rooms published in `/publicRooms` don't necessarily have `public` join rules or `world_readable` history visibility.

changelogs/client_server/newsfragments/2122.feature

@@ -0,0 +1 @@
+Include device keys with Olm-encrypted events as per [MSC4147](https://github.com/matrix-org/matrix-spec-proposals/pull/4147).

changelogs/client_server/newsfragments/2144.clarification

@@ -0,0 +1 @@
+Fix typo: as->has.

changelogs/server_server/newsfragments/2104.clarification

@@ -0,0 +1 @@
+Rooms published in `/publicRooms` don't necessarily have `public` join rules or `world_readable` history visibility.

content/application-service-api.md

@@ -492,10 +492,10 @@ via the query string). It is expected that the application service use
 the transactions pushed to it to handle events rather than syncing with
 the user implied by `sender_localpart`.
 
-#### Application service room directories
+#### Published room directories
 
-Application services can maintain their own room directories for their
+Application services can maintain their own published room directories for
-defined third-party protocols. These room directories may be accessed by
+their defined third-party protocols. These directories may be accessed by
 clients through additional parameters on the `/publicRooms`
 client-server endpoint.
 

content/client-server-api/_index.md

@@ -2846,7 +2846,35 @@ re-invited.
 
 {{% http-api spec="client-server" api="banning" %}}
 
-### Listing rooms
+### Published room directory
+
+Homeservers MAY publish a room directory to allow users to discover rooms. A room
+can have one of two visibility settings in the directory:
+
+-   `public`: The room will be shown in the published room directory.
+-   `private`: The room will be hidden from the published room directory.
+
+Clients can define a room's initial visibility in the directory via the `visibility`
+parameter in [`/createRoom`](#post_matrixclientv3createroom). Irrespective of room
+creation, clients can query and change a room's visibility in the directory through
+the endpoints listed below, provided that the server permits this.
+
+{{% boxes/warning %}}
+The visibility setting merely defines whether a room is included in the published
+room directory or not. It doesn't make any guarantees about the room's
+[join rule](#mroomjoin_rules) or [history visibility](#room-history-visibility).
+
+In particular, a visibility setting of `public` should not be confused with a `public`
+join rule. Rooms with a join rule of `knock`, for instance, could reasonably be published
+in the directory, too.
+
+Similarly, a visibility setting of `public` does not necessarily imply a `world_readable`
+history visibility.
+
+To increase performance or by preference, servers MAY apply additional filters when listing the
+directory, for instance, by automatically excluding rooms with `invite` join rules
+that are not `world_readable` regardless of their visibility.
+{{% /boxes/warning %}}
 
 {{% http-api spec="client-server" api="list_public_rooms" %}}
 

content/client-server-api/modules/end_to_end_encryption.md

@@ -528,7 +528,7 @@ messages, Alice only sends one request event (an event with type
 `m.room.message` with `msgtype: m.key.verification.request`, rather than an
 event with type `m.key.verification.request`), to the room. In addition, Alice
 does not send an `m.key.verification.cancel` event to tell Bob's other devices
-that the request as already been accepted; instead, when Bob's other devices
+that the request has already been accepted; instead, when Bob's other devices
 see his `m.key.verification.ready` event, they will know that the request has
 already been accepted, and that they should ignore the request.
 
@@ -1512,20 +1512,7 @@ message.
 
 The plaintext payload is of the form:
 
-```json
+{{% definition path="api/client-server/definitions/olm_payload" %}}
-{
-  "type": "<type of the plaintext event>",
-  "content": "<content for the plaintext event>",
-  "sender": "<sender_user_id>",
-  "recipient": "<recipient_user_id>",
-  "recipient_keys": {
-    "ed25519": "<our_ed25519_key>"
-  },
-  "keys": {
-    "ed25519": "<sender_ed25519_key>"
-  }
-}
-```
 
 The type and content of the plaintext message event are given in the
 payload.
@@ -1536,15 +1523,19 @@ claiming to have sent messages which they didn't. `sender` must
 correspond to the user who sent the event, `recipient` to the local
 user, and `recipient_keys` to the local ed25519 key.
 
-Clients must confirm that the `sender_key` property in the cleartext
+Clients must ensure that the sending device owns the private part of
-`m.room.encrypted` event body, and the `keys.ed25519` property in the
+the ed25519 key it claims to have in the Olm payload. This is crucial
-decrypted plaintext, match the keys returned by
+when the ed25519 key corresponds to a verified device. To perform
-[`/keys/query`](#post_matrixclientv3keysquery) for
+this check, clients MUST confirm that the `sender_key` property in the
-the given user. Clients must also verify the signature of the keys from the
+cleartext `m.room.encrypted` event body, and the `keys.ed25519` property
-`/keys/query` response. Without this check, a client cannot be sure that
+in the decrypted plaintext, match the keys under the `sender_device_keys`
-the sender device owns the private part of the ed25519 key it claims to
+property. Additionally, clients MUST also verify the signature of the keys.
-have in the Olm payload. This is crucial when the ed25519 key corresponds
+If `sender_device_keys` is absent, clients MUST retrieve the sender's
-to a verified device.
+keys from [`/keys/query`](#post_matrixclientv3keysquery) instead. This
+will not allow them to verify key ownership if the sending device was
+logged out or had its keys reset since sending the event. Therefore,
+clients MUST populate the `sender_device_keys` property when sending
+events themselves.
 
 If a client has multiple sessions established with another device, it
 should use the session from which it last received and successfully

content/server-server-api.md

@@ -1048,11 +1048,10 @@ user's Matrix ID and the token delivered when the invite was stored,
 this verification will prove that the `m.room.member` invite event comes
 from the user owning the invited third-party identifier.
 
-## Public Room Directory
+## Published Room Directory
 
-To complement the [Client-Server
+To complement the [room directory in the Client-Server API](/client-server-api#published-room-directory), 
-API](/client-server-api)'s room directory,
+homeservers need a way to query the published rooms of another server.
-homeservers need a way to query the public rooms for another server.
 This can be done by making a request to the `/publicRooms` endpoint for
 the server the room directory should be retrieved for.
 

data/api/client-server/appservice_room_directory.yaml

@@ -13,18 +13,21 @@
 # limitations under the License.
 openapi: 3.1.0
 info:
-  title: Matrix Client-Server Application Service Room Directory API
+  title: Matrix Client-Server Application Service Published Room Directory API
   version: 1.0.0
 paths:
   "/directory/list/appservice/{networkId}/{roomId}":
     put:
-      summary: Updates a room's visibility in the application service's room directory.
+      summary: |-
-      description: |-
+        Updates a room's visibility in the application service's published room
-        Updates the visibility of a given room on the application service's room
         directory.
+      description: |-
+        Updates the visibility of a given room in the application service's
+        published room directory.
 
-        This API is similar to the room directory visibility API used by clients
+        This API is similar to the
-        to update the homeserver's more general room directory.
+        [visibility API](/client-server-api#put_matrixclientv3directorylistroomroomid)
+        used by clients to update the homeserver's more general published room directory.
 
         This API requires the use of an application service access token (`as_token`)
         instead of a typical client's access_token. This API cannot be invoked by

data/api/client-server/create_room.yaml

@@ -87,12 +87,9 @@ paths:
                     - public
                     - private
                   description: |-
-                    A `public` visibility indicates that the room will be shown
+                    The room's visibility in the server's
-                    in the published room list. A `private` visibility will hide
+                    [published room directory](/client-server-api#published-room-directory).
-                    the room from the published room list. Rooms default to
+                    Defaults to `private`.
-                    `private` visibility if this key is not included. NB: This
-                    should not be confused with `join_rules` which also uses the
-                    word `public`.
                 room_alias_name:
                   type: string
                   description: |-

data/api/client-server/definitions/olm_payload.yaml

@@ -0,0 +1,88 @@
+# Copyright 2025 The Matrix.org Foundation C.I.C
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+type: object
+title: OlmPayload
+description: |-
+  The plaintext payload of Olm message events.
+properties:
+  type:
+    type: string
+    description: The type of the event.
+  content:
+    type: object
+    description: The event content.
+  sender:
+    type: string
+    description: The user ID of the event sender.
+  recipient:
+    type: string
+    description: The user ID of the intended event recipient.
+  recipient_keys:
+    description: The recipient's signing keys of the encrypted event.
+    $ref: "#/components/schemas/SigningKeys"
+  keys:
+    $ref: "#/components/schemas/SigningKeys"
+    description: The sender's signing keys of the encrypted event.
+  sender_device_keys:
+    $ref: device_keys.yaml 
+    description: The sender's device keys.
+    x-addedInMatrixVersion: "1.15"
+required:
+  - type
+  - content
+  - sender
+  - recipient
+  - recipient_keys
+  - keys
+components:
+  schemas:
+    SigningKeys:
+      type: object
+      title: SigningKeys
+      description: Public keys used for an `m.olm.v1.curve25519-aes-sha2` event.
+      properties:
+        ed25519:
+          type: string
+          description: The Ed25519 public key encoded using unpadded base64.
+      required:
+        - ed25519
+example: {
+  "type": "<type of the plaintext event>",
+  "content": "<content for the plaintext event>",
+  "sender": "<sender_user_id>",
+  "recipient": "<recipient_user_id>",
+  "recipient_keys": {
+    "ed25519": "<our_ed25519_key>"
+  },
+  "keys": {
+    "ed25519": "<sender_ed25519_key>"
+  },
+  "sender_device_keys": {
+    "algorithms": ["<supported>", "<algorithms>"],
+    "user_id": "<user_id>",
+    "device_id": "<device_id>",
+    "keys": {
+      "ed25519:<device_id>": "<sender_ed25519_key>",
+      "curve25519:<device_id>": "<sender_curve25519_key>"
+    },
+    "signatures": {
+      "<user_id>": {
+        "ed25519:<device_id>": "<device_signature>",
+        "ed25519:<ssk_id>": "<ssk_signature>",
+      }
+    }
+  }
+}

data/api/client-server/definitions/public_rooms_chunk.yaml

@@ -13,7 +13,7 @@
 # limitations under the License.
 
 type: object
-title: "PublicRoomsChunk"
+title: "PublishedRoomsChunk"
 properties:
   canonical_alias:
     type: string

data/api/client-server/definitions/public_rooms_response.yaml

@@ -13,28 +13,15 @@
 # limitations under the License.
 
 type: object
-description: A list of the rooms on the server.
+description: A list of the published rooms on the server.
 required: ["chunk"]
 properties:
   chunk:
     type: array
     description: |-
-      A paginated chunk of public rooms.
+      A paginated chunk of published rooms.
     items:
-      allOf:
+      $ref: "public_rooms_chunk.yaml"
-        - $ref: "public_rooms_chunk.yaml"
-        - type: object
-          title: PublicRoomsChunk
-          properties:
-            # Override description of join_rule
-            join_rule:
-              type: string
-              description: |-
-                The room's join rule. When not present, the room is assumed to
-                be `public`. Note that rooms with `invite` join rules are not
-                expected here, but rooms with `knock` rules are given their
-                near-public nature.
-              example: "public"
   next_batch:
     type: string
     description: |-
@@ -50,7 +37,7 @@ properties:
   total_room_count_estimate:
     type: integer
     description: |-
-        An estimate on the total number of public rooms, if the
+        An estimate on the total number of published rooms, if the
         server has an estimate.
 example: {
   "chunk": [

data/api/client-server/list_public_rooms.yaml

@@ -13,14 +13,15 @@
 # limitations under the License.
 openapi: 3.1.0
 info:
-  title: Matrix Client-Server Room Directory API
+  title: Matrix Client-Server Published Room Directory API
   version: 1.0.0
 paths:
   "/directory/list/room/{roomId}":
     get:
       summary: Gets the visibility of a room in the directory
-      description: Gets the visibility of a given room on the server's public room
+      description: |-
-        directory.
+        Gets the visibility of a given room in the server's
+        published room directory.
       operationId: getRoomVisibilityOnDirectory
       parameters:
         - in: path
@@ -32,7 +33,7 @@ paths:
             type: string
       responses:
         "200":
-          description: The visibility of the room in the directory
+          description: The visibility of the room in the directory.
           content:
             application/json:
               schema:
@@ -50,7 +51,7 @@ paths:
                     "visibility": "public"
                   }
         "404":
-          description: The room is not known to the server
+          description: The room is not known to the server.
           content:
             application/json:
               schema:
@@ -64,14 +65,13 @@ paths:
       tags:
         - Room discovery
     put:
-      summary: Sets the visibility of a room in the room directory
+      summary: Sets the visibility of a room in the directory
       description: |-
-        Sets the visibility of a given room in the server's public room
+        Sets the visibility of a given room in the server's published room directory.
-        directory.
 
-        Servers may choose to implement additional access control checks
+        Servers MAY implement additional access control checks, for instance,
-        here, for instance that room visibility can only be changed by
+        to ensure that a room's visibility can only be changed by the room creator
-        the room creator or a server administrator.
+        or a server administrator.
       operationId: setRoomVisibilityOnDirectory
       security:
         - accessTokenQuery: []
@@ -97,11 +97,11 @@ paths:
                     - public
                   description: |-
                     The new visibility setting for the room.
-                    Defaults to 'public'.
+                    Defaults to `public`.
               example: {
                 "visibility": "public"
               }
-        description: The new visibility for the room on the room directory.
+        description: The new visibility for the room in the published room directory.
         required: true
       responses:
         "200":
@@ -114,7 +114,7 @@ paths:
                 response:
                   value: {}
         "404":
-          description: The room is not known to the server
+          description: The room is not known to the server.
           content:
             application/json:
               schema:
@@ -129,9 +129,9 @@ paths:
         - Room discovery
   /publicRooms:
     get:
-      summary: Lists the public rooms on the server.
+      summary: Lists a server's published room directory
       description: |-
-        Lists the public rooms on the server.
+        Lists a server's published room directory.
 
         This API returns paginated responses. The rooms are ordered by the number
         of joined members, with the largest rooms first.
@@ -154,13 +154,13 @@ paths:
         - in: query
           name: server
           description: |-
-            The server to fetch the public room lists from. Defaults to the
+            The server to fetch the published room directory from. Defaults
-            local server. Case sensitive.
+            to the local server. Case sensitive.
           schema:
             type: string
       responses:
         "200":
-          description: A list of the rooms on the server.
+          description: A list of the published rooms on the server.
           content:
             application/json:
               schema:
@@ -168,9 +168,9 @@ paths:
       tags:
         - Room discovery
     post:
-      summary: Lists the public rooms on the server with optional filter.
+      summary: Lists a server's published room directory with an optional filter
       description: |-
-        Lists the public rooms on the server, with optional filter.
+        Lists a server's published room directory with an optional filter.
 
         This API returns paginated responses. The rooms are ordered by the number
         of joined members, with the largest rooms first.
@@ -182,8 +182,8 @@ paths:
         - in: query
           name: server
           description: |-
-            The server to fetch the public room lists from. Defaults to the
+            The server to fetch the published room directory from. Defaults
-            local server. Case sensitive.
+            to the local server. Case sensitive.
           schema:
             type: string
       requestBody:
@@ -253,7 +253,7 @@ paths:
         required: true
       responses:
         "200":
-          description: A list of the rooms on the server.
+          description: A filtered list of the published rooms on the server.
           content:
             application/json:
               schema:

data/api/server-server/public_rooms.yaml

@@ -13,16 +13,20 @@
 # limitations under the License.
 openapi: 3.1.0
 info:
-  title: Matrix Federation Public Rooms API
+  title: Matrix Federation Published Room Directory API
   version: 1.0.0
 paths:
   /publicRooms:
     get:
-      summary: Get all the public rooms for a homeserver
+      summary: Lists the server's published room directory
       description: |-
-        Gets all the public rooms for the homeserver. This should not return
+        Lists the server's published room directory.
-        rooms that are listed on another homeserver's directory, just those
+
-        listed on the receiving homeserver's directory.
+        This API returns paginated responses. The rooms are ordered by the number
+        of joined members, with the largest rooms first.
+
+        This SHOULD not return rooms that are listed on another homeserver's directory,
+        just those listed on the receiving homeserver's directory.
       operationId: getPublicRooms
       security:
         - signedRequest: []
@@ -62,21 +66,18 @@ paths:
             type: string
       responses:
         "200":
-          description: The public room list for the homeserver.
+          description: A list of the published rooms on the server.
           content:
             application/json:
               schema:
                 $ref: ../client-server/definitions/public_rooms_response.yaml
     post:
-      summary: Gets the public rooms on the server with optional filter.
+      summary: Lists the server's published room directory with an optional filter
       description: |-
-        Lists the public rooms on the server, with optional filter.
+        Lists the server's published room directory with an optional filter.
 
         This API returns paginated responses. The rooms are ordered by the number
         of joined members, with the largest rooms first.
-
-        Note that this endpoint receives and returns the same format that is seen
-        in the Client-Server API's `POST /publicRooms` endpoint.
       operationId: queryPublicRooms
       security:
         - signedRequest: []
@@ -147,69 +148,11 @@ paths:
         required: true
       responses:
         "200":
-          description: A list of the rooms on the server.
+          description: A filtered list of the published rooms on the server.
           content:
             application/json:
               schema:
-                type: object
+                $ref: ../client-server/definitions/public_rooms_response.yaml
-                description: A list of the rooms on the server.
-                required:
-                  - chunk
-                properties:
-                  chunk:
-                    title: PublicRoomsChunks
-                    type: array
-                    description: A paginated chunk of public rooms.
-                    items:
-                      allOf:
-                        - $ref: ../client-server/definitions/public_rooms_chunk.yaml
-                        - type: object
-                          properties:
-                            # Override description of join_rule
-                            join_rule:
-                              type: string
-                              description: |-
-                                The room's join rule. When not present, the room is assumed to
-                                be `public`. Note that rooms with `invite` join rules are not
-                                expected here, but rooms with `knock` rules are given their
-                                near-public nature.
-                  next_batch:
-                    type: string
-                    description: |-
-                      A pagination token for the response. The absence of this token
-                      means there are no more results to fetch and the client should
-                      stop paginating.
-                  prev_batch:
-                    type: string
-                    description: |-
-                      A pagination token that allows fetching previous results. The
-                      absence of this token means there are no results before this
-                      batch, i.e. this is the first batch.
-                  total_room_count_estimate:
-                    type: integer
-                    description: |-
-                      An estimate on the total number of public rooms, if the
-                      server has an estimate.
-              examples:
-                response:
-                  value: {
-                    "chunk": [
-                      {
-                        "avatar_url": "mxc://bleecker.street/CHEDDARandBRIE",
-                        "guest_can_join": false,
-                        "name": "CHEESE",
-                        "num_joined_members": 37,
-                        "room_id": "!ol19s:bleecker.street",
-                        "topic": "Tasty tasty cheese",
-                        "world_readable": true,
-                        "join_rule": "public",
-                        "room_type": "m.space"
-                      }
-                    ],
-                    "next_batch": "p190q",
-                    "prev_batch": "p1902",
-                    "total_room_count_estimate": 115
-                  }
 servers:
   - url: "{protocol}://{hostname}{basePath}"
     variables: