Merge branch 'main' into johannes/msc4147

Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
Co-authored-by: Kim Brose <2803622+HarHarLinks@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

changelogs/application_service/newsfragments/2130.feature

@@ -0,0 +1 @@
+Correct null value handling for the AS Registration's `url` property.

changelogs/client_server/newsfragments/2104.clarification

@@ -0,0 +1 @@
+Rooms published in `/publicRooms` don't necessarily have `public` join rules or `world_readable` history visibility.

changelogs/client_server/newsfragments/2144.clarification

@@ -0,0 +1 @@
+Fix typo: as->has.

changelogs/server_server/newsfragments/2104.clarification

@@ -0,0 +1 @@
+Rooms published in `/publicRooms` don't necessarily have `public` join rules or `world_readable` history visibility.

content/application-service-api.md

@@ -492,10 +492,10 @@ via the query string). It is expected that the application service use
 the transactions pushed to it to handle events rather than syncing with
 the user implied by `sender_localpart`.
 
-#### Application service room directories
+#### Published room directories
 
-Application services can maintain their own room directories for their
-defined third-party protocols. These room directories may be accessed by
+Application services can maintain their own published room directories for
+their defined third-party protocols. These directories may be accessed by
 clients through additional parameters on the `/publicRooms`
 client-server endpoint.
 

content/client-server-api/_index.md

@@ -2846,7 +2846,35 @@ re-invited.
 
 {{% http-api spec="client-server" api="banning" %}}
 
-### Listing rooms
+### Published room directory
+
+Homeservers MAY publish a room directory to allow users to discover rooms. A room
+can have one of two visibility settings in the directory:
+
+-   `public`: The room will be shown in the published room directory.
+-   `private`: The room will be hidden from the published room directory.
+
+Clients can define a room's initial visibility in the directory via the `visibility`
+parameter in [`/createRoom`](#post_matrixclientv3createroom). Irrespective of room
+creation, clients can query and change a room's visibility in the directory through
+the endpoints listed below, provided that the server permits this.
+
+{{% boxes/warning %}}
+The visibility setting merely defines whether a room is included in the published
+room directory or not. It doesn't make any guarantees about the room's
+[join rule](#mroomjoin_rules) or [history visibility](#room-history-visibility).
+
+In particular, a visibility setting of `public` should not be confused with a `public`
+join rule. Rooms with a join rule of `knock`, for instance, could reasonably be published
+in the directory, too.
+
+Similarly, a visibility setting of `public` does not necessarily imply a `world_readable`
+history visibility.
+
+To increase performance or by preference, servers MAY apply additional filters when listing the
+directory, for instance, by automatically excluding rooms with `invite` join rules
+that are not `world_readable` regardless of their visibility.
+{{% /boxes/warning %}}
 
 {{% http-api spec="client-server" api="list_public_rooms" %}}
 

content/client-server-api/modules/end_to_end_encryption.md

@@ -528,7 +528,7 @@ messages, Alice only sends one request event (an event with type
 `m.room.message` with `msgtype: m.key.verification.request`, rather than an
 event with type `m.key.verification.request`), to the room. In addition, Alice
 does not send an `m.key.verification.cancel` event to tell Bob's other devices
-that the request as already been accepted; instead, when Bob's other devices
+that the request has already been accepted; instead, when Bob's other devices
 see his `m.key.verification.ready` event, they will know that the request has
 already been accepted, and that they should ignore the request.
 
@@ -1517,25 +1517,63 @@ The plaintext payload is of the form:
 The type and content of the plaintext message event are given in the
 payload.
 
-Other properties are included in order to prevent an attacker from
-publishing someone else's curve25519 keys as their own and subsequently
-claiming to have sent messages which they didn't. `sender` must
-correspond to the user who sent the event, `recipient` to the local
-user, and `recipient_keys` to the local ed25519 key.
+###### Validation of incoming decrypted events
 
-Clients must ensure that the sending device owns the private part of
-the ed25519 key it claims to have in the Olm payload. This is crucial
-when the ed25519 key corresponds to a verified device. To perform
-this check, clients MUST confirm that the `sender_key` property in the
-cleartext `m.room.encrypted` event body, and the `keys.ed25519` property
-in the decrypted plaintext, match the keys under the `sender_device_keys`
-property. Additionally, clients MUST also verify the signature of the keys.
-If `sender_device_keys` is absent, clients MUST retrieve the sender's
-keys from [`/keys/query`](#post_matrixclientv3keysquery) instead. This
-will not allow them to verify key ownership if the sending device was
-logged out or had its keys reset since sending the event. Therefore,
-clients MUST populate the `sender_device_keys` property when sending
-events themselves.
+After decrypting an incoming encrypted event, clients MUST apply the
+following checks:
+
+1.  The `sender` property in the decrypted content must match the
+    `sender` of the event.
+2.  The `keys.ed25519` property in the decrypted content must match
+    the `sender_key` property in the cleartext `m.room.encrypted`
+    event body.
+3.  The `recipient` property in the decrypted content must match
+    the user ID of the local user.
+4.  The `recipient_keys.ed25519` property in the decrypted content
+    must match the client device's [Ed25519 signing key](#device-keys).
+5.  Where `sender_device_keys` is present in the decrypted content:
+    1.  `sender_device_keys.user_id` must also match the `sender`
+        of the event.
+    2.  `sender_device_keys.keys.ed25519:<device_id>` must also match
+        the `sender_key` property in the cleartext `m.room.encrypted`
+        event body.
+    3.  `sender_device_keys.keys.curve25519:<device_id>` must match
+        the Curve25519 key used to establish the Olm session.
+    4.  The `sender_device_keys` structure must have a valid signature
+        from the key with ID `ed25519:<device_id>` (i.e., the sending
+        device's Ed25519 key).
+
+Any event that does not comply with these checks MUST be discarded.
+
+###### Verification of the sending user for incoming events
+
+In addition, for each Olm session, clients MUST verify that the
+Curve25519 key used to establish the Olm session does indeed belong
+to the claimed `sender`. This requires a signed "device keys" structure
+for that Curve25519 key, which can be obtained in one of two ways:
+
+1.  An Olm message may be received with a `sender_device_keys` property
+    in the decrypted content.
+2.  The keys are returned via a [`/keys/query`](#post_matrixclientv3keysquery)
+    request. Note that both the Curve25519 key **and** the Ed25519 key in
+    the returned device keys structure must match those used in an
+    Olm-encrypted event as above. (In particular, the Ed25519 key must
+    be present in the **encrypted** content of an Olm-encrypted event
+    to prevent an attacker from claiming another user's Curve25519 key
+    as their own.)
+
+Ownership of the Curve25519 key is then established in one of two ways:
+
+1.  Via [cross-signing](#cross-signing). For this to be sufficient, the
+    device keys structure must be signed by the sender's self-signing key,
+    and that self-signing key must itself have been validated (either via
+    [explicit verification](#device-verification) or a TOFU mechanism).
+2.  Via explicit verification of the device's Ed25519 signing key, as
+    contained in the device keys structure. This is no longer recommended.
+
+A failure to complete these verifications does not necessarily mean that
+the session is bogus; however it is the case that there is no proof that
+the claimed sender is accurate, and the user should be warned accordingly.
 
 If a client has multiple sessions established with another device, it
 should use the session from which it last received and successfully

content/server-server-api.md

@@ -1048,11 +1048,10 @@ user's Matrix ID and the token delivered when the invite was stored,
 this verification will prove that the `m.room.member` invite event comes
 from the user owning the invited third-party identifier.
 
-## Public Room Directory
+## Published Room Directory
 
-To complement the [Client-Server
-API](/client-server-api)'s room directory,
-homeservers need a way to query the public rooms for another server.
+To complement the [room directory in the Client-Server API](/client-server-api#published-room-directory), 
+homeservers need a way to query the published rooms of another server.
 This can be done by making a request to the `/publicRooms` endpoint for
 the server the room directory should be retrieved for.
 

data/api/application-service/definitions/registration.yaml

@@ -19,7 +19,7 @@ properties:
     type: string
     description: A unique, user-defined ID of the application service which will never change.
   url:
-    type: string
+    type: ["null", "string"]
     description: The URL for the application service. May include a path after the domain name. Optionally set to null if no traffic is required.
   as_token:
     type: string

data/api/client-server/appservice_room_directory.yaml

@@ -13,18 +13,21 @@
 # limitations under the License.
 openapi: 3.1.0
 info:
-  title: Matrix Client-Server Application Service Room Directory API
+  title: Matrix Client-Server Application Service Published Room Directory API
   version: 1.0.0
 paths:
   "/directory/list/appservice/{networkId}/{roomId}":
     put:
-      summary: Updates a room's visibility in the application service's room directory.
-      description: |-
-        Updates the visibility of a given room on the application service's room
+      summary: |-
+        Updates a room's visibility in the application service's published room
         directory.
+      description: |-
+        Updates the visibility of a given room in the application service's
+        published room directory.
 
-        This API is similar to the room directory visibility API used by clients
-        to update the homeserver's more general room directory.
+        This API is similar to the
+        [visibility API](/client-server-api#put_matrixclientv3directorylistroomroomid)
+        used by clients to update the homeserver's more general published room directory.
 
         This API requires the use of an application service access token (`as_token`)
         instead of a typical client's access_token. This API cannot be invoked by

data/api/client-server/create_room.yaml

@@ -87,12 +87,9 @@ paths:
                     - public
                     - private
                   description: |-
-                    A `public` visibility indicates that the room will be shown
-                    in the published room list. A `private` visibility will hide
-                    the room from the published room list. Rooms default to
-                    `private` visibility if this key is not included. NB: This
-                    should not be confused with `join_rules` which also uses the
-                    word `public`.
+                    The room's visibility in the server's
+                    [published room directory](/client-server-api#published-room-directory).
+                    Defaults to `private`.
                 room_alias_name:
                   type: string
                   description: |-

data/api/client-server/definitions/public_rooms_chunk.yaml

@@ -13,7 +13,7 @@
 # limitations under the License.
 
 type: object
-title: "PublicRoomsChunk"
+title: "PublishedRoomsChunk"
 properties:
   canonical_alias:
     type: string

data/api/client-server/definitions/public_rooms_response.yaml

@@ -13,28 +13,15 @@
 # limitations under the License.
 
 type: object
-description: A list of the rooms on the server.
+description: A list of the published rooms on the server.
 required: ["chunk"]
 properties:
   chunk:
     type: array
     description: |-
-      A paginated chunk of public rooms.
+      A paginated chunk of published rooms.
     items:
-      allOf:
-        - $ref: "public_rooms_chunk.yaml"
-        - type: object
-          title: PublicRoomsChunk
-          properties:
-            # Override description of join_rule
-            join_rule:
-              type: string
-              description: |-
-                The room's join rule. When not present, the room is assumed to
-                be `public`. Note that rooms with `invite` join rules are not
-                expected here, but rooms with `knock` rules are given their
-                near-public nature.
-              example: "public"
+      $ref: "public_rooms_chunk.yaml"
   next_batch:
     type: string
     description: |-
@@ -50,7 +37,7 @@ properties:
   total_room_count_estimate:
     type: integer
     description: |-
-        An estimate on the total number of public rooms, if the
+        An estimate on the total number of published rooms, if the
         server has an estimate.
 example: {
   "chunk": [

data/api/client-server/list_public_rooms.yaml

@@ -13,14 +13,15 @@
 # limitations under the License.
 openapi: 3.1.0
 info:
-  title: Matrix Client-Server Room Directory API
+  title: Matrix Client-Server Published Room Directory API
   version: 1.0.0
 paths:
   "/directory/list/room/{roomId}":
     get:
       summary: Gets the visibility of a room in the directory
-      description: Gets the visibility of a given room on the server's public room
-        directory.
+      description: |-
+        Gets the visibility of a given room in the server's
+        published room directory.
       operationId: getRoomVisibilityOnDirectory
       parameters:
         - in: path
@@ -32,7 +33,7 @@ paths:
             type: string
       responses:
         "200":
-          description: The visibility of the room in the directory
+          description: The visibility of the room in the directory.
           content:
             application/json:
               schema:
@@ -50,7 +51,7 @@ paths:
                     "visibility": "public"
                   }
         "404":
-          description: The room is not known to the server
+          description: The room is not known to the server.
           content:
             application/json:
               schema:
@@ -64,14 +65,13 @@ paths:
       tags:
         - Room discovery
     put:
-      summary: Sets the visibility of a room in the room directory
+      summary: Sets the visibility of a room in the directory
       description: |-
-        Sets the visibility of a given room in the server's public room
-        directory.
+        Sets the visibility of a given room in the server's published room directory.
 
-        Servers may choose to implement additional access control checks
-        here, for instance that room visibility can only be changed by
-        the room creator or a server administrator.
+        Servers MAY implement additional access control checks, for instance,
+        to ensure that a room's visibility can only be changed by the room creator
+        or a server administrator.
       operationId: setRoomVisibilityOnDirectory
       security:
         - accessTokenQuery: []
@@ -97,11 +97,11 @@ paths:
                     - public
                   description: |-
                     The new visibility setting for the room.
-                    Defaults to 'public'.
+                    Defaults to `public`.
               example: {
                 "visibility": "public"
               }
-        description: The new visibility for the room on the room directory.
+        description: The new visibility for the room in the published room directory.
         required: true
       responses:
         "200":
@@ -114,7 +114,7 @@ paths:
                 response:
                   value: {}
         "404":
-          description: The room is not known to the server
+          description: The room is not known to the server.
           content:
             application/json:
               schema:
@@ -129,9 +129,9 @@ paths:
         - Room discovery
   /publicRooms:
     get:
-      summary: Lists the public rooms on the server.
+      summary: Lists a server's published room directory
       description: |-
-        Lists the public rooms on the server.
+        Lists a server's published room directory.
 
         This API returns paginated responses. The rooms are ordered by the number
         of joined members, with the largest rooms first.
@@ -154,13 +154,13 @@ paths:
         - in: query
           name: server
           description: |-
-            The server to fetch the public room lists from. Defaults to the
-            local server. Case sensitive.
+            The server to fetch the published room directory from. Defaults
+            to the local server. Case sensitive.
           schema:
             type: string
       responses:
         "200":
-          description: A list of the rooms on the server.
+          description: A list of the published rooms on the server.
           content:
             application/json:
               schema:
@@ -168,9 +168,9 @@ paths:
       tags:
         - Room discovery
     post:
-      summary: Lists the public rooms on the server with optional filter.
+      summary: Lists a server's published room directory with an optional filter
       description: |-
-        Lists the public rooms on the server, with optional filter.
+        Lists a server's published room directory with an optional filter.
 
         This API returns paginated responses. The rooms are ordered by the number
         of joined members, with the largest rooms first.
@@ -182,8 +182,8 @@ paths:
         - in: query
           name: server
           description: |-
-            The server to fetch the public room lists from. Defaults to the
-            local server. Case sensitive.
+            The server to fetch the published room directory from. Defaults
+            to the local server. Case sensitive.
           schema:
             type: string
       requestBody:
@@ -253,7 +253,7 @@ paths:
         required: true
       responses:
         "200":
-          description: A list of the rooms on the server.
+          description: A filtered list of the published rooms on the server.
           content:
             application/json:
               schema:

data/api/server-server/public_rooms.yaml

@@ -13,16 +13,20 @@
 # limitations under the License.
 openapi: 3.1.0
 info:
-  title: Matrix Federation Public Rooms API
+  title: Matrix Federation Published Room Directory API
   version: 1.0.0
 paths:
   /publicRooms:
     get:
-      summary: Get all the public rooms for a homeserver
+      summary: Lists the server's published room directory
       description: |-
-        Gets all the public rooms for the homeserver. This should not return
-        rooms that are listed on another homeserver's directory, just those
-        listed on the receiving homeserver's directory.
+        Lists the server's published room directory.
+
+        This API returns paginated responses. The rooms are ordered by the number
+        of joined members, with the largest rooms first.
+
+        This SHOULD not return rooms that are listed on another homeserver's directory,
+        just those listed on the receiving homeserver's directory.
       operationId: getPublicRooms
       security:
         - signedRequest: []
@@ -62,21 +66,18 @@ paths:
             type: string
       responses:
         "200":
-          description: The public room list for the homeserver.
+          description: A list of the published rooms on the server.
           content:
             application/json:
               schema:
                 $ref: ../client-server/definitions/public_rooms_response.yaml
     post:
-      summary: Gets the public rooms on the server with optional filter.
+      summary: Lists the server's published room directory with an optional filter
       description: |-
-        Lists the public rooms on the server, with optional filter.
+        Lists the server's published room directory with an optional filter.
 
         This API returns paginated responses. The rooms are ordered by the number
         of joined members, with the largest rooms first.
-
-        Note that this endpoint receives and returns the same format that is seen
-        in the Client-Server API's `POST /publicRooms` endpoint.
       operationId: queryPublicRooms
       security:
         - signedRequest: []
@@ -147,69 +148,11 @@ paths:
         required: true
       responses:
         "200":
-          description: A list of the rooms on the server.
+          description: A filtered list of the published rooms on the server.
           content:
             application/json:
               schema:
-                type: object
-                description: A list of the rooms on the server.
-                required:
-                  - chunk
-                properties:
-                  chunk:
-                    title: PublicRoomsChunks
-                    type: array
-                    description: A paginated chunk of public rooms.
-                    items:
-                      allOf:
-                        - $ref: ../client-server/definitions/public_rooms_chunk.yaml
-                        - type: object
-                          properties:
-                            # Override description of join_rule
-                            join_rule:
-                              type: string
-                              description: |-
-                                The room's join rule. When not present, the room is assumed to
-                                be `public`. Note that rooms with `invite` join rules are not
-                                expected here, but rooms with `knock` rules are given their
-                                near-public nature.
-                  next_batch:
-                    type: string
-                    description: |-
-                      A pagination token for the response. The absence of this token
-                      means there are no more results to fetch and the client should
-                      stop paginating.
-                  prev_batch:
-                    type: string
-                    description: |-
-                      A pagination token that allows fetching previous results. The
-                      absence of this token means there are no results before this
-                      batch, i.e. this is the first batch.
-                  total_room_count_estimate:
-                    type: integer
-                    description: |-
-                      An estimate on the total number of public rooms, if the
-                      server has an estimate.
-              examples:
-                response:
-                  value: {
-                    "chunk": [
-                      {
-                        "avatar_url": "mxc://bleecker.street/CHEDDARandBRIE",
-                        "guest_can_join": false,
-                        "name": "CHEESE",
-                        "num_joined_members": 37,
-                        "room_id": "!ol19s:bleecker.street",
-                        "topic": "Tasty tasty cheese",
-                        "world_readable": true,
-                        "join_rule": "public",
-                        "room_type": "m.space"
-                      }
-                    ],
-                    "next_batch": "p190q",
-                    "prev_batch": "p1902",
-                    "total_room_count_estimate": 115
-                  }
+                $ref: ../client-server/definitions/public_rooms_response.yaml
 servers:
   - url: "{protocol}://{hostname}{basePath}"
     variables: