Merge 6183f2410f into 643a6dca2d

Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
Co-authored-by: Richard van der Hoff <richard@matrix.org>

changelogs/client_server/newsfragments/2071.feature

@@ -0,0 +1 @@
+Feature: Update profile endpoints to become generic to support [MSC4133](https://github.com/matrix-org/matrix-spec-proposals/pull/4133) extended fields. Extended profile fields are now supported via the new `m.profile_fields` capability, which deprecates the previous `m.set_avatar_url` and `m.set_displayname` capabilities. Stabilised keys are explicitly enumerated, and custom keys must conform to the Common Namespaced Identifier Grammar.

changelogs/client_server/newsfragments/2122.feature

@@ -0,0 +1 @@
+Include device keys with Olm-encrypted events as per [MSC4147](https://github.com/matrix-org/matrix-spec-proposals/pull/4147).

content/client-server-api/modules/end_to_end_encryption.md

@@ -1512,40 +1512,11 @@ message.
 
 The plaintext payload is of the form:
 
-```json
-{
-  "type": "<type of the plaintext event>",
-  "content": "<content for the plaintext event>",
-  "sender": "<sender_user_id>",
-  "recipient": "<recipient_user_id>",
-  "recipient_keys": {
-    "ed25519": "<our_ed25519_key>"
-  },
-  "keys": {
-    "ed25519": "<sender_ed25519_key>"
-  }
-}
-```
+{{% definition path="api/client-server/definitions/olm_payload" %}}
 
 The type and content of the plaintext message event are given in the
 payload.
 
-Other properties are included in order to prevent an attacker from
-publishing someone else's curve25519 keys as their own and subsequently
-claiming to have sent messages which they didn't. `sender` must
-correspond to the user who sent the event, `recipient` to the local
-user, and `recipient_keys` to the local ed25519 key.
-
-Clients must confirm that the `sender_key` property in the cleartext
-`m.room.encrypted` event body, and the `keys.ed25519` property in the
-decrypted plaintext, match the keys returned by
-[`/keys/query`](#post_matrixclientv3keysquery) for
-the given user. Clients must also verify the signature of the keys from the
-`/keys/query` response. Without this check, a client cannot be sure that
-the sender device owns the private part of the ed25519 key it claims to
-have in the Olm payload. This is crucial when the ed25519 key corresponds
-to a verified device.
-
 If a client has multiple sessions established with another device, it
 should use the session from which it last received and successfully
 decrypted a message. For these purposes, a session that has not received
@@ -1555,6 +1526,68 @@ maximum number of olm sessions that it will maintain for each device,
 and expiring sessions on a Least Recently Used basis. The maximum number
 of olm sessions maintained per device should be at least 4.
 
+###### Validation of incoming decrypted events
+
+{{% changed-in v="1.15" %}} Existing checks made more explicit, and checks for `sender_device_keys` added.
+
+After decrypting an incoming encrypted event, clients MUST apply the
+following checks:
+
+1.  The `sender` property in the decrypted content must match the
+    `sender` of the event.
+2.  The `keys.ed25519` property in the decrypted content must match
+    the `sender_key` property in the cleartext `m.room.encrypted`
+    event body.
+3.  The `recipient` property in the decrypted content must match
+    the user ID of the local user.
+4.  The `recipient_keys.ed25519` property in the decrypted content
+    must match the client device's [Ed25519 signing key](#device-keys).
+5.  Where `sender_device_keys` is present in the decrypted content:
+    1.  `sender_device_keys.user_id` must also match the `sender`
+        of the event.
+    2.  `sender_device_keys.keys.ed25519:<device_id>` must also match
+        the `sender_key` property in the cleartext `m.room.encrypted`
+        event body.
+    3.  `sender_device_keys.keys.curve25519:<device_id>` must match
+        the Curve25519 key used to establish the Olm session.
+    4.  The `sender_device_keys` structure must have a valid signature
+        from the key with ID `ed25519:<device_id>` (i.e., the sending
+        device's Ed25519 key).
+
+Any event that does not comply with these checks MUST be discarded.
+
+###### Verification of the sending user for incoming events
+
+{{% added-in v="1.15" %}}
+
+In addition, for each Olm session, clients MUST verify that the
+Curve25519 key used to establish the Olm session does indeed belong
+to the claimed `sender`. This requires a signed "device keys" structure
+for that Curve25519 key, which can be obtained in one of two ways:
+
+1.  An Olm message may be received with a `sender_device_keys` property
+    in the decrypted content.
+2.  The keys are returned via a [`/keys/query`](#post_matrixclientv3keysquery)
+    request. Note that both the Curve25519 key **and** the Ed25519 key in
+    the returned device keys structure must match those used in an
+    Olm-encrypted event as above. (In particular, the Ed25519 key must
+    be present in the **encrypted** content of an Olm-encrypted event
+    to prevent an attacker from claiming another user's Curve25519 key
+    as their own.)
+
+Ownership of the Curve25519 key is then established in one of two ways:
+
+1.  Via [cross-signing](#cross-signing). For this to be sufficient, the
+    device keys structure must be signed by the sender's self-signing key,
+    and that self-signing key must itself have been validated (either via
+    [explicit verification](#device-verification) or a "trust on first use" (TOFU) mechanism).
+2.  Via explicit verification of the device's Ed25519 signing key, as
+    contained in the device keys structure. This is no longer recommended.
+
+A failure to complete these verifications does not necessarily mean that
+the session is bogus; however it is the case that there is no proof that
+the claimed sender is accurate, and the user should be warned accordingly.
+
 ###### Recovering from undecryptable messages
 
 Occasionally messages may be undecryptable by clients due to a variety

content/client-server-api/modules/guest_access.md

@@ -63,7 +63,7 @@ for sending events:
 The following API endpoints are allowed to be accessed by guest accounts
 for their own account maintenance:
 
-* [PUT /profile/{userId}/displayname](#put_matrixclientv3profileuseriddisplayname)
+* [PUT /profile/{userId}/{key_name}](#put_matrixclientv3profileuseridkeyname)
 * [GET /devices](#get_matrixclientv3devices)
 * [GET /devices/{deviceId}](#get_matrixclientv3devicesdeviceid)
 * [PUT /devices/{deviceId}](#put_matrixclientv3devicesdeviceid)

data/api/client-server/capabilities.yaml

@@ -73,11 +73,17 @@ paths:
                           - default
                           - available
                       m.set_displayname:
+                        deprecated: true
                         $ref: '#/components/schemas/booleanCapability'
-                        description: Capability to indicate if the user can change their display name.
+                        description: |
+                          **Deprecated:** Capability to indicate if the user can change their display name.
+                          Please refer to `m.profile_fields` for extended profile management.
                       m.set_avatar_url:
+                        deprecated: true
                         $ref: '#/components/schemas/booleanCapability'
-                        description: Capability to indicate if the user can change their avatar.
+                        description: |
+                          **Deprecated:** Capability to indicate if the user can change their avatar.
+                          Please refer to `m.profile_fields` for extended profile management.
                       m.3pid_changes:
                         $ref: '#/components/schemas/booleanCapability'
                         description: Capability to indicate if the user can change 3PID associations
@@ -86,6 +92,37 @@ paths:
                         $ref: '#/components/schemas/booleanCapability'
                         description: Capability to indicate if the user can generate tokens to log further
                           clients into their account.
+                      m.profile_fields:
+                        x-addedInMatrixVersion: "1.14"
+                        type: object
+                        title: ProfileFieldsCapability
+                        description: Capability to indicate if the user can set or modify extended profile fields.
+                          If absent, clients should assume custom profile fields are supported.
+                        properties:
+                          allowed:
+                            type: array
+                            description: List of allowed additional custom profile field keys. A `*` can be used as a
+                              wildcard to match any sequence of characters. This list takes precedence over the
+                              disallowed list if both are provided.
+                            items:
+                              type: string
+                            example:
+                              - "m.example_field"
+                              - "org.example/job_title"
+                          disallowed:
+                            type: array
+                            description: List of disallowed additional custom profile field keys. A `*` can be used as
+                              a wildcard to match any sequence of characters. Ignored if an allowed list is provided.
+                            items:
+                              type: string
+                            example:
+                              - "org.example.secret_field"
+                          enabled:
+                            type: boolean
+                            description: True if the user can set or modify any extended profile fields, false otherwise.
+                            example: true
+                        required:
+                          - enabled
               examples:
                 response:
                   value: {

data/api/client-server/definitions/olm_payload.yaml

@@ -0,0 +1,88 @@
+# Copyright 2025 The Matrix.org Foundation C.I.C
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+type: object
+title: OlmPayload
+description: |-
+  The plaintext payload of Olm message events.
+properties:
+  type:
+    type: string
+    description: The type of the event.
+  content:
+    type: object
+    description: The event content.
+  sender:
+    type: string
+    description: The user ID of the event sender.
+  recipient:
+    type: string
+    description: The user ID of the intended event recipient.
+  recipient_keys:
+    description: The recipient's signing keys of the encrypted event.
+    $ref: "#/components/schemas/SigningKeys"
+  keys:
+    $ref: "#/components/schemas/SigningKeys"
+    description: The sender's signing keys of the encrypted event.
+  sender_device_keys:
+    $ref: device_keys.yaml 
+    description: The sender's device keys.
+    x-addedInMatrixVersion: "1.15"
+required:
+  - type
+  - content
+  - sender
+  - recipient
+  - recipient_keys
+  - keys
+components:
+  schemas:
+    SigningKeys:
+      type: object
+      title: SigningKeys
+      description: Public keys used for an `m.olm.v1.curve25519-aes-sha2` event.
+      properties:
+        ed25519:
+          type: string
+          description: The Ed25519 public key encoded using unpadded base64.
+      required:
+        - ed25519
+example: {
+  "type": "<type of the plaintext event>",
+  "content": "<content for the plaintext event>",
+  "sender": "<sender_user_id>",
+  "recipient": "<recipient_user_id>",
+  "recipient_keys": {
+    "ed25519": "<our_ed25519_key>"
+  },
+  "keys": {
+    "ed25519": "<sender_ed25519_key>"
+  },
+  "sender_device_keys": {
+    "algorithms": ["<supported>", "<algorithms>"],
+    "user_id": "<user_id>",
+    "device_id": "<device_id>",
+    "keys": {
+      "ed25519:<device_id>": "<sender_ed25519_key>",
+      "curve25519:<device_id>": "<sender_curve25519_key>"
+    },
+    "signatures": {
+      "<user_id>": {
+        "ed25519:<device_id>": "<device_signature>",
+        "ed25519:<ssk_id>": "<ssk_signature>",
+      }
+    }
+  }
+}

data/api/client-server/profile.yaml

@@ -16,48 +16,105 @@ info:
   title: Matrix Client-Server Profile API
   version: 1.0.0
 paths:
-  "/profile/{userId}/displayname":
+  "/profile/{userId}/{keyName}":
     put:
-      summary: Set the user's display name.
+      x-changedInMatrixVersion:
+        "1.14": Endpoint now accepts variable `keyName` parameter.
+      summary: Set a profile field for a user.
       description: |-
-        This API sets the given user's display name. You must have permission to
-        set this user's display name, e.g. you need to have their `access_token`.
-      operationId: setDisplayName
+        Set or update a profile field for a user. Must be authenticated with an
+        access token authorised to make changes. Servers may impose size limits
+        on individual fields, and the total profile must be under 64 KiB.
+
+        **Note**: Setting a field to `null` keeps the key but with a `null` value,
+        which some servers may reject. To remove a field completely, use the
+        `DELETE` endpoint instead.
+      operationId: setProfileField
       security:
         - accessTokenQuery: []
         - accessTokenBearer: []
       parameters:
         - in: path
           name: userId
-          description: The user whose display name to set.
+          description: The user whose profile field to set.
           required: true
           example: "@alice:example.com"
           schema:
             type: string
+        - in: path
+          name: keyName
+          description: The profile field key name to set. It must be either
+            `avatar_url`, `displayname`, or a custom field following the
+            [Common Namespaced Identifier Grammar](/appendices/#common-namespaced-identifier-grammar).
+          required: true
+          example: "displayname"
+          schema:
+            type: string
+            pattern: '^(avatar_url|displayname|[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)+)$'
       requestBody:
+        description: A JSON object containing the property whose name matches
+          the `keyName` specified in the URL. See `additionalProperties` for
+          further details.
+        required: true
         content:
           application/json:
             schema:
               type: object
-              example: {
-                "displayname": "Alice Margatroid"
-              }
-              properties:
-                displayname:
-                  type: string
-                  description: The new display name for this user.
-        description: The new display name information.
-        required: true
+              minProperties: 1
+              additionalProperties:
+                description: The JSON object must include a property whose key
+                  matches the `keyName` specified in the URL. For `avatar_url`,
+                  the value must be an MXC URI string. For `displayname`, the value
+                  must be a string. For custom keys, any JSON type is allowed -
+                  servers may not validate these values, but clients should follow
+                  the format defined for that key.
+              example: { "displayname": "Alice Wonderland" }
       responses:
         "200":
-          description: The display name was set.
+          description: The profile field was set.
           content:
             application/json:
               schema:
-                type: object  # empty json object
+                type: object # empty JSON object
               examples:
                 response:
                   value: {}
+        "400":
+          description: The request is malformed, contains invalid JSON, missing
+            a required parameter, specifies an invalid key, or exceeds allowed
+            size limits.
+          content:
+            application/json:
+              schema:
+                $ref: definitions/errors/error.yaml
+              examples:
+                bad_json:
+                  value:
+                    {
+                      "errcode": "M_BAD_JSON",
+                      "error": "Malformed JSON payload.",
+                    }
+                invalid_key:
+                  value:
+                    {
+                      "errcode": "M_INVALID_PARAM",
+                      "error": "Invalid profile key.",
+                    }
+        "403":
+          description: The server is unwilling to perform the operation, either
+            due to insufficient permissions or because profile modifications
+            are disabled.
+          content:
+            application/json:
+              schema:
+                $ref: definitions/errors/error.yaml
+              examples:
+                forbidden:
+                  value:
+                    {
+                      "errcode": "M_FORBIDDEN",
+                      "error": "Profile modification is not permitted.",
+                    }
         "429":
           description: This request was rate-limited.
           content:
@@ -67,98 +124,133 @@ paths:
       tags:
         - User data
     get:
-      summary: Get the user's display name.
-      description: |-
-        Get the user's display name. This API may be used to fetch the user's
-        own displayname or to query the name of other users; either locally or
-        on remote homeservers.
-      operationId: getDisplayName
+      x-changedInMatrixVersion:
+        "1.14": Endpoint now accepts variable `keyName` parameter.
+      summary: Get a profile field for a user.
+      description: Get the value of a profile field for a user. Any individual
+        field must be within the total profile limit of 64 KiB.
+      operationId: getProfileField
       parameters:
         - in: path
           name: userId
-          description: The user whose display name to get.
+          description: The user whose profile field to get.
           required: true
           example: "@alice:example.com"
           schema:
             type: string
+        - in: path
+          name: keyName
+          description: The profile field key name to retrieve. It must be either
+            `avatar_url`, `displayname`, or a custom field following the
+            [Common Namespaced Identifier Grammar](/appendices/#common-namespaced-identifier-grammar).
+          required: true
+          example: "displayname"
+          schema:
+            type: string
+            pattern: '^(avatar_url|displayname|[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)+)$'
       responses:
         "200":
-          description: The display name for this user.
+          description: The profile field value was retrieved.
           content:
             application/json:
               schema:
                 type: object
-                properties:
-                  displayname:
-                    type: string
-                    description: The user's display name if they have set one, otherwise not
-                      present.
+                minProperties: 1
+                additionalProperties:
+                  description: The JSON response includes a property whose key
+                    matches the `keyName` specified in the URL. For `avatar_url`,
+                    the value will be an MXC URI string. For `displayname`, the
+                    value will be a string. For custom keys, any JSON type is
+                    possible - clients should expect the format defined for that key.
               examples:
                 response:
-                  value: {
-                    "displayname": "Alice Margatroid"
-                  }
+                  value: { "displayname": "Alice" }
         "403":
           x-addedInMatrixVersion: "1.12"
-          description: The server is unwilling to disclose whether the user exists and/or
-            has a display name.
+          description: The server is unwilling to disclose whether the user
+            exists and/or has the specified profile field.
           content:
             application/json:
               schema:
                 $ref: definitions/errors/error.yaml
               examples:
                 response:
-                  value: {
-                    "errcode": "M_FORBIDDEN",
-                    "error": "Profile lookup is disabled on this homeserver"
-                  }
+                  value:
+                    {
+                      "errcode": "M_FORBIDDEN",
+                      "error": "Profile lookup is disabled on this homeserver",
+                    }
         "404":
-          description: There is no display name for this user or this user does not exist.
+          description: There is no profile field with this key for this user, or
+            the user does not exist.
       tags:
         - User data
-  "/profile/{userId}/avatar_url":
-    put:
-      summary: Set the user's avatar URL.
-      description: |-
-        This API sets the given user's avatar URL. You must have permission to
-        set this user's avatar URL, e.g. you need to have their `access_token`.
-      operationId: setAvatarUrl
+    delete:
+      x-addedInMatrixVersion: "1.14"
+      summary: Remove a profile field from a user.
+      description: Remove a specific field from a user's profile.
+      operationId: deleteProfileField
       security:
         - accessTokenQuery: []
         - accessTokenBearer: []
       parameters:
         - in: path
           name: userId
-          description: The user whose avatar URL to set.
+          description: The user whose profile field to delete.
           required: true
           example: "@alice:example.com"
           schema:
             type: string
-      requestBody:
-        content:
-          application/json:
-            schema:
-              type: object
-              example: {
-                "avatar_url": "mxc://matrix.org/wefh34uihSDRGhw34"
-              }
-              properties:
-                avatar_url:
-                  type: string
-                  format: uri
-                  description: The new avatar URL for this user.
-        description: The new avatar information.
-        required: true
+        - in: path
+          name: keyName
+          description: The profile field key name to delete. It must be either
+            `avatar_url`, `displayname`, or a custom field following the
+            [Common Namespaced Identifier Grammar](/appendices/#common-namespaced-identifier-grammar).
+          required: true
+          example: "displayname"
+          schema:
+            type: string
+            pattern: '^(avatar_url|displayname|[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)+)$'
       responses:
         "200":
-          description: The avatar URL was set.
+          description: The profile field was deleted.
           content:
             application/json:
               schema:
-                type: object  # empty json object
+                type: object
               examples:
                 response:
                   value: {}
+        "400":
+          description: The request is malformed, contains invalid JSON, or
+            specifies an invalid key.
+          content:
+            application/json:
+              schema:
+                $ref: definitions/errors/error.yaml
+              examples:
+                bad_json:
+                  value:
+                    { "errcode": "M_BAD_JSON", "error": "Malformed request." }
+                invalid_key:
+                  value:
+                    {
+                      "errcode": "M_INVALID_PARAM",
+                      "error": "Invalid profile key.",
+                    }
+        "403":
+          description: The user is not authorised to delete this profile field.
+          content:
+            application/json:
+              schema:
+                $ref: definitions/errors/error.yaml
+              examples:
+                forbidden:
+                  value:
+                    {
+                      "errcode": "M_FORBIDDEN",
+                      "error": "Profile deletion is not permitted.",
+                    }
         "429":
           description: This request was rate-limited.
           content:
@@ -167,63 +259,15 @@ paths:
                 $ref: definitions/errors/rate_limited.yaml
       tags:
         - User data
-    get:
-      summary: Get the user's avatar URL.
-      description: |-
-        Get the user's avatar URL. This API may be used to fetch the user's
-        own avatar URL or to query the URL of other users; either locally or
-        on remote homeservers.
-      operationId: getAvatarUrl
-      parameters:
-        - in: path
-          name: userId
-          description: The user whose avatar URL to get.
-          required: true
-          example: "@alice:example.com"
-          schema:
-            type: string
-      responses:
-        "200":
-          description: The avatar URL for this user.
-          content:
-            application/json:
-              schema:
-                type: object
-                properties:
-                  avatar_url:
-                    type: string
-                    format: uri
-                    description: The user's avatar URL if they have set one, otherwise not present.
-              examples:
-                response:
-                  value: {
-                    "avatar_url": "mxc://matrix.org/SDGdghriugerRg"
-                  }
-        "403":
-          x-addedInMatrixVersion: "1.12"
-          description: The server is unwilling to disclose whether the user exists and/or
-            has an avatar URL.
-          content:
-            application/json:
-              schema:
-                $ref: definitions/errors/error.yaml
-              examples:
-                response:
-                  value: {
-                    "errcode": "M_FORBIDDEN",
-                    "error": "Profile lookup is disabled on this homeserver"
-                  }
-        "404":
-          description: There is no avatar URL for this user or this user does not exist.
-      tags:
-        - User data
   "/profile/{userId}":
     get:
-      summary: Get this user's profile information.
+      summary: Get all profile information for a user.
       description: |-
-        Get the combined profile information for this user. This API may be used
-        to fetch the user's own profile information or other users; either
-        locally or on remote homeservers.
+        Get the complete profile for a user. The response includes `avatar_url`
+        and `displayname` (unless set to `null`, as they can only be strings)
+        plus any custom profile fields.
+
+        **Note**: The complete profile must be under 64 KiB.
       operationId: getUserProfile
       parameters:
         - in: path
@@ -243,45 +287,49 @@ paths:
                 properties:
                   avatar_url:
                     type: string
-                    format: uri
-                    description: The user's avatar URL if they have set one, otherwise not present.
+                    format: mx-mxc-uri
+                    description: "Avatar URL value (MXC URI format)."
                   displayname:
                     type: string
-                    description: The user's display name if they have set one, otherwise not
-                      present.
+                additionalProperties:
+                  x-addedInMatrixVersion: "1.14"
+                  description: Any additional profile field value; may be any
+                    valid JSON type, with keys following the
+                    [Common Namespaced Identifier Grammar](/appendices/#common-namespaced-identifier-grammar).
               examples:
                 response:
-                  value: {
-                    "avatar_url": "mxc://matrix.org/SDGdghriugerRg",
-                    "displayname": "Alice Margatroid"
-                  }
+                  value:
+                    {
+                      "avatar_url": "mxc://matrix.org/SDGdghriugerRg",
+                      "displayname": "Alice Margatroid",
+                      "m.example_field": "custom_value",
+                    }
         "403":
           x-addedInMatrixVersion: "1.2"
-          description: The server is unwilling to disclose whether the user exists and/or
-            has profile information.
+          description: The server is unwilling to disclose whether the user
+            exists and/or has profile information.
           content:
             application/json:
               schema:
                 $ref: definitions/errors/error.yaml
               examples:
                 response:
-                  value: {
-                    "errcode": "M_FORBIDDEN",
-                    "error": "Profile lookup is disabled on this homeserver"
-                  }
+                  value:
+                    {
+                      "errcode": "M_FORBIDDEN",
+                      "error": "Profile lookup is disabled on this homeserver",
+                    }
         "404":
-          description: There is no profile information for this user or this user does not
-            exist.
+          description: There is no profile information for this user or this
+            user does not exist.
           content:
             application/json:
               schema:
                 $ref: definitions/errors/error.yaml
               examples:
                 response:
-                  value: {
-                    "errcode": "M_NOT_FOUND",
-                    "error": "Profile not found"
-                  }
+                  value:
+                    { "errcode": "M_NOT_FOUND", "error": "Profile not found" }
       tags:
         - User data
 servers: