Spec for MSC4376: Remove /v1/send_join and /v1/send_leave (#2319)

Signed-off-by: Kierre Sametti vel@riseup.net

changelogs/client_server/newsfragments/2301.feature

@@ -0,0 +1 @@
+Add recommendation about excluding non-cross-signed devices from encrypted conversations, as per [MSC4153](https://github.com/matrix-org/matrix-spec-proposals/pull/4153).

changelogs/client_server/newsfragments/2316.clarification

@@ -0,0 +1 @@
+Add 404 responses to the OpenAPI of `GET /login` and `GET /auth_metadata` endpoints. The responses were already defined in text but not written in OpenAPI.

changelogs/room_versions/newsfragments/2297.clarification

@@ -0,0 +1 @@
+Clarify meaning of floating-point powerlevels.

changelogs/server_server/newsfragments/2319.removal

@@ -0,0 +1 @@
+Remove `/v1/send_join` and `/v1/send_leave`, as per [MSC4376](https://github.com/matrix-org/matrix-spec-proposals/pull/4376).

content/client-server-api/modules/end_to_end_encryption.md

@@ -5,6 +5,91 @@ Matrix optionally supports end-to-end encryption, allowing rooms to be
 created whose conversation contents are not decryptable or interceptable
 on any of the participating homeservers.
 
+#### Recommended client behaviour
+
+{{% added-in v="1.18" %}}
+
+While clients are able to choose what encryption features they implement based
+on their threat model, this section recommends behaviours that will improve the
+overall user experience and security of encrypted conversations.
+
+While a user may be unable to [verify](#device-verification) every other user
+that they communicate with, or may be unaware of the need to verify other users,
+[cross-signing](#cross-signing) gives some measure of protection and so SHOULD
+be used where possible. In particular, clients SHOULD implement the following
+recommendations.
+
+* Clients SHOULD create new [cross-signing keys](#cross-signing) for users who
+  do not yet have cross-signing keys.
+* Clients SHOULD encourage users to set up their [Secret Storage](#storage) to
+  avoid needing to reset their cryptographic identity in case the user does not
+  have an existing device that can [share the secrets](#sharing) with the new
+  device. The user's Secret Storage SHOULD contain the user's cross-signing
+  private keys and the [key backup](#server-side-key-backups) decryption key
+  (if the user is using key backup). The user's Secret Storage SHOULD have a
+  [default key](#key-storage) (a key referred to by
+  `m.secret_storage.default_key`) that encrypts the private cross-signing keys
+  and key backup decryption key (if available).
+* Clients SHOULD encourage users to [cross-sign](#cross-signing) their devices.
+  This includes both when logging in a new device, and for existing devices.
+  Clients MAY even go so  far as to require cross-signing of devices by
+  preventing the user from using  the client until the device is cross-signed.
+  If the user cannot cross-sign their device (for example, if they have
+  forgotten their Secret Storage key), the client can allow users to reset their
+  [Secret Storage](#storage), cross-signing keys, and [key backup](#server-side-key-backups).
+* When Alice [verifies](#device-verification) Bob, the verification SHOULD
+  verify their [cross-signing keys](#cross-signing). Any flow between different
+  users that does not verify the users' cross-signing keys (it verifies only the
+  device keys) is deprecated.
+* Clients SHOULD flag when [cross-signing keys](#cross-signing) change. If
+  Alice's cross-signing keys change, Alice's own devices MUST alert her to this
+  fact, and prompt her to re-cross-sign those devices. If Bob is in an
+  encrypted room with Alice,  Bob's devices SHOULD inform him of Alice's key
+  change and SHOULD prevent him from sending an encrypted message to Alice
+  without acknowledging the change. Bob's clients may behave differently
+  depending on whether Bob had previously [verified](#device-verification)
+  Alice or not. For example, if Bob had previously verified Alice, and Alice's
+  keys change, Bob's client may require Bob to re-verify, or may display a more
+  aggressive warning.
+* Clients SHOULD NOT send encrypted [to-device](#send-to-device-messaging)
+  messages, such as [room keys](#sharing-keys-between-devices) or [secrets](#secrets)
+  (via [Secret Sharing](#sharing)), to [non-cross-signed](#cross-signing)
+  devices by default. Non-cross-signed devices don't provide any assurance that
+  the device belongs to the user, and server admins can trivially create new
+  devices for users. When sending room keys, clients can use a
+  [`m.room_key.withheld`](#mroom_keywithheld) message with a code of
+  `m.unverified` to indicate to the non-cross-signed device why it is not
+  receiving the room key.
+
+  Note that clients cannot selectively send room events only to cross-signed
+  devices. The only way to exclude non-cross-signed devices from encrypted
+  conversations is to not send the room keys so those devices won't be able to
+  decrypt the messages.
+* Similarly, messages sent from [non-cross-signed](#cross-signing) devices
+  cannot be trusted and SHOULD NOT be displayed to the user. Clients have no
+  assurance that encrypted messages sent from non-cross-signed devices were sent
+  by the user, rather than an impersonator.
+* Matrix clients MUST NOT consider non-cryptographic devices (devices which do
+  not have [device identity keys](#device-keys) uploaded to the homeserver) to
+  be equivalent to [non-cross-signed](#cross-signing) cryptographic devices for
+  purposes of enforcing E2EE policy. For example, clients SHOULD NOT warn nor
+  refuse to send messages due to the presence of non-cryptographic devices. For
+  all intents and purposes, non-cryptographic devices are a completely separate
+  concept and do not exist from the perspective of the cryptography layer since
+  they do not have identity keys, so it is impossible to send them decryption
+  keys.
+* Clients MAY make provisions for encrypted bridges. Some bridges are structured
+  in a way such that only one user controlled by the bridge (often called the
+  bridge bot) participates in encryption, and encrypted messages from other
+  bridge users are encrypted by the bridge bot. Thus encrypted messages sent by
+  one user could be encrypted by a [Megolm](#mmegolmv1aes-sha2) session sent by
+  a different user. Clients MAY accept such messages, provided the session
+  creator's device is [cross-signed](#cross-signing). However, the client MUST
+  annotate the message with a warning, unless the client has a way to check that
+  the bridge bot is permitted to encrypt messages on behalf of the user. Future
+  MSCs such as [MSC4350](https://github.com/matrix-org/matrix-spec-proposals/pull/4350)
+  may provide a secure way to allow such impersonation.
+
 #### Key Distribution
 
 Encryption and Authentication in Matrix is based around public-key
@@ -674,8 +759,11 @@ The process between Alice and Bob verifying each other would be:
     their devices if they match or not.
 15. Assuming they match, Alice and Bob's devices each calculate Message
     Authentication Codes (MACs) for:
-    * Each of the keys that they wish the other user to verify (usually their
-      device ed25519 key and their master signing key, see below).
+    * {{% changed-in v="1.18" %}} Each of the keys that they wish the other user
+      to verify (usually their device ed25519 key and their master signing key, 
+      see below). The master signing key SHOULD be included when two different
+      users are verifying each other. Verifying individual devices of other
+      users is deprecated.
     * The complete list of key IDs that they wish the other user to verify.
 
     The MAC calculation is defined [below](#mac-calculation).

content/rooms/fragments/v1-floaty-power-levels.md

@@ -0,0 +1,41 @@
+
+##### `m.room.power_levels` events accept values as floats
+
+When the value is a float
+ * First, exponential notation is applied: `5.114698E4` becomes `51146.98`
+ * Second, the value is truncated at the decimal point: `51146.98` becomes `51146`.
+
+Values outside the range represented by IEE754 binary64 (a "double") cause the
+powerlevel event to be rejected, as do `Infinity`, `-Infinity` and `NaN`.
+
+For example, this is a valid `m.room.power_levels` event in this room version:
+
+```json
+{
+  "content": {
+    "ban": 50,
+    "events": {
+      "m.room.power_levels": 100
+    },
+    "events_default": 0,
+    "state_default": 50,
+    "users": {
+      "@example:example.org": 100,
+      "@alice:localhost": 50,
+      "@bob:localhost": 50.57
+    },
+    "users_default": 0
+  },
+  "origin_server_ts": 1432735824653,
+  "room_id": "!jEsUZKDJdhlrceRyVU:example.org",
+  "sender": "@example:example.org",
+  "state_key": "",
+  "type": "m.room.power_levels"
+}
+```
+
+In this example, both `@bob:localhost` and `@alice:localhost` have the same effective
+power level of `50`, even though the values are technically different.
+
+Note that, since this room version does not enforce that events comply with the requirements
+of [Canonical JSON](/appendices#canonical-json), power levels can be formatted as floats.

content/rooms/v1.md

@@ -59,6 +59,8 @@ Events in version 1 rooms have the following structure:
 
 {{% rver-fragment name="v1-stringy-power-levels" %}}
 
+{{% rver-fragment name="v1-floaty-power-levels" %}}
+
 ### Authorization rules
 
 {{% rver-fragment name="v1-auth-rules" %}}

content/rooms/v2.md

@@ -57,6 +57,8 @@ Events in rooms of this version have the following structure:
 
 {{% rver-fragment name="v1-stringy-power-levels" %}}
 
+{{% rver-fragment name="v1-floaty-power-levels" %}}
+
 ### Authorization rules
 
 {{% rver-fragment name="v1-auth-rules" %}}

content/rooms/v3.md

@@ -87,6 +87,8 @@ The complete structure of a event in a v3 room is shown below.
 
 {{% rver-fragment name="v1-stringy-power-levels" %}}
 
+{{% rver-fragment name="v1-floaty-power-levels" %}}
+
 ### Authorization rules
 
 {{% boxes/note %}}

content/rooms/v4.md

@@ -76,6 +76,8 @@ the changes in this room version.
 
 {{% rver-fragment name="v1-stringy-power-levels" %}}
 
+{{% rver-fragment name="v1-floaty-power-levels" %}}
+
 ### Authorization rules
 
 {{% rver-fragment name="v3-auth-rules" %}}

content/rooms/v5.md

@@ -58,6 +58,8 @@ completeness.
 
 {{% rver-fragment name="v1-stringy-power-levels" %}}
 
+{{% rver-fragment name="v1-floaty-power-levels" %}}
+
 ### Authorization rules
 
 {{% rver-fragment name="v3-auth-rules" %}}

content/rooms/v6.md

@@ -42,7 +42,8 @@ in [room version 5](/rooms/v5).
 ### Event format
 
 {{% added-in v=6 %}} Through enforcement of [Canonical JSON](#canonical-json),
-the `depth` limit has been reduced in this room version.
+the `depth` limit has been reduced in this room version, and numeric values may
+no longer be formatted as floats.
 
 {{% rver-fragment name="v6-event-format" %}}
 

data/api/client-server/login.yaml

@@ -70,6 +70,21 @@ paths:
                       }
                     ]
                   }
+        "404":
+          description: |-
+            With `M_UNRECOGNIZED`: the homeserver does not support the legacy authentication API.
+            (See [Authentication API discovery](/client-server-api/#authentication-api-discovery).)
+          content:
+            application/json:
+              schema:
+                $ref: definitions/errors/error.yaml
+              examples:
+                response:
+                  value:
+                    {
+                      "errcode": "M_UNRECOGNIZED",
+                      "error": "OAuth 2.0 authentication is in use on this homeserver.",
+                    }
         "429":
           description: This request was rate-limited.
           content:

data/api/client-server/oauth_server_metadata.yaml

@@ -195,6 +195,21 @@ paths:
                     "org.matrix.cross_signing_reset",
                   ],
                 }
+        "404":
+          description: |-
+            With `M_UNRECOGNIZED`: the homeserver does not support the OAuth 2.0 API.
+            (See [Authentication API discovery](/client-server-api/#authentication-api-discovery).)
+          content:
+            application/json:
+              schema:
+                $ref: definitions/errors/error.yaml
+              examples:
+                response:
+                  value:
+                    {
+                      "errcode": "M_UNRECOGNIZED",
+                      "error": "Legacy authentication is in use on this homeserver.",
+                    }
       tags:
         - Session management
 servers:

data/api/server-server/joins-v1.yaml

@@ -234,207 +234,6 @@ paths:
                     "errcode": "M_NOT_FOUND",
                     "error": "Unknown room"
                   }
-  "/send_join/{roomId}/{eventId}":
-    put:
-      deprecated: true
-      summary: Submit a signed join event to a resident server
-      description: |-
-        **Note:**
-        Servers should instead prefer to use the v2 `/send_join` endpoint.
-
-        Submits a signed join event to the resident server for it
-        to accept it into the room's graph. Note that events have
-        a different format depending on the room version - check
-        the [room version specification](/rooms) for precise event formats.
-        **The request and response body here describe the common
-        event fields in more detail and may be missing other required
-        fields for a PDU.**
-
-        The receiving server MUST apply certain validation before accepting the event.
-        A specific error code is not mandated, but servers SHOULD return `M_INVALID_PARAM` if:
-
-        * The join event fails a [signature check](/server-server-api/#validating-hashes-and-signatures-on-received-events).
-        * The event type is not `m.room.member`.
-        * The `membership` field inside the event content is not `join`.
-        * The event sender is not a user ID on the origin server.
-        * The `state_key` is not equal to the `sender`.
-      operationId: sendJoinV1
-      security:
-        - signedRequest: []
-      parameters:
-        - in: path
-          name: roomId
-          description: The room ID that is about to be joined.
-          required: true
-          example: "!abc123:matrix.org"
-          schema:
-            type: string
-        - in: path
-          name: eventId
-          description: The event ID for the join event.
-          required: true
-          example: $abc123:example.org
-          schema:
-            type: string
-      requestBody:
-        content:
-          application/json:
-            schema:
-              type: object
-              properties:
-                sender:
-                  type: string
-                  description: The user ID of the joining member.
-                  example: "@someone:example.org"
-                origin:
-                  type: string
-                  description: The name of the joining homeserver.
-                  example: matrix.org
-                origin_server_ts:
-                  type: integer
-                  format: int64
-                  description: A timestamp added by the joining homeserver.
-                  example: 1234567890
-                type:
-                  type: string
-                  description: The value `m.room.member`.
-                  example: m.room.member
-                state_key:
-                  type: string
-                  description: The user ID of the joining member.
-                  example: "@someone:example.org"
-                content:
-                  type: object
-                  title: Membership Event Content
-                  description: The content of the event.
-                  example:
-                    membership: join
-                  properties:
-                    membership:
-                      type: string
-                      description: The value `join`.
-                      example: join
-                    join_authorised_via_users_server:
-                      type: string
-                      x-addedInMatrixVersion: "1.2"
-                      description: |-
-                        Required if the room is [restricted](/client-server-api/#restricted-rooms)
-                        and is joining through one of the conditions available. If the
-                        user is responding to an invite, this is not required.
-
-                        An arbitrary user ID belonging to the resident server in
-                        the room being joined that is able to issue invites to other
-                        users. This is used in later validation of the auth rules for
-                        the `m.room.member` event.
-
-                        The resident server which owns the provided user ID must have a
-                        valid signature on the event. If the resident server is receiving
-                        the `/send_join` request, the signature must be added before sending
-                        or persisting the event to other servers.
-                  required:
-                    - membership
-              required:
-                - state_key
-                - sender
-                - origin
-                - origin_server_ts
-                - type
-                - content
-        required: true
-      responses:
-        "200":
-          description: The join event has been accepted into the room.
-          content:
-            application/json:
-              schema:
-                type: array
-                minItems: 2
-                maxItems: 2
-                items:
-                  anyOf:
-                    - type: integer
-                      description: The value `200`.
-                      example: 200
-                    - type: object
-                      title: Room State
-                      description: The state for the room.
-                      properties:
-                        auth_chain:
-                          type: array
-                          description: |-
-                            The auth chain for the entire current room state prior to the join event.
-
-                            Note that events have a different format depending on the room version - check the
-                            [room version specification](/rooms) for precise event formats.
-                          items:
-                            type: object
-                            title: PDU
-                            description: |-
-                              The [PDUs](/server-server-api/#pdus) that make up the auth chain. The event format varies depending
-                              on the room version - check the [room version specification](/rooms) for precise event formats.
-                        state:
-                          type: array
-                          description: |-
-                            The resolved current room state prior to the join event.
-
-                            The event format varies depending on the room version - check the [room version specification](/rooms)
-                            for precise event formats.
-                          items:
-                            type: object
-                            title: PDU
-                            description: |-
-                              The [PDUs](/server-server-api/#pdus) for the fully resolved state of the room. The event format varies depending
-                              on the room version - check the [room version specification](/rooms) for precise event formats.
-                      required:
-                        - auth_chain
-                        - state
-              examples:
-                response:
-                  value: [
-                    200,
-                    {
-                      "auth_chain": [
-                        {
-                          "$ref": "examples/minimal_pdu.json"
-                        }
-                      ],
-                      "state": [
-                        {
-                          "$ref": "examples/minimal_pdu.json"
-                        }
-                      ],
-                      "event": {
-                        "$ref": "examples/pdu_v4_join_membership.json"
-                      }
-                    }
-                  ]
-        "400":
-          description: |-
-            The request is invalid in some way.
-          content:
-            application/json:
-              schema:
-                $ref: ../client-server/definitions/errors/error.yaml
-              examples:
-                response:
-                  value: {
-                    "errcode": "M_INVALID_PARAM",
-                    "error": "Not a join event."
-                  }
-        "403":
-          description: |-
-            The room that the joining server is attempting to join does not permit the user
-            to join.
-          content:
-            application/json:
-              schema:
-                $ref: ../client-server/definitions/errors/error.yaml
-              examples:
-                response:
-                  value: {
-                    "errcode": "M_FORBIDDEN",
-                    "error": "You are not invited to this room"
-                  }
 servers:
   - url: "{protocol}://{hostname}{basePath}"
     variables:

data/api/server-server/joins-v2.yaml

@@ -22,15 +22,6 @@ paths:
     put:
       summary: Submit a signed join event to a resident server
       description: |-
-        **Note:**
-        This API is nearly identical to the v1 API with the
-        exception of the response format being fixed.
-
-        This endpoint is preferred over the v1 API as it provides
-        a more standardised response format. Senders which receive
-        a 400, 404, or other status code which indicates this endpoint
-        is not available should retry using the v1 API instead.
-
         Submits a signed join event to the resident server for it
         to accept it into the room's graph. Note that events have
         a different format depending on the room version - check

data/api/server-server/leaving-v1.yaml

@@ -149,139 +149,6 @@ paths:
                     "errcode": "M_FORBIDDEN",
                     "error": "User is not in the room."
                   }
-  "/send_leave/{roomId}/{eventId}":
-    put:
-      deprecated: true
-      summary: Submit a signed leave event to a resident server
-      description: |-
-        **Note:**
-        Servers should instead prefer to use the v2 `/send_leave` endpoint.
-
-        Submits a signed leave event to the resident server for it
-        to accept it into the room's graph. Note that events have
-        a different format depending on the room version - check
-        the [room version specification](/rooms) for precise event formats.
-        **The request and response body here describe the common
-        event fields in more detail and may be missing other required
-        fields for a PDU.**
-
-        The receiving server MUST apply certain validation before accepting the event.
-        A specific error code is not mandated, but servers SHOULD return `M_INVALID_PARAM` if:
-
-        * The leave event fails a [signature check](/server-server-api/#validating-hashes-and-signatures-on-received-events).
-        * The event type is not `m.room.member`.
-        * The `membership` field inside the event content is not `leave`.
-        * The event sender is not a user ID on the origin server.
-        * The `state_key` is not equal to the `sender`.
-      operationId: sendLeaveV1
-      security:
-        - signedRequest: []
-      parameters:
-        - in: path
-          name: roomId
-          description: The room ID that is about to be left.
-          required: true
-          example: "!abc123:matrix.org"
-          schema:
-            type: string
-        - in: path
-          name: eventId
-          description: The event ID for the leave event.
-          required: true
-          example: $abc123:example.org
-          schema:
-            type: string
-      requestBody:
-        content:
-          application/json:
-            schema:
-              type: object
-              properties:
-                sender:
-                  type: string
-                  description: The user ID of the leaving member.
-                  example: "@someone:example.org"
-                origin:
-                  type: string
-                  description: The name of the leaving homeserver.
-                  example: matrix.org
-                origin_server_ts:
-                  type: integer
-                  format: int64
-                  description: A timestamp added by the leaving homeserver.
-                  example: 1234567890
-                type:
-                  type: string
-                  description: The value `m.room.member`.
-                  example: m.room.member
-                state_key:
-                  type: string
-                  description: The user ID of the leaving member.
-                  example: "@someone:example.org"
-                content:
-                  type: object
-                  title: Membership Event Content
-                  description: The content of the event.
-                  example:
-                    membership: leave
-                  properties:
-                    membership:
-                      type: string
-                      description: The value `leave`.
-                      example: leave
-                  required:
-                    - membership
-                depth:
-                  type: integer
-                  description: This field must be present but is ignored; it may be 0.
-                  example: 12
-              required:
-                - state_key
-                - sender
-                - origin
-                - origin_server_ts
-                - type
-                - depth
-                - content
-        required: true
-      responses:
-        "200":
-          description: |-
-            An empty response to indicate the event was accepted into the graph by
-            the receiving homeserver.
-          content:
-            application/json:
-              schema:
-                type: array
-                minItems: 2
-                maxItems: 2
-                items:
-                  anyOf:
-                    - type: integer
-                      description: The value `200`.
-                      example: 200
-                    - type: object
-                      title: Empty Object
-                      description: An empty object.
-              examples:
-                response:
-                  value: [
-                    200,
-                    {}
-                  ]
-        "400":
-          description: |-
-            The request is invalid in some way.
-          content:
-            application/json:
-              schema:
-                $ref: ../client-server/definitions/errors/error.yaml
-              examples:
-                response:
-                  value: {
-                    "errcode": "M_INVALID_PARAM",
-                    "error": "Not a leave event."
-                  }
 servers:
   - url: "{protocol}://{hostname}{basePath}"
     variables:

data/api/server-server/leaving-v2.yaml

@@ -22,15 +22,6 @@ paths:
     put:
       summary: Submit a signed leave event to a resident server
       description: |-
-        **Note:**
-        This API is nearly identical to the v1 API with the
-        exception of the response format being fixed.
-
-        This endpoint is preferred over the v1 API as it provides
-        a more standardised response format. Senders which receive
-        a 400, 404, or other status code which indicates this endpoint
-        is not available should retry using the v1 API instead.
-
         Submits a signed leave event to the resident server for it
         to accept it into the room's graph. Note that events have
         a different format depending on the room version - check