appendicies: update canonical JSON grammar (#2368)

Signed-off-by: famfo <famfo@famfo.xyz>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

changelogs/appendices/newsfragments/2368.clarification

@@ -0,0 +1,2 @@
+Update the canonical JSON grammar with case sensitive strings and ABNF builtins
+to be easier to understand.

changelogs/client_server/newsfragments/2370.clarification

@@ -0,0 +1 @@
+Clarify formats of string types.

content/appendices.md

@@ -131,32 +131,37 @@ def canonical_json(value):
 
 #### Grammar
 
-Adapted from the grammar in <http://tools.ietf.org/html/rfc7159>
+Adapted grammar from <http://tools.ietf.org/html/rfc7159> removing
-removing insignificant whitespace, fractions, exponents and redundant
+insignificant whitespace, fractions, exponents and redundant character escapes
-character escapes.
+written in [ABNF](https://datatracker.ietf.org/doc/html/rfc5234) with
+[case sensitive strings](https://datatracker.ietf.org/doc/html/rfc7405).
 
+```
 value     = false / null / true / object / array / number / string
-    false     = %x66.61.6C.73.65
+false     = %s"false"
-    null      = %x6E.75.6C.6C
+null      = %s"null"
-    true      = %x74.72.75.65
+true      = %s"true"
-    object    = %x7B [ member *( %x2C member ) ] %x7D
+object    = "{" [ member *( "," member ) ] "}"
-    member    = string %x3A value
+member    = string ":" value
-    array     = %x5B [ value *( %x2C value ) ] %x5D
+array     = "[" [ value *( "," value ) ] "]"
-    number    = [ %x2D ] int
+number    = [ "-" ] int
-    int       = %x30 / ( %x31-39 *digit )
+int       = %x30 / ( %x31-39 *DIGIT )   ; Integer without leading zeros
-    digit     = %x30-39
+string    = DQUOTE *char DQUOTE         ; Quoted characters
-    string    = %x22 *char %x22
+char      = unescaped / "\" escaped
-    char      = unescaped / %x5C escaped
+unescaped = %x20-21 / %x23-5B / %x5D-10FFFF ; All UTF-8 codepoints except ASCII control
-    unescaped = %x20-21 / %x23-5B / %x5D-10FFFF
+                                            ; characters, " and \
-    escaped   = %x22 ; "    quotation mark  U+0022
+escaped   = %x62 ; b    backspace       U+0008
-              / %x5C ; \    reverse solidus U+005C
-              / %x62 ; b    backspace       U+0008
-              / %x66 ; f    form feed       U+000C
-              / %x6E ; n    line feed       U+000A
-              / %x72 ; r    carriage return U+000D
           / %x74 ; t    tab             U+0009
-              / %x75.30.30.30 (%x30-37 / %x62 / %x65-66) ; u000X
+          / %x6E ; n    line feed       U+000A
-              / %x75.30.30.31 (%x30-39 / %x61-66)        ; u001X
+          / %x66 ; f    form feed       U+000C
+          / %x72 ; r    carriage return U+000D
+          / %x22 ; "    quotation mark  U+0022
+          / %x5C ; \    reverse solidus U+005C
+          / %s"u000" (%x30-37 / %x62 / %x65-66) ; All ASCII control characters which do not have
+                                                ; dedicated escape sequences (for example \n).
+                                                ; u000X, where X is [0-7, b, e, f]
+          / %s"u001" (%x30-39 / %x61-66)        ; u001X, where X is [0-9, a-f]
+```
 
 #### Examples
 

content/client-server-api/modules/instant_messaging.md

@@ -197,7 +197,7 @@ To ensure this is done consistently across clients, clients SHOULD use
 the following algorithm to calculate a disambiguated display name for a
 given user:
 
-1.  Inspect the `m.room.member` state event for the relevant user id.
+1.  Inspect the [`m.room.member`](#mroommember) state event for the relevant user id.
 2.  If the `m.room.member` state event has no `displayname` field, or if
     that field has a `null` value, use the raw user id as the display
     name. Otherwise:

data/event-schemas/schema/core-event-schema/msgtype_infos/avatar_info.yaml

@@ -22,6 +22,8 @@ properties:
     description: |-
       The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris)) to a thumbnail of the image.
     type: string
+    format: mx-mxc-uri
+    pattern: "^mxc:\\/\\/"
   thumbnail_info:
     allOf:
       - $ref: thumbnail_info.yaml

data/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml

@@ -23,6 +23,8 @@ properties:
       The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris)) to a thumbnail of the image.
       Only present if the thumbnail is unencrypted.
     type: string
+    format: mx-mxc-uri
+    pattern: "^mxc:\\/\\/"
   thumbnail_file:
     description: |-
       Information on the encrypted thumbnail file, as specified in

data/event-schemas/schema/m.room.avatar.yaml

@@ -16,6 +16,8 @@ properties:
           The URL to the image. If this property is not present, the room has no avatar. This can be useful
           to remove a previous room avatar.
         type: string
+        format: mx-mxc-uri
+        pattern: "^mxc:\\/\\/"
     type: object
   state_key:
     description: A zero-length string.

data/event-schemas/schema/m.room.message$m.audio.yaml

@@ -54,6 +54,8 @@ properties:
           Required if the file is unencrypted. The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris))
           to the audio clip.
         type: string
+        format: mx-mxc-uri
+        pattern: "^mxc:\\/\\/"
       file:
         description: |-
           Required if the file is encrypted. Information on the encrypted

data/event-schemas/schema/m.room.message$m.file.yaml

@@ -44,6 +44,8 @@ properties:
               The URL to the thumbnail of the file. Only present if the
               thumbnail is unencrypted.
             type: string
+            format: mx-mxc-uri
+            pattern: "^mxc:\\/\\/"
           thumbnail_file:
             description: |-
               Information on the encrypted thumbnail file, as specified in
@@ -66,6 +68,8 @@ properties:
           Required if the file is unencrypted. The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris))
           to the file.
         type: string
+        format: mx-mxc-uri
+        pattern: "^mxc:\\/\\/"
       file:
         description: |-
           Required if the file is encrypted. Information on the encrypted

data/event-schemas/schema/m.room.message$m.image.yaml

@@ -44,6 +44,8 @@ properties:
           Required if the file is unencrypted. The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris))
           to the image.
         type: string
+        format: mx-mxc-uri
+        pattern: "^mxc:\\/\\/"
       file:
         description: |-
           Required if the file is encrypted. Information on the encrypted

data/event-schemas/schema/m.room.message$m.location.yaml

@@ -13,6 +13,8 @@ properties:
       geo_uri:
         description: A [geo URI (RFC5870)](https://datatracker.ietf.org/doc/html/rfc5870) representing this location.
         type: string
+        format: uri
+        pattern: "^geo:"
       msgtype:
         enum:
           - m.location
@@ -25,6 +27,8 @@ properties:
               The URL to a thumbnail of the location being represented.
               Only present if the thumbnail is unencrypted.
             type: string
+            format: mx-mxc-uri
+            pattern: "^mxc:\\/\\/"
           thumbnail_file:
             description: |-
               Information on the encrypted thumbnail file, as specified in

data/event-schemas/schema/m.room.message$m.video.yaml

@@ -54,6 +54,8 @@ properties:
               The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris)) to an image thumbnail of
               the video clip. Only present if the thumbnail is unencrypted.
             type: string
+            format: mx-mxc-uri
+            pattern: "^mxc:\\/\\/"
           thumbnail_file:
             description: |-
               Information on the encrypted thumbnail file, as specified in
@@ -76,6 +78,8 @@ properties:
           Required if the file is unencrypted. The URL (typically [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris))
           to the video clip.
         type: string
+        format: mx-mxc-uri
+        pattern: "^mxc:\\/\\/"
       file:
         description: |-
           Required if the file is encrypted. Information on the encrypted

data/event-schemas/schema/m.room.name.yaml

@@ -4,16 +4,19 @@ $schema: https://json-schema.org/draft/2020-12/schema
 allOf:
   - $ref: core-event-schema/state_event.yaml
 description: |-
-    A room has an opaque room ID which is not human-friendly to read. A room
+    A room has an opaque [room ID](/appendices#room-ids) which is not
-    alias is human-friendly, but not all rooms have room aliases. The room name
+    human-friendly to read. A [room alias](appendices#room-aliases) is
-    is a human-friendly string designed to be displayed to the end-user. The
+    human-friendly, but not all rooms have [room aliases](/client-server-api/#room-aliases).
-    room name is not unique, as multiple rooms can have the same room name set.
+    The room name is a human-friendly string designed to be displayed to the
+    end-user. The room name is not unique, as multiple rooms can have the same
+    room name set.
 
     If a room has an `m.room.name` event with an absent, null, or empty `name`
     field, it should be treated the same as a room with no `m.room.name` event.
 
     An event of this type is automatically created when creating a room using
-    `/createRoom` with the `name` key.
+    [`/createRoom`](/client-server-api/#post_matrixclientv3createroom) with the
+    `name` key.
 properties:
   content:
     properties:

data/event-schemas/schema/m.room.pinned_events.yaml

@@ -11,6 +11,8 @@ properties:
         description: An ordered list of event IDs to pin.
         items:
           type: string
+          format: mx-event-id
+          pattern: "^\\$"
         type: array
     required:
       - pinned