diff --git a/changelogs/client_server/newsfragments/2397.feature b/changelogs/client_server/newsfragments/2397.feature new file mode 100644 index 00000000..acef1618 --- /dev/null +++ b/changelogs/client_server/newsfragments/2397.feature @@ -0,0 +1 @@ +Add support for image packs (`m.room.image_pack` and `m.image_pack.rooms`), allowing custom emoticons and stickers to be organised into packs and shared between users, as per [MSC2545](https://github.com/matrix-org/matrix-spec-proposals/pull/2545). diff --git a/content/client-server-api/_index.md b/content/client-server-api/_index.md index ed7e1e9f..84605d5c 100644 --- a/content/client-server-api/_index.md +++ b/content/client-server-api/_index.md @@ -4281,6 +4281,7 @@ that profile. | [Event Replacements](#event-replacements) | Optional | Optional | Optional | Optional | Optional | | [Read and Unread Markers](#read-and-unread-markers) | Optional | Optional | Optional | Optional | Optional | | [Guest Access](#guest-access) | Optional | Optional | Optional | Optional | Optional | +| [Image Packs](#image-packs) | Optional | Optional | Optional | Optional | Optional | | [Moderation Policy Lists](#moderation-policy-lists) | Optional | Optional | Optional | Optional | Optional | | [Policy Servers](#policy-servers) | Optional | Optional | Optional | Optional | Optional | | [OpenID](#openid) | Optional | Optional | Optional | Optional | Optional | @@ -4391,6 +4392,7 @@ systems. {{% cs-module name="Event replacements" filename="event_replacements" %}} {{% cs-module name="Event annotations and reactions" filename="event_annotations" %}} {{% cs-module name="Recently used emoji" filename="recent_emoji" %}} +{{% cs-module name="Image packs" filename="image_packs" %}} {{% cs-module name="Threading" filename="threading" %}} {{% cs-module name="Reference relations" filename="reference_relations" %}} {{% cs-module name="Mutual Rooms" filename="mutual_rooms" %}} diff --git a/content/client-server-api/modules/image_packs.md b/content/client-server-api/modules/image_packs.md new file mode 100644 index 00000000..9d431ddc --- /dev/null +++ b/content/client-server-api/modules/image_packs.md @@ -0,0 +1,205 @@ +### Image packs + +{{% added-in v="1.19" %}} + +Image packs allow users to organise custom emoticons and stickers into named +collections and share them with others. + +An **emoticon** (also called an emote) is a custom image sent inline within a +message, analogous to emoji but defined outside the Unicode standard. A +**sticker** is a standalone image sent as an [`m.sticker`](#msticker) event. +Image packs provide a distribution mechanism for both. + +{{% boxes/note %}} +Emoticons are distinct from the [`m.emote`](#mroommessage-msgtypes) message +type. `m.emote` is used to describe an action (for example, "/me waves +hello"), whereas emoticons are images expressing emotions or other concepts. +{{% /boxes/note %}} + +#### Shortcode grammar + +Each image in a pack is identified by a **shortcode**: a short, human-readable +string used to search for and reference images. Shortcodes are not intended to +serve as accessible descriptions of an image; that purpose is served by the +`body` property of the image object. + +A shortcode MUST match the following grammar: + +``` +shortcode = 1*100shortcode_char +shortcode_char = ALPHA / DIGIT / "-" / "_" +``` + +Where `ALPHA` and `DIGIT` are as defined in +[RFC 5234](https://datatracker.ietf.org/doc/html/rfc5234). Shortcodes are +case-sensitive. The length of a shortcode MUST NOT exceed 100 bytes. + +The `:` character is excluded because it is widely used across messaging +platforms as a delimiter for triggering emote search (for example, typing +`:cat` to search for an emote named `cat`). The `/` character is excluded +because clients MAY use it to separate a shortcode from a pack name in +completion UI (for example, `:cat/my_pack:`). Spaces are excluded to avoid +ambiguity and common usability issues. This character set matches that used +by Discord and Slack, simplifying bridging. + +Homeservers MAY enforce this grammar when `m.room.image_pack` events are +submitted by clients via +[`PUT /_matrix/client/v3/rooms/{roomId}/state/{eventType}/{stateKey}`](/client-server-api/#put_matrixclientv3roomsroomidstateeventtypestatekey). +However, homeservers MUST NOT reject or drop `m.room.image_pack` events +received over federation for failing this grammar. Homeservers MAY locally +soft-fail such events. + +Clients SHOULD render emotes and stickers that have malformed shortcodes, so +that users can identify and correct them. Clients SHOULD enforce this grammar +when creating or editing image packs. + +#### Events + +{{% event event="m.room.image_pack" %}} + +{{% event event="m.image_pack.rooms" %}} + +#### Image properties + +Emoticons SHOULD be at least 128×128 pixels. Stickers SHOULD be at least +512×512 pixels. These minimums ensure that images look sharp on high-DPI +displays. + +Accepted image formats are the same as those permitted for +[`m.image`](#mimage) events. Images MAY be animated; clients MAY pause +animations based on user preferences. + +#### Room image packs + +A room MAY contain any number of image packs, each defined by an +`m.room.image_pack` state event with a distinct `state_key`. Clients SHOULD +present the images in a room's packs only when the user is interacting in +that room. + +#### User image packs + +To make a room's image pack available globally across all rooms, a user adds +a reference to the pack in their `m.image_pack.rooms` account data event. The +reference consists of the room ID and the `state_key` of the pack. + +#### Space image packs + +Clients SHOULD surface image packs defined in the canonical space of the +current room, if the user is also a member of that space. This applies +recursively: if the canonical space itself has a canonical space, the packs +of that space SHOULD also be surfaced. Care SHOULD be taken to avoid cycles +when traversing the space hierarchy, and implementations SHOULD impose a +reasonable limit on the traversal depth. + +#### Image pack source priority + +When presenting image suggestions to the user, clients SHOULD display image +packs in the following order: + +1. Packs referenced in the user's `m.image_pack.rooms` account data. +2. Packs defined in the current room's state. +3. Packs from the canonical space hierarchy of the current room. + +The ordering of images *within* a pack is left to a future specification. + +#### Client behaviour + +##### Sending custom emotes + +Custom emotes are sent as `` elements within the `formatted_body` of an +[`m.room.message`](#mroommessage) event (with `format` set to +`org.matrix.custom.html`). The `data-mx-emoticon` attribute identifies the +element as a custom emote: + +```html +a waving cat +``` + +A client MUST treat an `` element as a custom emote if and only if the +`data-mx-emoticon` attribute is present. If the attribute has a value, that +value MUST be ignored (some HTML serialisers MAY produce +`data-mx-emoticon=""`). + +The attributes of the `` element are defined as follows. + +The `src` attribute MUST be a valid +[`mxc://` URI](/client-server-api/#matrix-content-mxc-uris). Clients MUST +NOT attempt to render images from other URI schemes, as this could result in +unintended network requests. + +The `alt` attribute SHOULD be present and set to the `body` of the image +object, or if absent, the image's shortcode. This attribute provides an +accessible text description of the emote as defined by the +[HTML specification](https://html.spec.whatwg.org/multipage/images.html#alt). + +The `title` attribute SHOULD be present and set to the shortcode of the +emote. Clients MAY display this as a tooltip. + +The `height` attribute MUST be present for backwards compatibility with +clients that do not support custom emotes. Clients SHOULD set this to `32`. +Clients implementing image pack support SHOULD override this value when +rendering based on the user's font size or other environmental factors. The +`width` attribute SHOULD be omitted to preserve the image's aspect ratio. + +Clients MAY render messages that consist entirely of custom emotes at a larger +size. + +##### Sending stickers + +When sending an image from a pack as a sticker, the `body` property of the +[`m.sticker`](#msticker) event SHOULD be set to the image object's `body` +property, or if absent, the image's shortcode. The `info` property of the +`m.sticker` event SHOULD be set to the image object's `info` property, or if +absent, an empty object. + +##### Emote picker suggestions + +Clients MAY use the `:` character as a trigger to initiate emote search. When +multiple packs contain images with the same shortcode, clients SHOULD provide +disambiguation UI rather than silently resolving to one image. Clients MAY +disambiguate by appending a slugified pack display name separated by `/` +(for example, `:cat_wave/my_pack:`). Because pack names are not globally +unique, clients SHOULD NOT attempt to automatically resolve a shortcode to a +specific image. Clients SHOULD instead present a search modal or similar UI +to allow the user to select the intended image. + +#### Security considerations + +##### Encrypted image packs + +Image packs and the images they contain are not encrypted. Media referenced +from image packs is therefore visible to homeservers. Encryption of image +packs depends on encrypted state events, which are not currently defined by +the Matrix specification. + +In addition, a homeserver could correlate the timing of encrypted message +events with media access requests to infer which emote was used in an +encrypted event. This attack is analogous to URL preview correlation and is +not unique to image packs. Client-side caching of media can reduce the +frequency of such requests. + +Clients SHOULD warn users that media referenced from image packs in +end-to-end encrypted rooms is not encrypted and is therefore visible to the +homeserver. + +##### Abusive content + +A user who enables a room image pack globally via `m.image_pack.rooms` +implicitly trusts the pack's administrator not to introduce abusive imagery. +If abusive content is added to a pack, the affected user SHOULD remove the +reference from their `m.image_pack.rooms` account data. + +#### Unstable prefix + +Before this feature was included in the Matrix specification, the following +unstable identifiers were in use. Clients SHOULD migrate to the stable +identifiers defined in this specification. + +| Stable identifier | Unstable identifier | +|---|---| +| `m.room.image_pack` | `im.ponies.room_emotes` | +| `m.image_pack.rooms` | `im.ponies.emote_rooms` | diff --git a/data/event-schemas/schema/m.image_pack.rooms.yaml b/data/event-schemas/schema/m.image_pack.rooms.yaml new file mode 100644 index 00000000..4c23bbf3 --- /dev/null +++ b/data/event-schemas/schema/m.image_pack.rooms.yaml @@ -0,0 +1,43 @@ +--- +$schema: https://json-schema.org/draft/2020-12/schema + +allOf: + - $ref: core-event-schema/event.yaml +description: |- + Stores which room image packs the user has enabled globally, so that the + images in those packs are available in all rooms. +properties: + type: + type: string + enum: + - m.image_pack.rooms + content: + type: object + properties: + rooms: + description: |- + A map of room ID to a map of `state_key` to an empty object. + Each entry references a specific `m.room.image_pack` state event + that the user has enabled globally. + + The bottom-level object is reserved for future use by a subsequent + MSC. Clients MUST treat it as opaque and preserve any unrecognised + properties when modifying this event. + + A room ID present as a key but with no `state_key` entries (i.e. an + empty inner object) currently has no defined meaning. + + Clients SHOULD be aware that the user may not be a member of a room + referenced here, and MAY present appropriate UI to handle this case. + type: object + additionalProperties: + type: object + additionalProperties: + type: object + required: + - rooms +required: + - type + - content +title: ImagePackRooms +type: object diff --git a/data/event-schemas/schema/m.room.image_pack.yaml b/data/event-schemas/schema/m.room.image_pack.yaml new file mode 100644 index 00000000..fc31c8f2 --- /dev/null +++ b/data/event-schemas/schema/m.room.image_pack.yaml @@ -0,0 +1,101 @@ +--- +$schema: https://json-schema.org/draft/2020-12/schema + +allOf: + - $ref: core-event-schema/state_event.yaml +description: |- + Defines an image pack in a room. An image pack is a named collection of + images (emoticons and/or stickers) that can be used by room members. + + A room MAY contain multiple image packs, each identified by a unique + `state_key`. +properties: + content: + type: object + properties: + images: + description: |- + A map from a [shortcode](#shortcode-grammar) to an image object. + Each entry defines one image available in this pack. + type: object + additionalProperties: + title: ImagePackImage + type: object + description: |- + An image available in the pack. + properties: + url: + description: |- + The [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris) + for this image. + type: string + format: mx-mxc-uri + pattern: "^mxc:\\/\\/" + body: + description: |- + A textual representation or description of the image. This is + used as the `alt` attribute when the emote is sent in a message, + and as the `body` when the image is sent as a sticker. + type: string + info: + allOf: + - $ref: core-event-schema/msgtype_infos/image_info.yaml + description: |- + Metadata about the image, using the same + [`ImageInfo`](/client-server-api/#msticker_imageinfo) structure + as [`m.sticker`](/client-server-api/#msticker) events. + required: + - url + pack: + title: ImagePackMeta + type: object + description: |- + Metadata about the image pack as a whole. If absent, clients SHOULD + use the room's name and avatar as the pack's display name and avatar. + properties: + display_name: + description: |- + A display name for the pack. If absent and the pack is defined in + a room, defaults to the room's name. + type: string + avatar_url: + description: |- + The [`mxc://` URI](/client-server-api/#matrix-content-mxc-uris) + of an avatar for the pack. If absent and the pack is defined in a + room, defaults to the room's avatar. + type: string + format: mx-mxc-uri + pattern: "^mxc:\\/\\/" + usage: + description: |- + The intended usages for this pack. The defined values are + `emoticon` (images intended to be sent inline in messages) and + `sticker` (images intended to be sent as standalone sticker + events). If absent or empty, all usage types are assumed. + + Clients SHOULD use this property to determine in which pickers + (emote picker, sticker picker) to surface images from this pack. + type: array + items: + type: string + enum: + - emoticon + - sticker + attribution: + description: |- + An attribution string for the pack, for example crediting the + original author or source. + type: string + required: + - images + state_key: + description: |- + A unique identifier for this image pack within the room. This is not + intended to be surfaced to users. + type: string + type: + enum: + - m.room.image_pack + type: string +title: RoomImagePack +type: object