Order error codes and remove duplicate M_THREEPID_IN_USE definition (#2336)

* Order error codes alphabetically

* Remove duplicated definition of M_THREEPID_IN_USE

* Changelog

* With inline comments about keeping them ordered

.github/workflows/main.yml

@@ -1,7 +1,9 @@
 name: "Spec"
 
 env:
-  HUGO_VERSION: 0.139.0
+  HUGO_VERSION: 0.153.3
+  PYTHON_VERSION: 3.13
+  NODE_VERSION: 24
 
 on:
   push:
@@ -26,7 +28,7 @@ jobs:
       - name: "➕ Setup Node"
         uses: actions/setup-node@v4
         with:
-          node-version: '20'
+          node-version: ${{ env.NODE_VERSION }}
       - name: "🔎 Run validator"
         run: |
           npx @redocly/cli@latest lint data/api/*/*.yaml
@@ -40,7 +42,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -59,7 +61,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -78,7 +80,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -120,7 +122,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip'
           cache-dependency-path: scripts/requirements.txt
       - name: "➕ Install dependencies"
@@ -154,6 +156,11 @@ jobs:
               --api server-server \
               -r "$RELEASE" \
               -o spec/server-server-api/api.json
+          scripts/dump-openapi.py \
+              --base-url "https://spec.matrix.org${{ needs.calculate-baseurl.outputs.baseURL }}" \
+              --api identity \
+              -r "$RELEASE" \
+              -o spec/identity-service-api/api.json
           tar -czf openapi.tar.gz spec
       - name: "📤 Artifact upload"
         uses: actions/upload-artifact@v4
@@ -172,7 +179,7 @@ jobs:
       - name: "➕ Setup Python"
         uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: ${{ env.PYTHON_VERSION }}
       - name: "➕ Install towncrier"
         run: "pip install 'towncrier'"
       - name: "Generate changelog"
@@ -189,11 +196,13 @@ jobs:
     needs: [calculate-baseurl, build-openapi, generate-changelog]
     # run even if generate-changelog was skipped
     if: ${{ always() }}
+    env:
+      baseURL: "${{ needs.calculate-baseurl.outputs.baseURL }}"
     steps:
       - name: "➕ Setup Node"
         uses: actions/setup-node@v4
         with:
-          node-version: '20'
+          node-version: ${{ env.NODE_VERSION }}
       - name: "➕ Setup Hugo"
         uses: peaceiris/actions-hugo@75d2e84710de30f6ff7268e08f310b60ef14033f # v3.0.0
         with:
@@ -211,8 +220,10 @@ jobs:
         with:
           name: changelog-artifact
           path: content/changelog
+
       - name: "⚙️ hugo"
-        run: hugo --baseURL "${{ needs.calculate-baseurl.outputs.baseURL }}" -d "spec"
+        run: hugo --baseURL "${baseURL}" -d "spec${baseURL}"
+
       # We manually unpack the spec OpenAPI definition JSON to the website tree
       # to make it available to the world in a canonical place:
       # https://spec.matrix.org/latest/client-server-api/api.json
@@ -223,10 +234,13 @@ jobs:
           name: openapi-artifact
       - name: "📝 Unpack the OpenAPI definitions in the right location"
         run: |
-          tar -xzf openapi.tar.gz
+          tar -C "spec${baseURL}" --strip-components=1 -xzf openapi.tar.gz
 
       - name: "📦 Tarball creation"
-        run: tar -czf spec.tar.gz spec
+        run: |
+          cd spec
+          tar -czf ../spec.tar.gz *
+
       - name: "📤 Artifact upload"
         uses: actions/upload-artifact@v4
         with:
@@ -237,6 +251,14 @@ jobs:
     name: "🔎 Validate generated HTML"
     runs-on: ubuntu-latest
     needs: [calculate-baseurl, build-spec]
+    # Run even if `generate-changelog` was skipped.
+    #
+    # `build-spec` has a dependency on `generate-changelog` to ensure order of execution
+    # and to access `needs.generate-changelog.result`. However, `generate-changelog` is
+    # skipped on tag builds; even a transient dependency on `generate-changelog` is then
+    # enough for this step to also be skipped by default on tag builds. Hence the need for
+    # this explicit `if`.
+    if: ${{ !failure() && !cancelled() }}
     steps:
       - name: "📥 Source checkout"
         uses: actions/checkout@v4
@@ -247,14 +269,9 @@ jobs:
           name: spec-artifact
 
       - name: "📝 Unpack the spec"
-        # we have to unpack it into the right path given the baseurl, so that the
-        # links are correct.
-        # eg if baseurl is `/unstable`, we want to put the site in `spec/unstable`.
         run: |
-          mkdir -p "spec${baseURL}"
-          tar -C "spec${baseURL}" --strip-components=1 -xvzf spec.tar.gz
-        env:
-          baseURL: "${{ needs.calculate-baseurl.outputs.baseURL }}"
+          mkdir spec
+          tar -C spec -xvzf spec.tar.gz
 
       - name: "Run htmltest"
         uses: wjdp/htmltest-action@master
@@ -264,13 +281,15 @@ jobs:
   build-historical-spec:
     name: "📖 Build the historical backup spec"
     runs-on: ubuntu-latest
-    needs: [build-openapi]
+    needs: [calculate-baseurl, build-openapi]
     if: ${{ startsWith(github.ref, 'refs/tags/') }}
+    env:
+      baseURL: "${{ needs.calculate-baseurl.outputs.baseURL }}"
     steps:
       - name: "➕ Setup Node"
         uses: actions/setup-node@v4
         with:
-          node-version: '20'
+          node-version: ${{ env.NODE_VERSION }}
       - name: "➕ Setup Hugo"
         uses: peaceiris/actions-hugo@75d2e84710de30f6ff7268e08f310b60ef14033f # v3.0.0
         with:
@@ -283,10 +302,10 @@ jobs:
           npm i
           npm run get-proposals
       - name: "⚙️ hugo"
-        # Create a baseURL like `/v1.2` out of the `v1.2` tag
+        env:
+          HUGO_PARAMS_VERSION_STATUS: "historical"
         run: |
-          echo -e '[params.version]\nstatus="historical"' > historical.toml
-          hugo --config config.toml,historical.toml --baseURL "/${GITHUB_REF/refs\/tags\//}" -d "spec"
+          hugo --baseURL "${baseURL}" -d "spec${baseURL}"
 
       - name: "📥 Spec definition download"
         uses: actions/download-artifact@v4
@@ -294,12 +313,51 @@ jobs:
           name: openapi-artifact
       - name: "📝 Unpack the OpenAPI definitions in the right location"
         run: |
-          tar -xzf openapi.tar.gz
+          tar -C "spec${baseURL}" --strip-components=1 -xzf openapi.tar.gz
 
       - name: "📦 Tarball creation"
-        run: tar -czf spec-historical.tar.gz spec
+        run: |
+          cd spec
+          tar -czf ../spec-historical.tar.gz *
+
       - name: "📤 Artifact upload"
         uses: actions/upload-artifact@v4
         with:
           name: spec-historical-artifact
           path: spec-historical.tar.gz
+
+  # If we're building a tag, create a release and publish the artifacts
+  create_release:
+    name: "Create release"
+    if: ${{ !failure() && !cancelled() && startsWith(github.ref, 'refs/tags/') }}
+    needs:
+      - build-spec
+      - build-historical-spec
+    runs-on: ubuntu-latest
+    steps:
+      - name: "📥 Check out changelogs"
+        uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
+        with:
+          sparse-checkout: |
+            content/changelog
+      - name: "📥 Download built spec"
+        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
+        with:
+          name: spec-artifact
+      - name: "📥 Download historical spec artifact"
+        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
+        with:
+          name: spec-historical-artifact
+      - name: "✨ Create draft release"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          # Remove front-matter from changelog
+          sed '1,/^---$/d' "content/changelog/${{ github.ref_name }}.md" > changelog.md
+
+          # Create a draft release, using the changelog as release notes, and attaching the spec artifacts.
+          gh release create -d -t "${{ github.ref_name }}" \
+            -F "changelog.md" \
+            "${{ github.ref_name }}" \
+            spec.tar.gz \
+            spec-historical.tar.gz

.github/workflows/netlify.yaml

@@ -45,7 +45,9 @@ jobs:
           name: spec-artifact
 
       - name: "📦 Extract Artifacts"
-        run: tar -xzvf spec.tar.gz && rm spec.tar.gz
+        run: |
+          mkdir spec
+          tar -C spec -xzvf spec.tar.gz && rm spec.tar.gz
         
       - name: "📤 Deploy to Netlify"
         id: netlify

.github/workflows/release.yaml

@@ -12,6 +12,9 @@ jobs:
     defaults:
       run:
         working-directory: packages/npm
+    permissions:
+      contents: read
+      id-token: write
     steps:
       - name: 🧮 Checkout code
         uses: actions/checkout@v4
@@ -23,6 +26,10 @@ jobs:
           cache-dependency-path: packages/npm/yarn.lock
           registry-url: "https://registry.npmjs.org"
 
+      # Ensure npm 11.5.1 or later is installed
+      - name: Update npm
+        run: npm install -g npm@latest
+
       - name: 🔨 Install dependencies
         run: "yarn install --frozen-lockfile"
 
@@ -33,10 +40,4 @@ jobs:
           VERSION: ${{ github.event.release.tag_name }}.0
 
       - name: 🚀 Publish to npm
-        id: npm-publish
-        uses: JS-DevTools/npm-publish@19c28f1ef146469e409470805ea4279d47c3d35c # v3.1.1
-        with:
-          token: ${{ secrets.NPM_TOKEN }}
-          package: packages/npm
-          access: public
-          ignore-scripts: false
+        run: npm publish --provenance --access public --tag latest

README.md

@@ -61,7 +61,7 @@ place after an MSC has been accepted, not as part of a proposal itself.
 
 1. Install the extended version (often the OS default) of Hugo:
    <https://gohugo.io/getting-started/installing>. Note that at least Hugo
-   v0.123.1 is required.
+   v0.146.0 is required.
 
    Alternatively, use the Docker image at
    https://hub.docker.com/r/klakegg/hugo/. (The "extended edition" is required

assets/js/toc.js

@@ -50,7 +50,7 @@ function getHeadings() {
   let headings = [];
 
   // First get the anchors in the ToC.
-  const toc_anchors = document.querySelectorAll("#toc nav a");
+  const toc_anchors = document.querySelectorAll("#TableOfContents a");
 
   for (const anchor of toc_anchors) {
     // Then get the heading from its selector in the anchor's href.
@@ -122,13 +122,13 @@ function getCurrentHeading(headings, headerOffset) {
 */
 function selectTocEntry(id) {
   // Deselect previously selected entries.
-  const activeEntries = document.querySelectorAll("#toc nav a.active");
+  const activeEntries = document.querySelectorAll("#TableOfContents a.active");
   for (const activeEntry of activeEntries) {
     activeEntry.classList.remove('active');
   }
 
   // Find the new entry and select it.
-  const newEntry = document.querySelector(`#toc nav a[href="#${id}"]`);
+  const newEntry = document.querySelector(`#TableOfContents a[href="#${id}"]`);
   if (!newEntry) {
     console.error("ToC entry not found for ID:", id);
     return;

assets/js/versions.template.js

@@ -0,0 +1,114 @@
+/*
+Copyright 2025 The Matrix.org Foundation C.I.C.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Determine the current version as defined in hugo.toml. This will either be
+// "unstable" or "vX.X" and doesn't depend on the current URL.
+//
+// The oddity below is an attempt at producing a readable Hugo template while
+// avoiding JS syntax errors in your IDE.
+const currentVersion = `{{ if eq .Site.Params.version.status "unstable" }}
+    {{- /**/ -}}
+    unstable
+    {{- /**/ -}}
+{{ else }}
+    {{- /**/ -}}
+    {{ printf "v%s.%s" .Site.Params.version.major .Site.Params.version.minor }}
+    {{- /**/ -}}
+{{ end }}`;
+
+// Determine the current version segment by regex matching the URL. This will either
+// be "unstable", "latest", "vX.X" (production) or undefined (local & netlify).
+const href = window.location.href;
+const segmentMatches = href.match(/(?<=\/)unstable|latest|v\d+.\d+(?=\/)/);
+const currentSegment = segmentMatches ? segmentMatches[0] : undefined;
+
+// Determine the selected menu element. If we were able to obtain the version
+// segment from the URL (production), use that. Otherwise (local & netlify),
+// fall back to the version as defined in Hugo.
+const selected = currentSegment ?? currentVersion;
+
+function appendVersion(parent, name, segment, url) {
+    // The list item
+    const li = document.createElement("li");
+    if (segment === selected) {
+        li.classList.add("version-picker-selected");
+    }
+    if (segment === "latest") {
+        li.classList.add("version-picker-latest");
+    }
+    parent.appendChild(li);
+
+    // The link
+    const a = document.createElement("a");
+    a.classList.add("dropdown-item");
+    a.setAttribute("href", url);
+    li.appendChild(a);
+
+    // Handle clicks manually to preserve the current path / fragment
+    a.addEventListener("click", (ev) => {
+        // If the URL is a relative link (i.e. the historical versions changelog), just
+        // let the browser load it
+        if (url.startsWith("/")) {
+            return;
+        }
+
+        // If we couldn't determine the current segment, we cannot safely replace
+        // it and have to let the browser load the (root) URL instead
+        if (!currentSegment) {
+            return;
+        }
+
+        // Otherwise, stop further event handling and replace the segment
+        ev.preventDefault();
+        ev.stopPropagation();
+        window.location.href = href.replace(`/${currentSegment}/`, `/${segment}/`);
+    });
+
+    // The link text
+    const text = document.createTextNode(name);
+    a.appendChild(text);
+}
+
+// If we're in the unstable version, we're the latest thing and can just load
+// versions.json from our own resources. Otherwise, we fall back to loading it
+// from /unstable/versions.json, assuming we are on the spec.matrix.org deployment.
+const url = currentVersion === "unstable"
+    ? '{{ .Site.Home.Permalink }}versions.json'
+    : "/unstable/versions.json";
+
+fetch(url)
+    .then(r => r.json())
+    .then(versions => {
+        // Find the surrounding list element
+        const ul = document.querySelector("ul#version-selector");
+        if (!ul) {
+            console.error("Cannot populate version selector: ul element not found");
+            return;
+        }
+
+        // Add a entries for the unstable version and the "latest" shortcut
+        appendVersion(ul, "unstable", "unstable", "https://spec.matrix.org/unstable");
+        const latestName = versions?.length ? `latest (${versions[0].name})` : "latest";
+        appendVersion(ul, latestName, "latest", "https://spec.matrix.org/latest");
+
+        // Add an entry for each proper version
+        for (const version of versions) {
+            appendVersion(ul, version.name, version.name, `https://spec.matrix.org/${version.name}`);
+        }
+
+        // For historical versions, simply link to the changelog
+        appendVersion(ul, "historical", "historical", '{{ (site.GetPage "changelog/historical").RelPermalink }}');
+    });

assets/scss/_styles_project.scss

@@ -50,6 +50,20 @@ Custom SCSS for the Matrix spec
   a {
     color: $black;
   }
+
+  /* Make the version dropdown scroll if it's too large */
+  ul#version-selector {
+    max-height: 80vh;
+    overflow-y: auto;
+  }
+
+  ul#version-selector li.version-picker-selected a {
+    font-weight: bold;
+  }
+
+  ul#version-selector li.version-picker-latest a {
+    color: $secondary;
+  }
 }
 
 /* Styles for the sidebar nav */
@@ -64,9 +78,8 @@ Custom SCSS for the Matrix spec
 
   & > .td-sidebar-nav__section {
     margin-top: 1rem;
-  }
 
-  .td-sidebar-nav__section .ul-1 ul {
+    .ul-1 ul {
       padding-left: 0;
     }
 
@@ -83,8 +96,83 @@ Custom SCSS for the Matrix spec
     .ul-2 > li > a {
       padding-left: 2rem !important;
     }
+  }
 
-  a.td-sidebar-link.tree-root {
+  /* Styles for the table of contents */
+  & > .td-toc {
+    padding-top: 1rem;
+    padding-left: 1.5rem;
+    /* Add border above the toc */
+    border-top: 1px solid var(--bs-tertiary-color);
+
+    ol {
+      padding-left: 1rem;
+      counter-reset: section;
+      list-style-type: none;
+    }
+
+    #TableOfContents {
+      /* Remove the space between the title and the ToC */
+      margin-top: 0;
+
+      &>ol>li {
+        margin-bottom: .5rem;
+
+        &>a {
+          font-weight: $font-weight-bold;
+        }
+      }
+
+      ol {
+        padding-left: 0;
+      }
+
+      &>ol>li>a {
+        padding-left: 1rem;
+      }
+
+      &>ol>li>ol>li>a {
+        padding-left: 2rem;
+      }
+
+      &>ol>li>ol>li>ol>li>a {
+        padding-left: 3rem;
+      }
+
+      &>ol>li>ol>li>ol>li>ol>li>a {
+        padding-left: 4rem;
+      }
+
+      &>ol>li>ol>li>ol>li>ol>li>ol>li>a {
+        padding-left: 5rem;
+      }
+
+    }
+
+    li a:before {
+      counter-increment: section;
+      content: counters(section, ".") " ";
+    }
+
+    .td-toc-title {
+      font-weight: $font-weight-bold;
+      font-size: 1.3rem;
+
+      /* Remove the border under the title */
+      border-bottom: 0;
+      /* Remove the space under the title */
+      margin-bottom: 0;
+
+      /* Fix the top of page link color */
+      a, a:hover {
+        color: $secondary;
+      }
+    }
+  }
+
+  /* Apply the same style to links in the navigation and in the ToC */
+  & > .td-sidebar-nav__section, & > .td-toc #TableOfContents {
+    li a.td-sidebar-link.tree-root {
       color: $gray-800;
       font-weight: $font-weight-bold;
       font-size: 1.3rem;
@@ -92,13 +180,12 @@ Custom SCSS for the Matrix spec
       border-bottom: none;
     }
 
-  a, a.td-sidebar-link {
+    li a, li a.td-sidebar-link {
       color: $gray-800;
       font-weight: $font-weight-normal;
       padding-top: .2rem;
       padding-bottom: .2rem;
 
-    display: block;
       transition: all 100ms ease-in-out;
 
       &:hover {
@@ -111,6 +198,7 @@ Custom SCSS for the Matrix spec
       }
     }
   }
+}
 
 @include media-breakpoint-up(md) {
   @supports (position: sticky) {
@@ -185,62 +273,67 @@ Custom SCSS for the Matrix spec
   scroll-margin-top: 5.5rem;
 }
 
-/* Styles for the table of contents */
-#toc {
-  padding-top: .5rem;
-  padding-left: 1.5rem;
-
-  ol {
-    padding-left: 1rem;
-    counter-reset: section;
-    list-style-type: none;
-  }
-
-  #TableOfContents {
-    &>ol>li {
-      margin-bottom: .5rem;
-
-      &>a {
+.endpoints-toc {
+  summary {
+    cursor: pointer;
     font-weight: $font-weight-bold;
-      }
+    font-size: 1.05rem;
+    margin-bottom: 0.5rem;
   }
 
-    ol {
+  .endpoint-list {
+    list-style: none;
     padding-left: 0;
+    margin: 0;
   }
 
-    &>ol>li>a {
-      padding-left: 1rem;
+  .endpoint-list li {
+    margin: 0.2rem 0;
   }
 
-    &>ol>li>ol>li>a {
-      padding-left: 2rem;
+  .endpoint-list a {
+    text-decoration: none;
+    color: inherit;
+    padding: 0.05rem 0.25rem;
+    border-radius: 0.2rem;
+
+    &:hover {
+      background-color: $secondary-background;
+    }
   }
 
-    &>ol>li>ol>li>ol>li>a {
-      padding-left: 3rem;
-    }
-
-    &>ol>li>ol>li>ol>li>ol>li>a {
-      padding-left: 4rem;
-    }
-
-    &>ol>li>ol>li>ol>li>ol>li>ol>li>a {
-      padding-left: 5rem;
-    }
-
-  }
-
-  li a:before {
-    counter-increment: section;
-    content: counters(section, ".") " ";
-  }
-
-  #toc-title {
+  .endpoint-list .http-api-method {
+    margin-right: 0.35rem;
     font-weight: $font-weight-bold;
-    font-size: 1.3rem;
   }
 
+  .endpoint-path {
+    font-family: $font-family-monospace;
+    color: $secondary;
+  }
+
+  .endpoint-deprecated {
+    color: $danger;
+    font-weight: $font-weight-bold;
+    margin-left: 0.35rem;
+  }
+
+  .endpoint-module {
+    &:not(:first-child) {
+      margin-top: 0.75rem;
+    }
+  }
+
+  .endpoint-module-title {
+    // font-weight: $font-weight-bold;
+    font-size: 1.20rem;
+    margin-bottom: 0.35rem;
+  }
+}
+
+.page-description {
+  margin-bottom: 1rem;
+  color: inherit;
 }
 
 /* Styles for alert boxes */
@@ -316,13 +409,19 @@ Custom SCSS for the Matrix spec
   h2 {
     font-weight: $font-weight-bold;
     font-size: 1.3rem;
-    margin: 3rem 0 .5rem 0;
+    margin: 1.5rem 0 1rem 0;
   }
 
   h3 {
     font-weight: $font-weight-bold;
     font-size: 1.1rem;
-    margin: 1.5rem 0 .75rem 0;
+    margin: 1.5rem 0 1rem 0;
+
+  }
+
+  /* Reduce top margin of h3 if previous sibling is a h2 */
+  h2 + h3 {
+    margin-top: 1rem;
   }
 
   hr {
@@ -367,11 +466,6 @@ Custom SCSS for the Matrix spec
       }
     }
 
-    // add some space between two tables when they are right next to each other
-    & + table {
-        margin-top: 4rem;
-    }
-
     caption {
       caption-side: top;
       color: $dark;
@@ -443,6 +537,17 @@ Custom SCSS for the Matrix spec
     }
   }
 
+  /* Have consistent spacing around tables and examples */
+  table, .highlight {
+    margin-top: 0;
+    margin-bottom: 2rem;
+
+    /* We don't need the margin on the last child of the .rendered-data block */
+    &:last-child {
+      margin-bottom: 0;
+    } 
+  }
+
   pre {
     border: 0;
     border-left: solid 5px $secondary;
@@ -490,6 +595,13 @@ Make padding symmetrical (this selector is used in the default styles to apply p
   }
 }
 
+/* Adjust the width of math to match normal paragraphs */
+@include media-breakpoint-up(lg) {
+  .katex-display {
+    max-width: 80%;
+  }
+}
+
 /* Adjust default styles for info banner */
 .pageinfo-primary {
   @include media-breakpoint-up(lg) {

assets/scss/_variables_project.scss

@@ -18,6 +18,7 @@ $primary: #FFF;
 $secondary: #0098D4;
 $dark: #333;
 $gray-100: #FBFBFB;
+$code-color: #005b7f;
 
 $secondary-background: #E5F5FB;
 $secondary-lighter-background: #F4FAFC;

changelogs/appendices/newsfragments/2307.clarification

@@ -0,0 +1 @@
+Add identifier pronunciation guidelines. Contributed by @HarHarLinks.

changelogs/application_service/newsfragments/2330.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification.

changelogs/client_server/newsfragments/2270.feature

@@ -0,0 +1 @@
+Add the account management capabilities for the OAuth 2.0 authentication API, as per [MSC4191](https://github.com/matrix-org/matrix-spec-proposals/pull/4191).

changelogs/client_server/newsfragments/2272.feature

@@ -0,0 +1 @@
+Add OAuth 2.0 aware clients, as per [MSC3824](https://github.com/matrix-org/matrix-spec-proposals/pull/3824).

changelogs/client_server/newsfragments/2277.clarification

@@ -0,0 +1,3 @@
+The optional `submit_url` response parameter of the `/requestToken` endpoints uses the same request
+and response parameters and error codes as the Identity Service API's `POST /_matrix/identity/v2/validate/email/submitToken`,
+as per [MSC4183](https://github.com/matrix-org/matrix-spec-proposals/pull/4183).

changelogs/client_server/newsfragments/2278.feature

@@ -0,0 +1 @@
+Add administrator endpoints to lock and suspend server-local users and add the `m.account_management` capability, as per [MSC4323](https://github.com/matrix-org/matrix-spec-proposals/pull/4323).

changelogs/client_server/newsfragments/2278.new.1

@@ -0,0 +1 @@
+Add `GET /_matrix/client/v1/admin/suspend/{userId}`, as per [MSC4323](https://github.com/matrix-org/matrix-spec-proposals/pull/4323).

changelogs/client_server/newsfragments/2278.new.2

@@ -0,0 +1 @@
+Add `PUT /_matrix/client/v1/admin/suspend/{userId}`, as per [MSC4323](https://github.com/matrix-org/matrix-spec-proposals/pull/4323).

changelogs/client_server/newsfragments/2278.new.3

@@ -0,0 +1 @@
+Add `GET /_matrix/client/v1/admin/lock/{userId}`, as per [MSC4323](https://github.com/matrix-org/matrix-spec-proposals/pull/4323).

changelogs/client_server/newsfragments/2278.new.4

@@ -0,0 +1 @@
+Add `PUT /_matrix/client/v1/admin/lock/{userId}`, as per [MSC4323](https://github.com/matrix-org/matrix-spec-proposals/pull/4323).

changelogs/client_server/newsfragments/2280.clarification

@@ -0,0 +1 @@
+Update non-historic mentions of matrix-doc repo to matrix-spec/-proposals. Contributed by @HarHarLinks.

changelogs/client_server/newsfragments/2283.clarification

@@ -0,0 +1 @@
+Remove unintended TeX formatting. Contributed by @HarHarLinks.

changelogs/client_server/newsfragments/2291.feature

@@ -0,0 +1 @@
+Add `m.recent_emoji` account data event to track recently used emoji as per [MSC4356](https://github.com/matrix-org/matrix-spec-proposals/pull/4356).

changelogs/client_server/newsfragments/2292.feature

@@ -0,0 +1 @@
+Add `m.forget_forced_upon_leave` capability for servers to transparently auto-forget rooms that the user leaves as per [MSC4267](https://github.com/matrix-org/matrix-spec-proposals/pull/4267).

changelogs/client_server/newsfragments/2298.feature

@@ -0,0 +1 @@
+Add support for `m.room.redaction` events at the `PUT /rooms/{roomId}/send/{eventType}/{txnId}` endpoint, as per [MSC4169](https://github.com/matrix-org/matrix-spec-proposals/pull/4169).

changelogs/client_server/newsfragments/2299.feature

@@ -0,0 +1 @@
+Clients supporting the `ol` HTML element must also support the `start` attribute, as per [MSC4313](https://github.com/matrix-org/matrix-spec-proposals/pull/4313).

changelogs/client_server/newsfragments/2301.feature

@@ -0,0 +1 @@
+Add recommendation about excluding non-cross-signed devices from encrypted conversations, as per [MSC4153](https://github.com/matrix-org/matrix-spec-proposals/pull/4153).

changelogs/client_server/newsfragments/2304.clarification

@@ -0,0 +1 @@
+Clarify the requiredness of `event_id` in `predecessor`.

changelogs/client_server/newsfragments/2305.feature

@@ -0,0 +1 @@
+Add invite blocking, as per [MSC4380](https://github.com/matrix-org/matrix-spec-proposals/pull/4380).

changelogs/client_server/newsfragments/2306.clarification

@@ -0,0 +1 @@
+Clarify terminology for keys in cross-signing module.

changelogs/client_server/newsfragments/2311.feature

@@ -0,0 +1 @@
+`/_matrix/client/v3/rooms/{roomId}/report` and `/_matrix/client/v3/rooms/{roomId}/report/{eventId}` may respond with HTTP 200 regardless of the reported subject's existence or add a random delay when generating responses as per [MSC4277](https://github.com/matrix-org/matrix-spec-proposals/pull/4277).

changelogs/client_server/newsfragments/2311.removal

@@ -0,0 +1 @@
+The `score` request parameter on `/_matrix/client/v3/rooms/{roomId}/report/{eventId}` was removed as per [MSC4277](https://github.com/matrix-org/matrix-spec-proposals/pull/4277).

changelogs/client_server/newsfragments/2315.feature

@@ -0,0 +1 @@
+Add `M_USER_LIMIT_EXCEEDED` common error code, as per [MSC4335](https://github.com/matrix-org/matrix-spec-proposals/pull/4335).

changelogs/client_server/newsfragments/2316.clarification

@@ -0,0 +1 @@
+Add 404 responses to the OpenAPI of `GET /login` and `GET /auth_metadata` endpoints. The responses were already defined in text but not written in OpenAPI.

changelogs/client_server/newsfragments/2318.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification. Contributed by @HarHarLinks.

changelogs/client_server/newsfragments/2320.feature

@@ -0,0 +1 @@
+Add the OAuth 2.0 Device Authorization Grant (RFC 8628) as a supported grant type, as per [MSC4341](https://github.com/matrix-org/matrix-spec-proposals/pull/4341).

changelogs/client_server/newsfragments/2324.clarification

@@ -0,0 +1 @@
+Clarified attachment encryption to require secure generation of keys and hash verification.

changelogs/client_server/newsfragments/2328.feature

@@ -0,0 +1 @@
+Add the `is_animated` flag to the `info` object of the `m.image` msgtype and the `m.sticker` event, as per [MSC4230](https://github.com/matrix-org/matrix-spec-proposals/pull/4230).

changelogs/client_server/newsfragments/2336.clarification

@@ -0,0 +1 @@
+Order the common and other error codes alphabetically and remove duplicate `M_THREEPID_IN_USE` definition.

changelogs/client_server/newsfragments/2337.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification.

changelogs/client_server/newsfragments/2338.feature

@@ -0,0 +1 @@
+Add the `is_animated` flag to the `info` object of the `m.image` msgtype and the `m.sticker` event, as per [MSC4230](https://github.com/matrix-org/matrix-spec-proposals/pull/4230).

changelogs/identity_service/newsfragments/2277.clarification

@@ -0,0 +1,3 @@
+Clarify the error codes that can be returned with a 400 HTTP status code by the `POST /_matrix/identity/v2/validate/email/submitToken`
+and `POST /_matrix/identity/v2/validate/msisdn/submitToken` endpoints, introducing the `M_TOKEN_INCORRECT`
+error code, as per [MSC4183](https://github.com/matrix-org/matrix-spec-proposals/pull/4183).

changelogs/identity_service/newsfragments/2336.clarification

@@ -0,0 +1 @@
+Order the standard error codes alphabetically.

changelogs/internal/newsfragments/2222.clarification

@@ -0,0 +1 @@
+Clarify vendor prefixing requirements.

changelogs/internal/newsfragments/2275.clarification

@@ -0,0 +1 @@
+Auto-create draft releases when building release tags.

changelogs/internal/newsfragments/2276.feature

@@ -0,0 +1 @@
+Include the spec release version in the filenames in the tarballs generated by CI.

changelogs/internal/newsfragments/2282.clarification

@@ -0,0 +1 @@
+Replace the Twitter link in the footer with our BlueSky and Mastodon socials.

changelogs/internal/newsfragments/2287.clarification

@@ -0,0 +1 @@
+Upgrade to docsy v0.13.0.

changelogs/internal/newsfragments/2289.clarification

@@ -0,0 +1 @@
+Updates to the release documentation.

changelogs/internal/newsfragments/2290.clarification

@@ -0,0 +1 @@
+Remove unused leftover CSS files.

changelogs/internal/newsfragments/2317.clarification

@@ -0,0 +1 @@
+Update the footer social links to match matrix.org. Contributed by @HarHarLinks.

changelogs/internal/newsfragments/2318.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification. Contributed by @HarHarLinks.

changelogs/internal/newsfragments/2323.clarification

@@ -0,0 +1 @@
+Render error code sections as definition lists to improve readability.

changelogs/room_versions/newsfragments/2297.clarification

@@ -0,0 +1 @@
+Clarify meaning of floating-point powerlevels.

changelogs/room_versions/newsfragments/2303.clarification

@@ -0,0 +1 @@
+Remove the post-1.16 release note for room version 12.

changelogs/server_server/newsfragments/2191.clarification

@@ -0,0 +1 @@
+Clarify what the `minimum_valid_until_ts` field means when it is set in key queries.

changelogs/server_server/newsfragments/2284.clarification

@@ -0,0 +1 @@
+Specify validation for PDUs passed to and returned from federation membership endpoints.

changelogs/server_server/newsfragments/2288.clarification

@@ -0,0 +1 @@
+Specify that callers of `/_matrix/federation/v1/openid/userinfo` must validate the returned user ID.

changelogs/server_server/newsfragments/2300.clarification

@@ -0,0 +1 @@
+Change `m.signing_update` typo to `m.signing_key_update`. Contributed by @velikopter

changelogs/server_server/newsfragments/2319.removal

@@ -0,0 +1 @@
+Remove `/v1/send_join` and `/v1/send_leave`, as per [MSC4376](https://github.com/matrix-org/matrix-spec-proposals/pull/4376).

changelogs/server_server/newsfragments/2329.clarification

@@ -0,0 +1 @@
+Add link to JSON signing algorithm in server-server auth section for clarity. Contributed by @thetayloredman.

changelogs/server_server/newsfragments/2338.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification.

config/_default/hugo.toml

@@ -8,7 +8,7 @@ enableRobotsTXT = true
 # We disable RSS, because (a) it's useless, (b) Hugo seems to generate broken
 # links to it when used with a --baseURL (for example, https://spec.matrix.org/v1.4/
 # contains `<link rel="alternate" type="application/rss&#43;xml" href="/v1.4/v1.4/index.xml">`).
-disableKinds = ["taxonomy", "RSS"]
+disableKinds = ["taxonomy", "rss"]
 
 [languages]
 [languages.en]
@@ -23,15 +23,15 @@ description = "Home of the Matrix specification for decentralised communication"
 [menus]
 [[menus.main]]
     name = 'Foundation'
-    url = 'https://matrix.org/foundation/'
+    url = 'https://matrix.org/foundation/about/'
     weight = 10
 [[menus.main]]
-    name = 'FAQs'
-    url = 'https://matrix.org/faq'
+    name = 'User Docs'
+    url = 'https://matrix.org/docs/'
     weight = 20
 [[menus.main]]
     name = 'Blog'
-    url = 'https://matrix.org/blog/posts'
+    url = 'https://matrix.org/blog/'
     weight = 30
 
 [markup]
@@ -43,6 +43,15 @@ description = "Home of the Matrix specification for decentralised communication"
     [markup.goldmark.renderer]
       # Enables us to render raw HTML
       unsafe = true
+    [markup.goldmark.extensions]
+      # Tell Goldmark to pass delimited blocks through the `render-passthrough` render hook.
+      # This is used to render the maths in the Olm spec.
+      # See: https://gohugo.io/functions/transform/tomath/#step-1.
+      [markup.goldmark.extensions.passthrough]
+        enable = true
+        [markup.goldmark.extensions.passthrough.delimiters]
+          block = [['\[', '\]']]
+          inline = [['\(', '\)']]
   [markup.highlight]
       # See a complete list of available styles at https://xyproto.github.io/splash/docs/all.html
       # If the style is changed, remember to regenerate the CSS with:
@@ -56,18 +65,22 @@ description = "Home of the Matrix specification for decentralised communication"
 # Everything below this are Site Params
 
 [params]
-copyright = "The Matrix.org Foundation CIC"
+copyright = "The Matrix.org Foundation C.I.C."
 
 [params.version]
 # must be one of "unstable", "current", "historical"
 # this is used to decide whether to show a banner pointing to the current release
-status = "stable"
+status = "unstable"
 # A URL pointing to the latest, stable release of the spec. To be shown in the unstable version warning banner.
 current_version_url = "https://spec.matrix.org/latest"
 # The following is used when status = "stable", and is displayed in various UI elements on a released version
 # of the spec.
-major = "1"
-minor = "14"
+#major = "1"
+#minor = "17"
+
+[[params.versions]]
+# We must include this parameter to enable docsy's version picker in the navbar. The picker
+# is populated automatically in navbar-version-selector.html.
 
 # User interface configuration
 [params.ui]
@@ -102,16 +115,31 @@ sidebar_menu_compact = true
   url = "https://gitlab.matrix.org/matrix-org"
   icon = "fab fa-gitlab"
   desc = "Matrix on GitLab"
+[[params.links.bottom]]
+  name = "Mastodon"
+  url = "https://mastodon.matrix.org/@matrix"
+  icon = "fab fa-mastodon"
+  desc = "Matrix on Mastodon"
+[[params.links.bottom]]
+  name = "Bluesky"
+  url = "https://bsky.app/profile/matrix.org"
+  icon = "fab fa-bluesky"
+  desc = "Matrix on Bluesky"
+[[params.links.bottom]]
+  name = "LinkedIn"
+  url = "https://www.linkedin.com/company/matrix-org/"
+  icon = "fab fa-linkedin"
+  desc = "Matrix on LinkedIn"
 [[params.links.bottom]]
   name = "YouTube"
   url = "https://www.youtube.com/channel/UCVFkW-chclhuyYRbmmfwt6w"
   icon = "fab fa-youtube"
   desc = "Matrix YouTube channel"
 [[params.links.bottom]]
-	name = "Twitter"
-	url = "https://twitter.com/matrixdotorg"
-	icon = "fab fa-twitter"
-  desc = "Matrix on Twitter"
+  name = "Matrix.org Blog Feed"
+  url = "https://matrix.org/atom.xml"
+  icon = "fas fa-rss"
+  desc = "Matrix.org Blog Atom Feed"
 
 
 # configuration for the hugo development server
@@ -121,7 +149,9 @@ sidebar_menu_compact = true
 [[server.headers]]
   for = '/**'
   [server.headers.values]
-    Content-Security-Policy = "default-src 'self'; style-src 'self'; script-src 'self'; img-src 'self' data:; connect-src 'self'; font-src 'self' data:; media-src 'self'; child-src 'self'; form-action 'self'; object-src 'self'"
+    # `style-src 'unsafe-inline'` is needed to correctly render the maths in the Olm spec:
+    # https://github.com/KaTeX/KaTeX/issues/4096
+    Content-Security-Policy = "default-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; img-src 'self' data:; connect-src 'self'; font-src 'self' data:; media-src 'self'; child-src 'self'; form-action 'self'; object-src 'self'"
     X-XSS-Protection = "1; mode=block"
     X-Content-Type-Options = "nosniff"
     # Strict-Transport-Security = "max-age=31536000; includeSubDomains; preload"
@@ -134,7 +164,7 @@ sidebar_menu_compact = true
 [module]
   [module.hugoVersion]
     extended = true
-    min = "0.123.1"
+    min = "0.146.0"
   [[module.imports]]
     path = "github.com/matrix-org/docsy"
     disable = false
@@ -146,3 +176,11 @@ sidebar_menu_compact = true
     mediaType = "text/markdown"
     isPlainText = true
     baseName = "checklist"
+
+# Add font media types for downloading KaTeX fonts.
+# See: https://www.docsy.dev/docs/content/diagrams-and-formulae/#create-media-types-for-katex-fonts
+[mediaTypes]
+  [mediaTypes.'font/woff']
+    suffixes = ['woff']
+  [mediaTypes.'font/woff2']
+    suffixes = ['woff2']

content/_index.md

@@ -25,6 +25,7 @@ The specification consists of the following parts:
 * [Identity Service API](/identity-service-api)
 * [Push Gateway API](/push-gateway-api)
 * [Room Versions](/rooms)
+* [Olm & Megolm](/olm-megolm)
 * [Appendices](/appendices)
 
 Additionally, this introduction page contains the key baseline
@@ -151,7 +152,7 @@ request.
 
 How data flows between clients:
 
-```
+```nohighlight
     { Matrix client A }                             { Matrix client B }
         ^          |                                    ^          |
         |  events  |  Client-Server API                 |  events  |

content/appendices.md

@@ -533,6 +533,11 @@ where `domain` is the [server name](#server-name) of the homeserver
 which allocated the identifier, and `localpart` is an identifier
 allocated by that homeserver.
 
+Because the domain part identifies the server on which the ID resolves,
+the canonical pronunciation of the separating `:` is "on".
+For example, `@user:matrix.org` would be pronounced as "at user on matrix dot
+org".
+
 The precise grammar defining the allowable format of an identifier
 depends on the type of identifier. For example, event IDs can sometimes
 be represented with a `domain` component under some conditions - see the
@@ -657,27 +662,48 @@ provides no way to encode ASCII punctuation).
 
 #### Room IDs
 
-A room has exactly one room ID. A room ID has the format:
+{{% changed-in v="1.16" %}} Room IDs can now appear without a domain depending on
+the room version.
+
+A room has exactly one room ID. Room IDs take the form:
+
+    !opaque_id
+
+However, the precise format depends upon the [room version specification](/rooms):
+some room versions included a `domain` component, whereas more recent room versions
+omit the domain and use a base64-encoded hash instead.
+
+Room IDs are case-sensitive and not meant to be human-readable. They are intended
+to be used as fully opaque strings by clients, even when a `domain` component is
+present.
+
+If the room version requires a `domain` component, room IDs take the following
+form:
 
     !opaque_id:domain
 
-The `domain` of a room ID is the [server name](#server-name) of the
-homeserver which created the room. The domain is used only for
-namespacing to avoid the risk of clashes of identifiers between
-different homeservers. There is no implication that the room in
-question is still available at the corresponding homeserver.
+In such a form, the `opaque_id` is a localpart. The localpart MUST only contain
+valid non-surrogate Unicode code points, including control characters, except `:`
+and `NUL` (U+0000). The localpart SHOULD only consist of alphanumeric characters
+(`A-Z`, `a-z`, `0-9`) when generating them. The `domain` is the [server name](#server-name)
+of the homeserver which created the room - it is only used to reduce namespace
+collisions. There is no implication that the room in question is still available
+at the corresponding homeserver. Combined, the localpart, domain, and `!` sigil
+MUST NOT exceed 255 bytes.
 
-Room IDs are case-sensitive. They are not meant to be
-human-readable. They are intended to be treated as fully opaque strings
-by clients.
+When a room version requires the `domain`-less format, room IDs are simply the
+[event ID](#event-ids) of the `m.room.create` event using `!` as the sigil instead
+of `$`. The grammar is otherwise inherited verbatim.
 
-The localpart of a room ID (`opaque_id` above) may contain any valid
-non-surrogate Unicode code points, including control characters, except `:` and `NUL`
-(U+0000), but it is recommended to only include ASCII letters and
-digits (`A-Z`, `a-z`, `0-9`) when generating them.
+{{% boxes/note %}}
+Applications which previously relied upon the `domain` in a room ID can instead
+parse the [user IDs](#user-identifiers) found in the `m.room.create` event's `sender`.
 
-The length of a room ID, including the `!` sigil and the domain, MUST
-NOT exceed 255 bytes.
+Though the `m.room.create` event's `additional_creators` (in `content`) may be
+used when present, applications should take care when parsing or interpreting the
+list. The user IDs in `additional_creators` will have correct grammar, but may
+not be real users or may not belong to actual Matrix homeservers.
+{{% /boxes/note %}}
 
 #### Room Aliases
 
@@ -728,13 +754,13 @@ history (a permalink).
 
 The Matrix URI scheme is defined as follows (`[]` enclose optional parts, `{}`
 enclose variables):
-```
+```nohighlight
 matrix:[//{authority}/]{type}/{id without sigil}[/{type}/{id without sigil}...][?{query}][#{fragment}]
 ```
 
 As a schema, this can be represented as:
 
-```
+```nohighlight
 MatrixURI = "matrix:" hier-part [ "?" query ] [ "#" fragment ]
 hier-part = [ "//" authority "/" ] path
 path = entity-descriptor ["/" entity-descriptor]
@@ -844,7 +870,7 @@ below for more details.
 A matrix.to URI has the following format, based upon the specification
 defined in [RFC 3986](https://tools.ietf.org/html/rfc3986):
 
-```
+```nohighlight
 https://matrix.to/#/<identifier>/<extra parameter>?<additional arguments>
 ```
 

content/application-service-api.md

@@ -2,16 +2,14 @@
 title: "Application Service API"
 weight: 30
 type: docs
+description: |
+  The Matrix client-server API and server-server APIs provide a consistent,
+  self-contained federated messaging fabric but leave little room for custom
+  server-side behaviour such as gateways, filters, or extensible hooks. The
+  Application Service API defines a standard way to add this extensible
+  functionality, independent of the underlying homeserver implementation.
 ---
 
-The Matrix client-server API and server-server APIs provide the means to
-implement a consistent self-contained federated messaging fabric.
-However, they provide limited means of implementing custom server-side
-behaviour in Matrix (e.g. gateways, filters, extensible hooks etc). The
-Application Service API (AS API) defines a standard API to allow such
-extensible functionality to be implemented irrespective of the
-underlying homeserver implementation.
-
 ## Application Services
 
 Application services are passive and can only observe events from the
@@ -86,7 +84,7 @@ For the `users` namespace, application services can only register interest in
 homeserver). Events affecting users on other homeservers are not sent to an application
 service, even if the user happens to match the one of the `users` namespaces (unless,
 of course, the event affects a room that the application service is interested in
-for another room - for example, because there is another user in the room that the
+for another reason - for example, because there is another user in the room that the
 application service is interested in).
 
 For the `rooms` and `aliases` namespaces, all events in a matching room will be
@@ -178,13 +176,13 @@ The application service API provides a transaction API for sending a
 list of events. Each list of events includes a transaction ID, which
 works as follows:
 
-```
+```nohighlight
     Typical
     HS ---> AS : Homeserver sends events with transaction ID T.
        <---    : Application Service sends back 200 OK.
 ```
 
-```
+```nohighlight
     AS ACK Lost
     HS ---> AS : Homeserver sends events with transaction ID T.
        <-/-    : AS 200 OK is lost.
@@ -258,7 +256,7 @@ have been omitted for brevity):
 
 **Typical**
 
-```
+```nohighlight
 AS ---> HS : /_matrix/client/v1/appservice/{appserviceId}/ping {"transaction_id": "meow"}
     HS ---> AS : /_matrix/app/v1/ping {"transaction_id": "meow"}
     HS <--- AS : 200 OK {}
@@ -267,7 +265,7 @@ AS <--- HS : 200 OK {"duration_ms": 123}
 
 **Incorrect `hs_token`**
 
-```
+```nohighlight
 AS ---> HS : /_matrix/client/v1/appservice/{appserviceId}/ping {"transaction_id": "meow"}
     HS ---> AS : /_matrix/app/v1/ping {"transaction_id": "meow"}
     HS <--- AS : 403 Forbidden {"errcode": "M_FORBIDDEN"}
@@ -276,7 +274,7 @@ AS <--- HS : 502 Bad Gateway {"errcode": "M_BAD_STATUS", "status": 403, "body":
 
 **Can't connect to appservice**
 
-```
+```nohighlight
 AS ---> HS : /_matrix/client/v1/appservice/{appserviceId}/ping {"transaction_id": "meow"}
     HS -/-> AS : /_matrix/app/v1/ping {"transaction_id": "meow"}
 AS <--- HS : 502 Bad Gateway {"errcode": "M_CONNECTION_FAILED"}
@@ -356,6 +354,7 @@ service would like to masquerade as.
 Inputs:
 -   Application service token (`as_token`)
 -   User ID in the AS namespace to act as.
+-   Device ID belonging to the User ID to act with.
 
 Notes:
 -   This applies to all aspects of the Client-Server API, except for
@@ -375,9 +374,19 @@ service's `user` namespaces. If the parameter is missing, the homeserver
 is to assume the application service intends to act as the user implied
 by the `sender_localpart` property of the registration.
 
+{{% added-in v="1.17" %}} Application services MAY similarly masquerade
+as a specific device ID belonging the user ID through use of the `device_id`
+query string parameter on the request. If the given device ID is not known
+to belong to the user, the server will return a 400 `M_UNKNOWN_DEVICE` error.
+If no `user_id` is supplied, the `device_id` MUST belong to the user implied
+by the `sender_localpart` property of the application service's registration.
+If no `device_id` is supplied, the homeserver is to assume the request is
+being made without a device ID and will fail to complete operations which
+require a device ID (such as uploading one-time keys).
+
 An example request would be:
 
-    GET /_matrix/client/v3/account/whoami?user_id=@_irc_user:example.org
+    GET /_matrix/client/v3/account/whoami?user_id=@_irc_user:example.org&device_id=ABC123
     Authorization: Bearer YourApplicationServiceTokenHere
 
 #### Timestamp massaging
@@ -417,6 +426,8 @@ imports and similar behaviour).
 
 #### Server admin style permissions
 
+{{% changed-in v="1.17" %}}
+
 The homeserver needs to give the application service *full control* over
 its namespace, both for users and for room aliases. This means that the
 AS should be able to manage any users and room alias in its namespace. No additional API
@@ -433,33 +444,59 @@ achieved by including the `as_token` on a `/register` request, along
 with a login type of `m.login.application_service` to set the desired
 user ID without a password.
 
+```http
 POST /_matrix/client/v3/register
 Authorization: Bearer YourApplicationServiceTokenHere
+```
 
-    Content:
+```json
 {
-      type: "m.login.application_service",
-      username: "_irc_example"
+  "type": "m.login.application_service",
+  "username": "_irc_example"
 }
+```
 
-Similarly, logging in as users needs API changes in order to allow the AS to
-log in without needing the user's password. This is achieved by including the
-`as_token` on a `/login` request, along with a login type of
-`m.login.application_service`:
+{{% boxes/note %}}
+{{% added-in v="1.17" %}}
+Servers MUST still allow application services to use the `/register` endpoint
+with a login type of `m.login.application_service` even if they don't support
+the [Legacy Authentication API](/client-server-api/#legacy-api).
+
+In that case application services MUST set the `"inhibit_login": true` parameter
+as they cannot use it to log in as users. If the `inhibit_login` parameter is
+not set to `true`, the server MUST return a 400 HTTP status code with an
+`M_APPSERVICE_LOGIN_UNSUPPORTED` error code.
+{{% /boxes/note %}}
+
+Similarly, logging in as users using the [Legacy authentication API](/client-server-api/#legacy-api)
+needs API changes in order to allow the AS to log in without needing the user's
+password. This is achieved by including the `as_token` on a `/login` request,
+along with a login type of `m.login.application_service`:
 
 {{% added-in v="1.2" %}}
 
+```http
 POST /_matrix/client/v3/login
 Authorization: Bearer YourApplicationServiceTokenHere
+```
 
-    Content:
+```json
 {
-      type: "m.login.application_service",
+  "type": "m.login.application_service",
   "identifier": {
     "type": "m.id.user",
     "user": "_irc_example"
   }
 }
+```
+
+{{% boxes/note %}}
+{{% added-in v="1.17" %}}
+Application services MUST NOT use the `/login` endpoint if the server doesn't
+support the Legacy authentication API. If `/login` is called with the
+`m.login.application_service` login type the server MUST return a 400 HTTP
+status code with an `M_APPSERVICE_LOGIN_UNSUPPORTED` error code.
+{{% /boxes/note %}}
 
 Application services which attempt to create users or aliases *outside*
 of their defined namespaces, or log in as users outside of their defined
@@ -492,15 +529,47 @@ via the query string). It is expected that the application service use
 the transactions pushed to it to handle events rather than syncing with
 the user implied by `sender_localpart`.
 
-#### Application service room directories
+#### Published room directories
 
-Application services can maintain their own room directories for their
-defined third-party protocols. These room directories may be accessed by
+Application services can maintain their own published room directories for
+their defined third-party protocols. These directories may be accessed by
 clients through additional parameters on the `/publicRooms`
 client-server endpoint.
 
 {{% http-api spec="client-server" api="appservice_room_directory" %}}
 
+#### Device management
+
+{{% added-in v="1.17" %}}
+
+Application services need to be able to create and delete devices to manage the
+encryption for their users without having to rely on `/login`, which also
+generates an access token for the user, and which might not be available for
+homeservers that only support the [OAuth 2.0 API](/client-server-api/#oauth-20-api).
+
+##### Creating devices
+
+Application services can use the [`PUT /_matrix/client/v3/devices/{deviceId}`](/client-server-api/#put_matrixclientv3devicesdeviceid)
+endpoint to create new devices.
+
+##### Deleting devices
+
+The following endpoints used to delete devices MUST NOT require [User-Interactive
+Authentication](/client-server-api/#user-interactive-authentication-api) when
+used by an application service:
+
+* [`DELETE /_matrix/client/v3/devices/{deviceId}`](/client-server-api/#delete_matrixclientv3devicesdeviceid)
+* [`POST /_matrix/client/v3/delete_devices`](/client-server-api/#post_matrixclientv3delete_devices)
+
+#### Cross-signing
+
+{{% added-in v="1.17" %}}
+
+Appservices need to be able to verify themselves and replace their cross-signing
+keys, so the [`POST /_matrix/client/v3/keys/device_signing/upload`](/client-server-api/#post_matrixclientv3keysdevice_signingupload)
+endpoint MUST NOT require [User-Interactive Authentication](/client-server-api/#user-interactive-authentication-api)
+when used by an application service, even if cross-signing keys already exist.
+
 ### Referencing messages from a third-party network
 
 Application services should include an `external_url` in the `content`

content/changelog/_index.md

@@ -1,6 +1,7 @@
 ---
 title: Changelog
 type: docs
+layout: changelog-index
 weight: 1000
 ---
 

content/changelog/v1.15.md

@@ -0,0 +1,97 @@
+---
+title: v1.15 Changelog
+linkTitle: v1.15
+type: docs
+layout: changelog
+outputs:
+  - html
+  - checklist
+date: 2025-06-26
+---
+
+## Client-Server API
+
+**New Endpoints**
+
+- Add `GET /_matrix/client/v1/room_summary/{roomIdOrAlias}`, as per [MSC3266](https://github.com/matrix-org/matrix-spec-proposals/pull/3266). ([#2125](https://github.com/matrix-org/matrix-spec/issues/2125))
+- Add `GET /_matrix/client/v1/auth_metadata`, as per [MSC2965](https://github.com/matrix-org/matrix-spec-proposals/pull/2965). ([#2147](https://github.com/matrix-org/matrix-spec/issues/2147))
+
+**Backwards Compatible Changes**
+
+- Add `m.topic` content block to enable rich text in `m.room.topic` events as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765). ([#2095](https://github.com/matrix-org/matrix-spec/issues/2095))
+- Include device keys with Olm-encrypted events as per [MSC4147](https://github.com/matrix-org/matrix-spec-proposals/pull/4147). ([#2122](https://github.com/matrix-org/matrix-spec/issues/2122))
+- Add `/_matrix/client/v1/room_summary/{roomIdOrAlias}` and extend `/_matrix/client/v1/rooms/{roomId}/hierarchy` with the new optional properties `allowed_room_ids`, `encryption` and `room_version` as per [MSC3266](https://github.com/matrix-org/matrix-spec-proposals/pull/3266). ([#2125](https://github.com/matrix-org/matrix-spec/issues/2125), [#2158](https://github.com/matrix-org/matrix-spec/issues/2158))
+- Add the OAuth 2.0 based authentication API, as per [MSC3861](https://github.com/matrix-org/matrix-spec-proposals/pull/3861) and its sub-proposals. ([#2141](https://github.com/matrix-org/matrix-spec/issues/2141), [#2148](https://github.com/matrix-org/matrix-spec/issues/2148), [#2149](https://github.com/matrix-org/matrix-spec/issues/2149), [#2150](https://github.com/matrix-org/matrix-spec/issues/2150), [#2151](https://github.com/matrix-org/matrix-spec/issues/2151), [#2159](https://github.com/matrix-org/matrix-spec/issues/2159), [#2164](https://github.com/matrix-org/matrix-spec/issues/2164))
+
+**Spec Clarifications**
+
+- Clarify behaviour when the `topic` key of a `m.room.topic` event is absent, null, or empty. ([#2068](https://github.com/matrix-org/matrix-spec/issues/2068))
+- Fix the example of the `GET /sync` endpoint and the `m.room.member` example used in several places. ([#2077](https://github.com/matrix-org/matrix-spec/issues/2077))
+- Clarify the format of third-party invites, including the fact that identity server public keys can be encoded using standard or URL-safe base64. ([#2083](https://github.com/matrix-org/matrix-spec/issues/2083))
+- "Public" rooms in profile look-ups are defined through their join rule and history visibility. ([#2101](https://github.com/matrix-org/matrix-spec/issues/2101))
+- "Public" rooms in user directory queries are defined through their join rule and history visibility. ([#2102](https://github.com/matrix-org/matrix-spec/issues/2102))
+- Rooms published in `/publicRooms` don't necessarily have `public` join rules or `world_readable` history visibility. ([#2104](https://github.com/matrix-org/matrix-spec/issues/2104))
+- "Public" rooms with respect to call invites are defined through their join rule. ([#2106](https://github.com/matrix-org/matrix-spec/issues/2106))
+- "Public" rooms have no specific meaning with respect to moderation policy lists. ([#2107](https://github.com/matrix-org/matrix-spec/issues/2107))
+- "Public" rooms with respect to presence are defined through their join rule. ([#2108](https://github.com/matrix-org/matrix-spec/issues/2108))
+- Spaces are subject to the same access mechanisms as rooms. ([#2109](https://github.com/matrix-org/matrix-spec/issues/2109))
+- Fix various typos throughout the specification. ([#2121](https://github.com/matrix-org/matrix-spec/issues/2121), [#2144](https://github.com/matrix-org/matrix-spec/issues/2144))
+- Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks. ([#2140](https://github.com/matrix-org/matrix-spec/issues/2140))
+- Add missing fields in example for `ExportedSessionData`. ([#2154](https://github.com/matrix-org/matrix-spec/issues/2154))
+
+
+## Server-Server API
+
+**Backwards Compatible Changes**
+
+- Add `m.topic` content block to enable rich text in `m.room.topic` events as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765). ([#2095](https://github.com/matrix-org/matrix-spec/issues/2095))
+- Extend `/_matrix/federation/v1/hierarchy/{roomId}` with the new optional properties `encryption` and `room_version` as per [MSC3266](https://github.com/matrix-org/matrix-spec-proposals/pull/3266). ([#2125](https://github.com/matrix-org/matrix-spec/issues/2125))
+
+**Spec Clarifications**
+
+- Add a note to the invite endpoints that invites to local users may be received twice over federation if the homeserver is already in the room. ([#2067](https://github.com/matrix-org/matrix-spec/issues/2067))
+- Clarify the format of third-party invites, including the fact that identity server public keys can be encoded using standard or URL-safe base64. ([#2083](https://github.com/matrix-org/matrix-spec/issues/2083))
+- Clarify that auth event of `content.join_authorised_via_users_server` is only necessary for `m.room.member` with a `membership` of `join`. ([#2100](https://github.com/matrix-org/matrix-spec/issues/2100))
+- Rooms published in `/publicRooms` don't necessarily have `public` join rules or `world_readable` history visibility. ([#2104](https://github.com/matrix-org/matrix-spec/issues/2104))
+- Fix various typos throughout the specification. ([#2128](https://github.com/matrix-org/matrix-spec/issues/2128))
+- Clarify that Well-Known URIs are available on the server name's hostname. Contributed by @HarHarLinks. ([#2140](https://github.com/matrix-org/matrix-spec/issues/2140))
+
+
+## Application Service API
+
+**Spec Clarifications**
+
+- Clarify in the application service registration schema the `url: null` behaviour. ([#2130](https://github.com/matrix-org/matrix-spec/issues/2130))
+
+
+## Identity Service API
+
+**Spec Clarifications**
+
+- Clarify that public keys can be encoded using standard or URL-safe base64. ([#2083](https://github.com/matrix-org/matrix-spec/issues/2083))
+
+
+## Push Gateway API
+
+No significant changes.
+
+
+## Room Versions
+
+No significant changes.
+
+
+## Appendices
+
+No significant changes.
+
+
+## Internal Changes/Tooling
+
+**Spec Clarifications**
+
+- Adjust margins in rendered endpoints. ([#2081](https://github.com/matrix-org/matrix-spec/issues/2081))
+- Replace Hugo shortcodes in OpenAPI output. ([#2088](https://github.com/matrix-org/matrix-spec/issues/2088))
+- Add [well-known funding manifest urls](https://floss.fund/funding-manifest/) to spec to authorise https://matrix.org/funding.json. Contributed by @HarHarLinks. ([#2115](https://github.com/matrix-org/matrix-spec/issues/2115))
+- Fix the historical info box when generating the historical spec in CI. ([#2123](https://github.com/matrix-org/matrix-spec/issues/2123))
+- Update the header navigation menu with links to modern matrix.org. Contributed by @HarHarLinks. ([#2137](https://github.com/matrix-org/matrix-spec/issues/2137))

content/changelog/v1.16.md

@@ -0,0 +1,103 @@
+---
+title: v1.16 Changelog
+linkTitle: v1.16
+type: docs
+layout: changelog
+outputs:
+  - html
+  - checklist
+date: 2025-09-17
+---
+
+## Client-Server API
+
+**Deprecations**
+
+- Deprecate `m.set_avatar_url` and `m.set_displayname` capabilities, as per [MSC4133](https://github.com/matrix-org/matrix-spec-proposals/pull/4133). ([#2071](https://github.com/matrix-org/matrix-spec/issues/2071))
+
+**Removed Endpoints**
+
+- Remove unintentional intentional mentions in replies, as per [MSC4142](https://github.com/matrix-org/matrix-spec-proposals/pull/4142). ([#2210](https://github.com/matrix-org/matrix-spec/issues/2210))
+
+**Backwards Compatible Changes**
+
+- Update user profile endpoints to handle custom fields, and add a new `m.profile_fields` capability, as per [MSC4133](https://github.com/matrix-org/matrix-spec-proposals/pull/4133). ([#2071](https://github.com/matrix-org/matrix-spec/issues/2071))
+- Add `format` query parameter to `GET /state/{eventType}/{stateKey}` to allow fetching metadata of a specific state event. ([#2175](https://github.com/matrix-org/matrix-spec/issues/2175))
+- Add the `use_state_after` query parameter and `state_after` response property to `GET /sync`, as per [MSC4222](https://github.com/matrix-org/matrix-spec-proposals/issues/4222). ([#2187](https://github.com/matrix-org/matrix-spec/issues/2187))
+- When upgrading rooms to [room version 12](/rooms/v12), `additional_creators` may be specified on the [`POST /_matrix/client/v3/rooms/{roomId}/upgrade`](/client-server-api/#post_matrixclientv3roomsroomidupgrade) endpoint, as per [MSC4289](https://github.com/matrix-org/matrix-spec-proposals/pull/4289). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- When creating rooms with [room version 12](/rooms/v12), the `trusted_private_chat` preset should merge the invitees into the supplied `content.additional_creators` in the resulting room, as per [MSC4289](https://github.com/matrix-org/matrix-spec-proposals/pull/4289). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- In [room version 12](/rooms/v12), the power level of room creators is now infinitely high as per [MSC4289](https://github.com/matrix-org/matrix-spec-proposals/pull/4289). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- In [room version 12](/rooms/v12), room IDs no longer have a domain component as per [MSC4291](https://github.com/matrix-org/matrix-spec-proposals/pull/4291). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- When creating rooms with [room version 12](/rooms/v12), the initial power levels will restrict the ability to upgrade rooms by default, as per [MSC4289](https://github.com/matrix-org/matrix-spec-proposals/pull/4289). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- Add a profile field for a user's time zone, as per [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175). ([#2206](https://github.com/matrix-org/matrix-spec/issues/2206))
+- Invites and knocks are now expected to contain the `m.room.create` event in their stripped state entries, as per [MSC4311](https://github.com/matrix-org/matrix-spec-proposals/pull/4311). ([#2207](https://github.com/matrix-org/matrix-spec/issues/2207))
+
+**Spec Clarifications**
+
+- Clarify that `format` is required if `formatted_body` is specified. ([#2167](https://github.com/matrix-org/matrix-spec/issues/2167))
+- The `latest_event` in an aggregated set of thread events uses the same format as the event itself. ([#2169](https://github.com/matrix-org/matrix-spec/issues/2169))
+- Fix various typos throughout the specification. ([#2171](https://github.com/matrix-org/matrix-spec/issues/2171), [#2177](https://github.com/matrix-org/matrix-spec/issues/2177), [#2179](https://github.com/matrix-org/matrix-spec/issues/2179))
+- Clarify that clients should replace events with the most recent replacement by `origin_server_ts`. ([#2190](https://github.com/matrix-org/matrix-spec/issues/2190))
+- Fix `/sync` flow referencing incorrect parameter to use with `/messages`. ([#2195](https://github.com/matrix-org/matrix-spec/issues/2195))
+- Clarify wording around the `world_readable` history visibility setting. Contributed by @HarHarLinks. ([#2204](https://github.com/matrix-org/matrix-spec/issues/2204))
+
+
+## Server-Server API
+
+**Backwards Compatible Changes**
+
+- `invite_room_state` and `knock_room_state` now have additional requirements and validation depending on the room version, as per [MSC4311](https://github.com/matrix-org/matrix-spec-proposals/pull/4311). ([#2207](https://github.com/matrix-org/matrix-spec/issues/2207))
+
+
+## Application Service API
+
+No significant changes.
+
+
+## Identity Service API
+
+No significant changes.
+
+
+## Push Gateway API
+
+No significant changes.
+
+
+## Room Versions
+
+**Backwards Compatible Changes**
+
+- Room IDs in room version 12 are now the event ID of the create event with the normal room ID sigil (`!`), as per [MSC4291](https://github.com/matrix-org/matrix-spec-proposals/pull/4291). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- Room creators are formalized in room version 12 and have infinitely high power level, as per [MSC4289](https://github.com/matrix-org/matrix-spec-proposals/pull/4289). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- State Resolution is updated in room version 12 to reduce the opportunity for "state resets", as per [MSC4297](https://github.com/matrix-org/matrix-spec-proposals/pull/4297). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- The default room version is now room version 12, though servers SHOULD keep using room version 11 for a little while, as per [MSC4304](https://github.com/matrix-org/matrix-spec-proposals/pull/4304). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+- Add [room version 12](/rooms/v12) as per [MSC4304](https://github.com/matrix-org/matrix-spec-proposals/pull/4304). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193), [#2199](https://github.com/matrix-org/matrix-spec/issues/2199))
+
+**Spec Clarifications**
+
+- In room versions 1 through 12, an event's `auth_events` have always needed to belong to the same room as per [MSC4307](https://github.com/matrix-org/matrix-spec-proposals/pull/4307). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+
+
+## Appendices
+
+**Backwards Compatible Changes**
+
+- Room IDs can now appear without a domain component in [room version 12](/rooms/v12), as per [MSC4291](https://github.com/matrix-org/matrix-spec-proposals/pull/4291). ([#2193](https://github.com/matrix-org/matrix-spec/issues/2193))
+
+
+## Internal Changes/Tooling
+
+**Backwards Compatible Changes**
+
+- Add "placeholder MSC" process definition. ([#2157](https://github.com/matrix-org/matrix-spec/issues/2157))
+
+**Spec Clarifications**
+
+- Declare the Application Service Registration schema to follow JSON Schema spec 2020-12. ([#2132](https://github.com/matrix-org/matrix-spec/issues/2132))
+- Declare the event schemas to follow JSON Schema spec 2020-12. ([#2132](https://github.com/matrix-org/matrix-spec/issues/2132))
+- Upgrade the docsy theme to version 0.12.0. ([#2160](https://github.com/matrix-org/matrix-spec/issues/2160))
+- GitHub actions are now building the OpenAPI `spec/identity-service-api/api.json` file. ([#2172](https://github.com/matrix-org/matrix-spec/issues/2172))
+- Minor fixes to JSON schemas. ([#2182](https://github.com/matrix-org/matrix-spec/issues/2182))
+- Specify a correct spelling for "display name". ([#2189](https://github.com/matrix-org/matrix-spec/issues/2189))
+- Fix a grammatical typo on the Matrix Spec Process documentation page. ([#2205](https://github.com/matrix-org/matrix-spec/issues/2205))

content/changelog/v1.17.md

@@ -0,0 +1,91 @@
+---
+title: v1.17 Changelog
+linkTitle: v1.17
+type: docs
+layout: changelog
+outputs:
+  - html
+  - checklist
+date: 2025-12-18
+---
+
+## Client-Server API
+
+**Removed Endpoints**
+
+- Remove legacy mentions, as per [MSC4210](https://github.com/matrix-org/matrix-spec-proposals/issues/4210). ([#2186](https://github.com/matrix-org/matrix-spec/issues/2186))
+
+**Backwards Compatible Changes**
+
+- Allow application services to masquerade as specific devices belonging to users, as per [MSC4326](https://github.com/matrix-org/matrix-spec-proposals/pull/4326). ([#2221](https://github.com/matrix-org/matrix-spec/issues/2221))
+- Add the `m.oauth` authentication type for User-Interactive Authentication, as per [MSC4312](https://github.com/matrix-org/matrix-spec-proposals/pull/4312). ([#2234](https://github.com/matrix-org/matrix-spec/issues/2234))
+- Allow application services to manage devices and register users without the legacy authentication API, as per [MSC4190](https://github.com/matrix-org/matrix-spec-proposals/pull/4190). ([#2267](https://github.com/matrix-org/matrix-spec/issues/2267))
+
+**Spec Clarifications**
+
+- Push rule IDs are globally unique within their kind. ([#2214](https://github.com/matrix-org/matrix-spec/issues/2214))
+- Don't advertise `creator` field in description of room creation. ([#2215](https://github.com/matrix-org/matrix-spec/issues/2215))
+- `room_id` is required for peeking via `/_matrix/client/v3/events`. ([#2216](https://github.com/matrix-org/matrix-spec/issues/2216))
+- The `server-name` segment of MXC URIs is sanitised differently from the `media-id` segment. ([#2217](https://github.com/matrix-org/matrix-spec/issues/2217))
+- Add note to each endpoint that uses capability negotiation. ([#2223](https://github.com/matrix-org/matrix-spec/issues/2223))
+- Additional OpenGraph properties can be present in URL previews. ([#2225](https://github.com/matrix-org/matrix-spec/issues/2225))
+- Clarify the special casing of membership events and redactions in power levels. ([#2231](https://github.com/matrix-org/matrix-spec/issues/2231))
+- `M_RESOURCE_LIMIT_EXCEEDED` is now listed as a common error code. ([#2232](https://github.com/matrix-org/matrix-spec/issues/2232))
+- Add `m.login.terms` to enumeration of authentication types. ([#2233](https://github.com/matrix-org/matrix-spec/issues/2233))
+- Clarify how to use `state_after` ahead of declaring full support for its spec version. ([#2240](https://github.com/matrix-org/matrix-spec/issues/2240))
+- `device_one_time_keys_count` is only optional if no unclaimed one-time keys exist. ([#2245](https://github.com/matrix-org/matrix-spec/issues/2245))
+- Clarify that servers may choose not to use `M_USER_DEACTIVATED` at login time, for example for privacy reasons when they can't authenticate deactivated users. ([#2246](https://github.com/matrix-org/matrix-spec/issues/2246))
+- Usage of the `event_id_only` format for push notifications is not mandatory. ([#2255](https://github.com/matrix-org/matrix-spec/issues/2255))
+- Fix various typos throughout the specification. ([#2224](https://github.com/matrix-org/matrix-spec/issues/2224), [#2227](https://github.com/matrix-org/matrix-spec/issues/2227), [#2250](https://github.com/matrix-org/matrix-spec/issues/2250))
+
+
+## Server-Server API
+
+No significant changes.
+
+
+## Application Service API
+
+**Backwards Compatible Changes**
+
+- Allow application services to masquerade as specific devices belonging to users, as per [MSC4326](https://github.com/matrix-org/matrix-spec-proposals/pull/4326). ([#2221](https://github.com/matrix-org/matrix-spec/issues/2221))
+- Allow application services to manage devices and register users without the legacy authentication API, as per [MSC4190](https://github.com/matrix-org/matrix-spec-proposals/pull/4190). ([#2267](https://github.com/matrix-org/matrix-spec/issues/2267))
+
+**Spec Clarifications**
+
+- Fix JSON formatting in the "Server admin style permissions" examples. ([#2213](https://github.com/matrix-org/matrix-spec/issues/2213))
+
+
+## Identity Service API
+
+No significant changes.
+
+
+## Push Gateway API
+
+No significant changes.
+
+
+## Room Versions
+
+**Spec Clarifications**
+
+- In room versions 8 through 12, clarify that "sufficient permission to invite users" on restricted joins also includes being a joined member of the room. ([#2220](https://github.com/matrix-org/matrix-spec/issues/2220))
+- In room versions 3 through 12, clarify that when you have the power to redact, it is possible to redact events that you don't have the power to send. ([#2249](https://github.com/matrix-org/matrix-spec/issues/2249))
+
+
+## Appendices
+
+No significant changes.
+
+
+## Internal Changes/Tooling
+
+**Spec Clarifications**
+
+- Swapped icon for X (fka. twitter) to updated logo in footer. ([#2219](https://github.com/matrix-org/matrix-spec/issues/2219))
+- Inline Olm & Megolm specifications. ([#2226](https://github.com/matrix-org/matrix-spec/issues/2226), [#2241](https://github.com/matrix-org/matrix-spec/issues/2241), [#2242](https://github.com/matrix-org/matrix-spec/issues/2242))
+- Silence failing redocly-cli rule. ([#2238](https://github.com/matrix-org/matrix-spec/issues/2238))
+- Use NPM Trusted Publishers for publishing `@matrix-org/spec` to npm. ([#2239](https://github.com/matrix-org/matrix-spec/issues/2239))
+- Add version picker in the navbar. ([#2256](https://github.com/matrix-org/matrix-spec/issues/2256), [#2258](https://github.com/matrix-org/matrix-spec/issues/2258), [#2259](https://github.com/matrix-org/matrix-spec/issues/2259), [#2260](https://github.com/matrix-org/matrix-spec/issues/2260), [#2261](https://github.com/matrix-org/matrix-spec/issues/2261), [#2264](https://github.com/matrix-org/matrix-spec/issues/2264), [#2268](https://github.com/matrix-org/matrix-spec/issues/2268))
+- Add a list of endpoints to the top of each spec page. ([#2262](https://github.com/matrix-org/matrix-spec/issues/2262))

content/client-server-api/modules/content_repo.md

@@ -33,7 +33,7 @@ specification.
 Content locations are represented as Matrix Content (`mxc://`) URIs. They
 look like:
 
-```
+```nohighlight
 mxc://<server-name>/<media-id>
 
 <server-name> : The name of the homeserver where this content originated, e.g. matrix.org
@@ -87,7 +87,7 @@ Matrix 1.12 is expected to be released in the July-September 2024 calendar quart
 The homeserver SHOULD be able to supply thumbnails for uploaded images
 and videos. The exact file types which can be thumbnailed are not
 currently specified - see [Issue
-\#1938](https://github.com/matrix-org/matrix-doc/issues/1938) for more
+\#1938](https://github.com/matrix-org/matrix-spec/issues/453) for more
 information.
 
 The thumbnail methods are "crop" and "scale". "scale" tries to return an
@@ -134,9 +134,14 @@ entity isn't in the room.
 `mxc://` URIs are vulnerable to directory traversal attacks such as
 `mxc://127.0.0.1/../../../some_service/etc/passwd`. This would cause the
 target homeserver to try to access and return this file. As such,
-homeservers MUST sanitise `mxc://` URIs by allowing only alphanumeric
-(`A-Za-z0-9`), `_` and `-` characters in the `server-name` and
-`media-id` values. This set of whitelisted characters allows URL-safe
+homeservers MUST sanitise `mxc://` URIs by:
+
+-   restricting the `server-name` segment to valid
+    [server names](/appendices/#server-name)
+-   allowing only alphanumeric (`A-Za-z0-9`), `_` and `-` characters in
+    the `media-id` segment
+
+The resulting set of whitelisted characters allows URL-safe
 base64 encodings specified in RFC 4648. Applying this character
 whitelist is preferable to blacklisting `.` and `/` as there are
 techniques around blacklisted characters (percent-encoded characters,

content/client-server-api/modules/end_to_end_encryption.md

@@ -5,6 +5,91 @@ Matrix optionally supports end-to-end encryption, allowing rooms to be
 created whose conversation contents are not decryptable or interceptable
 on any of the participating homeservers.
 
+#### Recommended client behaviour
+
+{{% added-in v="1.18" %}}
+
+While clients are able to choose what encryption features they implement based
+on their threat model, this section recommends behaviours that will improve the
+overall user experience and security of encrypted conversations.
+
+While a user may be unable to [verify](#device-verification) every other user
+that they communicate with, or may be unaware of the need to verify other users,
+[cross-signing](#cross-signing) gives some measure of protection and so SHOULD
+be used where possible. In particular, clients SHOULD implement the following
+recommendations.
+
+* Clients SHOULD create new [cross-signing keys](#cross-signing) for users who
+  do not yet have cross-signing keys.
+* Clients SHOULD encourage users to set up their [Secret Storage](#storage) to
+  avoid needing to reset their cryptographic identity in case the user does not
+  have an existing device that can [share the secrets](#sharing) with the new
+  device. The user's Secret Storage SHOULD contain the user's cross-signing
+  private keys and the [key backup](#server-side-key-backups) decryption key
+  (if the user is using key backup). The user's Secret Storage SHOULD have a
+  [default key](#key-storage) (a key referred to by
+  `m.secret_storage.default_key`) that encrypts the private cross-signing keys
+  and key backup decryption key (if available).
+* Clients SHOULD encourage users to [cross-sign](#cross-signing) their devices.
+  This includes both when logging in a new device, and for existing devices.
+  Clients MAY even go so  far as to require cross-signing of devices by
+  preventing the user from using  the client until the device is cross-signed.
+  If the user cannot cross-sign their device (for example, if they have
+  forgotten their Secret Storage key), the client can allow users to reset their
+  [Secret Storage](#storage), cross-signing keys, and [key backup](#server-side-key-backups).
+* When Alice [verifies](#device-verification) Bob, the verification SHOULD
+  verify their [cross-signing keys](#cross-signing). Any flow between different
+  users that does not verify the users' cross-signing keys (it verifies only the
+  device keys) is deprecated.
+* Clients SHOULD flag when [cross-signing keys](#cross-signing) change. If
+  Alice's cross-signing keys change, Alice's own devices MUST alert her to this
+  fact, and prompt her to re-cross-sign those devices. If Bob is in an
+  encrypted room with Alice,  Bob's devices SHOULD inform him of Alice's key
+  change and SHOULD prevent him from sending an encrypted message to Alice
+  without acknowledging the change. Bob's clients may behave differently
+  depending on whether Bob had previously [verified](#device-verification)
+  Alice or not. For example, if Bob had previously verified Alice, and Alice's
+  keys change, Bob's client may require Bob to re-verify, or may display a more
+  aggressive warning.
+* Clients SHOULD NOT send encrypted [to-device](#send-to-device-messaging)
+  messages, such as [room keys](#sharing-keys-between-devices) or [secrets](#secrets)
+  (via [Secret Sharing](#sharing)), to [non-cross-signed](#cross-signing)
+  devices by default. Non-cross-signed devices don't provide any assurance that
+  the device belongs to the user, and server admins can trivially create new
+  devices for users. When sending room keys, clients can use a
+  [`m.room_key.withheld`](#mroom_keywithheld) message with a code of
+  `m.unverified` to indicate to the non-cross-signed device why it is not
+  receiving the room key.
+
+  Note that clients cannot selectively send room events only to cross-signed
+  devices. The only way to exclude non-cross-signed devices from encrypted
+  conversations is to not send the room keys so those devices won't be able to
+  decrypt the messages.
+* Similarly, messages sent from [non-cross-signed](#cross-signing) devices
+  cannot be trusted and SHOULD NOT be displayed to the user. Clients have no
+  assurance that encrypted messages sent from non-cross-signed devices were sent
+  by the user, rather than an impersonator.
+* Matrix clients MUST NOT consider non-cryptographic devices (devices which do
+  not have [device identity keys](#device-keys) uploaded to the homeserver) to
+  be equivalent to [non-cross-signed](#cross-signing) cryptographic devices for
+  purposes of enforcing E2EE policy. For example, clients SHOULD NOT warn nor
+  refuse to send messages due to the presence of non-cryptographic devices. For
+  all intents and purposes, non-cryptographic devices are a completely separate
+  concept and do not exist from the perspective of the cryptography layer since
+  they do not have identity keys, so it is impossible to send them decryption
+  keys.
+* Clients MAY make provisions for encrypted bridges. Some bridges are structured
+  in a way such that only one user controlled by the bridge (often called the
+  bridge bot) participates in encryption, and encrypted messages from other
+  bridge users are encrypted by the bridge bot. Thus encrypted messages sent by
+  one user could be encrypted by a [Megolm](#mmegolmv1aes-sha2) session sent by
+  a different user. Clients MAY accept such messages, provided the session
+  creator's device is [cross-signed](#cross-signing). However, the client MUST
+  annotate the message with a warning, unless the client has a way to check that
+  the bridge bot is permitted to encrypt messages on behalf of the user. Future
+  MSCs such as [MSC4350](https://github.com/matrix-org/matrix-spec-proposals/pull/4350)
+  may provide a secure way to allow such impersonation.
+
 #### Key Distribution
 
 Encryption and Authentication in Matrix is based around public-key
@@ -18,7 +103,7 @@ exchange fingerprints between users to build a web of trust.
 device. This may include long-term identity keys, and/or one-time
 keys.
 
-```
+```nohighlight
       +----------+  +--------------+
       | Bob's HS |  | Bob's Device |
       +----------+  +--------------+
@@ -29,7 +114,7 @@ keys.
 
 2) Alice requests Bob's public identity keys and supported algorithms.
 
-```
+```nohighlight
       +----------------+  +------------+  +----------+
       | Alice's Device |  | Alice's HS |  | Bob's HS |
       +----------------+  +------------+  +----------+
@@ -40,7 +125,7 @@ keys.
 
 3) Alice selects an algorithm and claims any one-time keys needed.
 
-```
+```nohighlight
       +----------------+  +------------+  +----------+
       | Alice's Device |  | Alice's HS |  | Bob's HS |
       +----------------+  +------------+  +----------+
@@ -93,7 +178,7 @@ Example:
 ```
 
 `ed25519` and `curve25519` keys are used for [device keys](#device-keys).
-Additionally, `ed25519` keys are used for [cross-signing keys](#cross-signing).
+Additionally, `ed25519` keys are used for [cross-signing](#cross-signing).
 
 `signed_curve25519` keys are used for [one-time and fallback keys](#one-time-and-fallback-keys).
 
@@ -265,11 +350,14 @@ field, and as a result should remove her from its list of tracked users.
 When encryption is enabled in a room, files should be uploaded encrypted
 on the homeserver.
 
-In order to achieve this, a client should generate a single-use 256-bit
-AES key, and encrypt the file using AES-CTR. The counter should be
-64-bit long, starting at 0 and prefixed by a random 64-bit
-Initialization Vector (IV), which together form a 128-bit unique counter
-block.
+In order to achieve this, the client generates a single-use 256-bit AES
+key, and encrypts the file using AES-CTR. The counter is 64 bits long,
+starting at 0 and prefixed by a random 64-bit Initialization Vector (IV),
+which together form a 128-bit unique counter block.
+
+Clients MUST generate both the AES key and IV using a cryptographically 
+secure random source and MUST NOT use the same key or IV multiple
+times. The latter 64 bits of the 128-bit counter block MUST start at zero.
 
 {{% boxes/warning %}}
 An IV must never be used multiple times with the same key. This implies
@@ -279,13 +367,14 @@ same key and IV.
 {{% /boxes/warning %}}
 
 Then, the encrypted file can be uploaded to the homeserver. The key and
-the IV must be included in the room event along with the resulting
-`mxc://` in order to allow recipients to decrypt the file. As the event
+the IV are included in the room event along with the resulting `mxc://`
+in order to allow recipients to decrypt the file. As the event
 containing those will be Megolm encrypted, the server will never have
 access to the decrypted file.
 
-A hash of the ciphertext must also be included, in order to prevent the
-homeserver from changing the file content.
+A hash of the ciphertext MUST also be included, in order to prevent the
+homeserver from changing the file content. Clients MUST verify the hash
+before using the file contents.
 
 A client should send the data as an encrypted `m.room.message` event,
 using either `m.file` as the msgtype, or the appropriate msgtype for the
@@ -307,7 +396,7 @@ properties.
 | url       | string           | **Required.** The URL to the file.                                                             |
 | key       | JWK              | **Required.** A [JSON Web Key](https://tools.ietf.org/html/rfc7517#appendix-A.3) object.       |
 | iv        | string           | **Required.** The 128-bit unique counter block used by AES-CTR, encoded as unpadded base64.    |
-| hashes    | {string: string} | **Required.** A map from an algorithm name to a hash of the ciphertext, encoded as unpadded base64. Clients should support the SHA-256 hash, which uses the key `sha256`. |
+| hashes    | {string: string} | **Required.** A map from an algorithm name to a hash of the ciphertext, encoded as unpadded base64. Clients MUST support the SHA-256 hash, which uses the key `sha256`. |
 | v         | string           | **Required.** Version of the encrypted attachment's protocol. Must be `v2`.                    |
 
 `JWK`
@@ -491,7 +580,7 @@ this example, Bob's device sends the `m.key.verification.start`, Alice's device
 could also send that message. As well, the order of the
 `m.key.verification.done` messages could be reversed.
 
-```
+```nohighlight
     +---------------+ +---------------+                    +-------------+ +-------------+
     | AliceDevice1  | | AliceDevice2  |                    | BobDevice1  | | BobDevice2  |
     +---------------+ +---------------+                    +-------------+ +-------------+
@@ -528,7 +617,7 @@ messages, Alice only sends one request event (an event with type
 `m.room.message` with `msgtype: m.key.verification.request`, rather than an
 event with type `m.key.verification.request`), to the room. In addition, Alice
 does not send an `m.key.verification.cancel` event to tell Bob's other devices
-that the request as already been accepted; instead, when Bob's other devices
+that the request has already been accepted; instead, when Bob's other devices
 see his `m.key.verification.ready` event, they will know that the request has
 already been accepted, and that they should ignore the request.
 
@@ -548,7 +637,7 @@ request.
 
 The prompt for Bob to accept/reject Alice's request (or the unsupported method
 prompt) should be automatically dismissed 10 minutes after the `timestamp` (in
-the case of to-device messages) or `origin_ts` (in the case of in-room
+the case of to-device messages) or `origin_server_ts` (in the case of in-room
 messages) field or 2 minutes after Bob's client receives the message, whichever
 comes first, if Bob does not interact with the prompt. The prompt should
 additionally be hidden if an appropriate `m.key.verification.cancel` message is
@@ -674,8 +763,11 @@ The process between Alice and Bob verifying each other would be:
     their devices if they match or not.
 15. Assuming they match, Alice and Bob's devices each calculate Message
     Authentication Codes (MACs) for:
-    * Each of the keys that they wish the other user to verify (usually their
-      device ed25519 key and their master cross-signing key).
+    * {{% changed-in v="1.18" %}} Each of the keys that they wish the other user
+      to verify (usually their device ed25519 key and their master signing key, 
+      see below). The master signing key SHOULD be included when two different
+      users are verifying each other. Verifying individual devices of other
+      users is deprecated.
     * The complete list of key IDs that they wish the other user to verify.
 
     The MAC calculation is defined [below](#mac-calculation).
@@ -695,7 +787,7 @@ The process between Alice and Bob verifying each other would be:
 The wire protocol looks like the following between Alice and Bob's
 devices:
 
-```
+```nohighlight
     +-------------+                    +-----------+
     | AliceDevice |                    | BobDevice |
     +-------------+                    +-----------+
@@ -921,7 +1013,7 @@ collaborate to create a common set of translations for all languages.
 
 {{% boxes/note %}}
 Known translations for the emoji are available from
-<https://github.com/matrix-org/matrix-doc/blob/master/data-definitions/>
+<https://github.com/matrix-org/matrix-spec/tree/main/data-definitions/>
 and can be translated online:
 <https://translate.riot.im/projects/matrix-doc/sas-emoji-v1>
 {{% /boxes/note %}}
@@ -931,45 +1023,47 @@ and can be translated online:
 Rather than requiring Alice to verify each of Bob's devices with each of
 her own devices and vice versa, the cross-signing feature allows users
 to sign their device keys such that Alice and Bob only need to verify
-once. With cross-signing, each user has a set of cross-signing keys that
+once. With cross-signing, each user has a set of cross-signing key pairs that
 are used to sign their own device keys and other users' keys, and can be
 used to trust device keys that were not verified directly.
 
-Each user has three ed25519 key pairs for cross-signing:
+Each user has three ed25519 key pairs used for cross-signing (cross-signing keys):
 
--   a master key (MSK) that serves as the user's identity in
-    cross-signing and signs their other cross-signing keys;
+-   a master signing key (MSK, for historical reasons sometimes known as
+    `master_key`) that serves as the user's identity in cross-signing and signs
+    their user-signing and self-signing keys;
 -   a user-signing key (USK) -- only visible to the user that it belongs
-    to --that signs other users' master keys; and
+    to -- that signs other users' master signing keys; and
 -   a self-signing key (SSK) that signs the user's own device keys.
 
-The master key may also be used to sign other items such as the backup
-key. The master key may also be signed by the user's own device keys to
+The master signing key may also be used to sign other items such as the backup
+key. The master signing key may also be signed by the user's own device keys to
 aid in migrating from device verifications: if Alice's device had
 previously verified Bob's device and Bob's device has signed his master
-key, then Alice's device can trust Bob's master key, and she can sign it
+key, then Alice's device can trust Bob's master signing key, and she can sign it
 with her user-signing key.
 
-Users upload their cross-signing keys to the server using [POST
+Users upload the public parts of their master signing, user-signing and
+self-signing keys to the server using [POST
 /\_matrix/client/v3/keys/device\_signing/upload](/client-server-api/#post_matrixclientv3keysdevice_signingupload). When Alice uploads
-new cross-signing keys, her user ID will appear in the `changed`
+new keys, her user ID will appear in the `changed`
 property of the `device_lists` field of the `/sync` of response of all
 users who share an encrypted room with her. When Bob sees Alice's user
 ID in his `/sync`, he will call [POST /\_matrix/client/v3/keys/query](/client-server-api/#post_matrixclientv3keysquery)
-to retrieve Alice's device and cross-signing keys.
+to retrieve Alice's device keys, as well as their cross-signing keys.
 
 If Alice has a device and wishes to send an encrypted message to Bob,
 she can trust Bob's device if:
 
--   Alice's device is using a master key that has signed her
+-   Alice's device is using a master signing key that has signed her
     user-signing key,
--   Alice's user-signing key has signed Bob's master key,
--   Bob's master key has signed Bob's self-signing key, and
+-   Alice's user-signing key has signed Bob's master signing key,
+-   Bob's master signing key has signed Bob's self-signing key, and
 -   Bob's self-signing key has signed Bob's device key.
 
 The following diagram illustrates how keys are signed:
 
-```
+```nohighlight
     +------------------+                ..................   +----------------+
     | +--------------+ |   ..................            :   | +------------+ |
     | |              v v   v            :   :            v   v v            | |
@@ -1000,7 +1094,7 @@ the user who created them.
 The following diagram illustrates Alice's view, hiding the keys and
 signatures that she cannot see:
 
-```
+```nohighlight
     +------------------+                +----------------+   +----------------+
     | +--------------+ |                |                |   | +------------+ |
     | |              v v                |                v   v v            | |
@@ -1024,27 +1118,28 @@ signatures that she cannot see:
 ```
 
 [Verification methods](#device-verification) can be used to verify a
-user's master key by using the master public key, encoded using unpadded
-base64, as the device ID, and treating it as a normal device. For
-example, if Alice and Bob verify each other using SAS, Alice's
+user's master signing key by treating its public key (master signing public
+key), encoded using unpadded base64, as the device ID, and treating it as a
+normal device. For example, if Alice and Bob verify each other using SAS,
+Alice's
 `m.key.verification.mac` message to Bob may include
 `"ed25519:alices+master+public+key": "alices+master+public+key"` in the
 `mac` property. Servers therefore must ensure that device IDs will not
 collide with cross-signing public keys.
 
-The cross-signing private keys can be stored on the server or shared with other
-devices using the [Secrets](#secrets) module.  When doing so, the master,
-user-signing, and self-signing keys are identified using the names
-`m.cross_signing.master`, `m.cross_signing.user_signing`, and
+Using the [Secrets](#secrets) module the private parts of the cross-signing keys can
+be stored on the server or shared with other devices.  When doing so, the
+master signing, user-signing, and self-signing keys are identified using the
+names `m.cross_signing.master`, `m.cross_signing.user_signing`, and
 `m.cross_signing.self_signing`, respectively, and the keys are base64-encoded
 before being encrypted.
 
 ###### Key and signature security
 
-A user's master key could allow an attacker to impersonate that user to
+A user's master signing key could allow an attacker to impersonate that user to
 other users, or other users to that user. Thus clients must ensure that
-the private part of the master key is treated securely. If clients do
-not have a secure means of storing the master key (such as a secret
+the private part of the master signing key is treated securely. If clients do
+not have a secure means of storing the master signing key (such as a secret
 storage system provided by the operating system), then clients must not
 store the private part.
 
@@ -1057,9 +1152,9 @@ Since device key IDs (`ed25519:DEVICE_ID`) and cross-signing key IDs
 use the correct keys when verifying.
 
 While servers MUST not allow devices to have the same IDs as cross-signing
-keys, a malicious server could construct such a situation, so clients must not
-rely on the server being well-behaved and should take the following precautions
-against this.
+keys, a malicious server could construct such a situation, so clients
+must not rely on the server being well-behaved and should take the following
+precautions against this:
 
 1. Clients MUST refer to keys by their public keys during the verification
    process, rather than only by the key ID.
@@ -1067,31 +1162,32 @@ against this.
    verification process, and ensure that they do not change in the course of
    verification.
 3. Clients SHOULD also display a warning and MUST refuse to verify a user when
-   they detect that the user has a device with the same ID as a cross-signing key.
+   they detect that the user has a device with the same ID as a cross-signing
+   key.
 
 A user's user-signing and self-signing keys are intended to be easily
 replaceable if they are compromised by re-issuing a new key signed by
-the user's master key and possibly by re-verifying devices or users.
+the user's master signing key and possibly by re-verifying devices or users.
 However, doing so relies on the user being able to notice when their
 keys have been compromised, and it involves extra work for the user, and
 so although clients do not have to treat the private parts as
-sensitively as the master key, clients should still make efforts to
+sensitively as the master signing key, clients should still make efforts to
 store the private part securely, or not store it at all. Clients will
 need to balance the security of the keys with the usability of signing
 users and devices when performing key verification.
 
 To avoid leaking of social graphs, servers will only allow users to see:
 
--   signatures made by the user's own master, self-signing or
+-   signatures made by the user's own master signing, self-signing or
     user-signing keys,
 -   signatures made by the user's own devices about their own master
     key,
 -   signatures made by other users' self-signing keys about their
     respective devices,
--   signatures made by other users' master keys about their respective
+-   signatures made by other users' master signing keys about their respective
     self-signing key, or
 -   signatures made by other users' devices about their respective
-    master keys.
+    master signing keys.
 
 Users will not be able to see signatures made by other users'
 user-signing keys.
@@ -1124,7 +1220,7 @@ The process between Alice and Bob verifying each other would be:
    framework as described above.
 3. Alice's client displays a QR code that Bob is able to scan if Bob's client
    indicated the ability to scan, an option to scan Bob's QR code if her client
-   is able to scan.  Bob's client prompts displays a QR code that Alice can
+   is able to scan. Bob's client displays a QR code that Alice can
    scan if Alice's client indicated the ability to scan, and an option to scan
    Alice's QR code if his client is able to scan. The format for the QR code
    is described below. Other options, like starting SAS Emoji verification,
@@ -1193,24 +1289,24 @@ The binary segment MUST be of the following form:
 - one byte indicating the QR code verification mode.  Should be one of the
   following values:
   - `0x00` verifying another user with cross-signing
-  - `0x01` self-verifying in which the current device does trust the master key
+  - `0x01` self-verifying in which the current device does trust the master signing key
   - `0x02` self-verifying in which the current device does not yet trust the
-    master key
+    master signing key
 - the event ID or `transaction_id` of the associated verification
   request event, encoded as:
   - two bytes in network byte order (big-endian) indicating the length in
     bytes of the ID as a UTF-8 string
   - the ID encoded as a UTF-8 string
 - the first key, as 32 bytes.  The key to use depends on the mode field:
-  - if `0x00` or `0x01`, then the current user's own master cross-signing public key
+  - if `0x00` or `0x01`, then the current user's own master signing public key
   - if `0x02`, then the current device's Ed25519 signing key
 - the second key, as 32 bytes.  The key to use depends on the mode field:
   - if `0x00`, then what the device thinks the other user's master
-    cross-signing public key is
+    public key is
   - if `0x01`, then what the device thinks the other device's Ed25519 signing
     public key is
-  - if `0x02`, then what the device thinks the user's master cross-signing public
-    key is
+  - if `0x02`, then what the device thinks the user's master signing public key
+    is
 - a random shared secret, as a sequence of bytes.  It is suggested to use a secret
   that is about 8 bytes long.  Note: as we do not share the length of the
   secret, and it is not a fixed size, clients will just use the remainder of
@@ -1218,17 +1314,17 @@ The binary segment MUST be of the following form:
 
 For example, if Alice displays a QR code encoding the following binary data:
 
-```
+```nohighlight
       "MATRIX"    |ver|mode| len   | event ID
  4D 41 54 52 49 58  02  00   00 2D   21 41 42 43 44 ...
-| user's cross-signing key    | other user's cross-signing key | shared secret
+| the first key               | the second key              | shared secret
   00 01 02 03 04 05 06 07 ...   10 11 12 13 14 15 16 17 ...   20 21 22 23 24 25 26 27
 ```
 
-this indicates that Alice is verifying another user (say Bob), in response to
-the request from event "$ABCD...", her cross-signing key is
+Mode `0x00` indicates that Alice is verifying another user (say Bob), in
+response to the request from event "$ABCD...", her master signing key is
 `0001020304050607...` (which is "AAECAwQFBg..." in base64), she thinks that
-Bob's cross-signing key is `1011121314151617...` (which is "EBESExQVFh..." in
+Bob's master signing key is `1011121314151617...` (which is "EBESExQVFh..." in
 base64), and the shared secret is `2021222324252627` (which is "ICEiIyQlJic" in
 base64).
 
@@ -1300,8 +1396,8 @@ one of its variants.
 Clients must only store keys in backups after they have ensured that the
 `auth_data` is trusted. This can be done either by:
 
-- checking that it is signed by the user's [master cross-signing
-  key](#cross-signing) or by a verified device belonging to the same user, or
+- checking that it is signed by the user's [master signing key](#cross-signing)
+  or by a verified device belonging to the same user, or
 - deriving the public key from a private key that it obtained from a trusted
   source. Trusted sources for the private key include the user entering the
   key, retrieving the key stored in [secret storage](#secret-storage), or
@@ -1457,8 +1553,8 @@ readers without adding any useful extra information.
 ##### `m.olm.v1.curve25519-aes-sha2`
 
 The name `m.olm.v1.curve25519-aes-sha2` corresponds to version 1 of the
-Olm ratchet, as defined by the [Olm
-specification](http://matrix.org/docs/spec/olm.html). This uses:
+Olm ratchet, as defined by the [Olm specification](/olm-megolm/olm).
+This uses:
 
 -   Curve25519 for the initial key agreement.
 -   HKDF-SHA-256 for ratchet key derivation.
@@ -1512,40 +1608,11 @@ message.
 
 The plaintext payload is of the form:
 
-```json
-{
-  "type": "<type of the plaintext event>",
-  "content": "<content for the plaintext event>",
-  "sender": "<sender_user_id>",
-  "recipient": "<recipient_user_id>",
-  "recipient_keys": {
-    "ed25519": "<our_ed25519_key>"
-  },
-  "keys": {
-    "ed25519": "<sender_ed25519_key>"
-  }
-}
-```
+{{% definition path="api/client-server/definitions/olm_payload" %}}
 
 The type and content of the plaintext message event are given in the
 payload.
 
-Other properties are included in order to prevent an attacker from
-publishing someone else's curve25519 keys as their own and subsequently
-claiming to have sent messages which they didn't. `sender` must
-correspond to the user who sent the event, `recipient` to the local
-user, and `recipient_keys` to the local ed25519 key.
-
-Clients must confirm that the `sender_key` property in the cleartext
-`m.room.encrypted` event body, and the `keys.ed25519` property in the
-decrypted plaintext, match the keys returned by
-[`/keys/query`](#post_matrixclientv3keysquery) for
-the given user. Clients must also verify the signature of the keys from the
-`/keys/query` response. Without this check, a client cannot be sure that
-the sender device owns the private part of the ed25519 key it claims to
-have in the Olm payload. This is crucial when the ed25519 key corresponds
-to a verified device.
-
 If a client has multiple sessions established with another device, it
 should use the session from which it last received and successfully
 decrypted a message. For these purposes, a session that has not received
@@ -1555,6 +1622,68 @@ maximum number of olm sessions that it will maintain for each device,
 and expiring sessions on a Least Recently Used basis. The maximum number
 of olm sessions maintained per device should be at least 4.
 
+###### Validation of incoming decrypted events
+
+{{% changed-in v="1.15" %}} Existing checks made more explicit, and checks for `sender_device_keys` added.
+
+After decrypting an incoming encrypted event, clients MUST apply the
+following checks:
+
+1.  The `sender` property in the decrypted content must match the
+    `sender` of the event.
+2.  The `keys.ed25519` property in the decrypted content must match
+    the `sender_key` property in the cleartext `m.room.encrypted`
+    event body.
+3.  The `recipient` property in the decrypted content must match
+    the user ID of the local user.
+4.  The `recipient_keys.ed25519` property in the decrypted content
+    must match the client device's [Ed25519 signing key](#device-keys).
+5.  Where `sender_device_keys` is present in the decrypted content:
+    1.  `sender_device_keys.user_id` must also match the `sender`
+        of the event.
+    2.  `sender_device_keys.keys.ed25519:<device_id>` must also match
+        the `sender_key` property in the cleartext `m.room.encrypted`
+        event body.
+    3.  `sender_device_keys.keys.curve25519:<device_id>` must match
+        the Curve25519 key used to establish the Olm session.
+    4.  The `sender_device_keys` structure must have a valid signature
+        from the key with ID `ed25519:<device_id>` (i.e., the sending
+        device's Ed25519 key).
+
+Any event that does not comply with these checks MUST be discarded.
+
+###### Verification of the sending user for incoming events
+
+{{% added-in v="1.15" %}}
+
+In addition, for each Olm session, clients MUST verify that the
+Curve25519 key used to establish the Olm session does indeed belong
+to the claimed `sender`. This requires a signed "device keys" structure
+for that Curve25519 key, which can be obtained in one of two ways:
+
+1.  An Olm message may be received with a `sender_device_keys` property
+    in the decrypted content.
+2.  The keys are returned via a [`/keys/query`](#post_matrixclientv3keysquery)
+    request. Note that both the Curve25519 key **and** the Ed25519 key in
+    the returned device keys structure must match those used in an
+    Olm-encrypted event as above. (In particular, the Ed25519 key must
+    be present in the **encrypted** content of an Olm-encrypted event
+    to prevent an attacker from claiming another user's Curve25519 key
+    as their own.)
+
+Ownership of the Curve25519 key is then established in one of two ways:
+
+1.  Via [cross-signing](#cross-signing). For this to be sufficient, the
+    device keys structure must be signed by the sender's self-signing key,
+    and that self-signing key must itself have been validated (either via
+    [explicit verification](#device-verification) or a "trust on first use" (TOFU) mechanism).
+2.  Via explicit verification of the device's Ed25519 signing key, as
+    contained in the device keys structure. This is no longer recommended.
+
+A failure to complete these verifications does not necessarily mean that
+the session is bogus; however it is the case that there is no proof that
+the claimed sender is accurate, and the user should be warned accordingly.
+
 ###### Recovering from undecryptable messages
 
 Occasionally messages may be undecryptable by clients due to a variety
@@ -1598,8 +1727,8 @@ This is due to a deprecation of the fields. See
 {{% changed-in v="1.3" %}}
 
 The name `m.megolm.v1.aes-sha2` corresponds to version 1 of the Megolm
-ratchet, as defined by the [Megolm
-specification](http://matrix.org/docs/spec/megolm.html). This uses:
+ratchet, as defined by the [Megolm specification](/olm-megolm/megolm).
+This uses:
 
 -   HMAC-SHA-256 for the hash ratchet.
 -   HKDF-SHA-256, AES-256 in CBC mode, and 8 byte truncated HMAC-SHA-256
@@ -1742,17 +1871,16 @@ property is required for inclusion, though previous versions of the
 specification did not have it. In addition to `/versions`, this can be
 a way to identify the server's support for fallback keys.
 
-
 | Parameter                        | Type              | Description                                                                                                            |
-|----------------------------------|--------------------|------------------------------------------------------------------------------------------------------------------------|
+|----------------------------------|-------------------|------------------------------------------------------------------------------------------------------------------------|
 | device_lists                     | DeviceLists       | Optional. Information on e2e device updates. Note: only present on an incremental sync.                                |
-| device_one_time_keys_count       | {string: integer}  | Optional. For each key algorithm, the number of unclaimed one-time keys currently held on the server for this device.  If an algorithm is unlisted, the count for that algorithm is assumed to be zero.  If this entire parameter is missing, the count for all algorithms is assumed to be zero.  |
+| device_one_time_keys_count       | {string: integer} | **Required if any unclaimed one-time keys exist.** For each key algorithm, the number of unclaimed one-time keys currently held on the server for this device. If the count for an algorithm is zero, servers MAY omit that algorithm. If the count for all algorithms is zero, servers MAY omit this parameter entirely. |
 | device_unused_fallback_key_types | [string]          | **Required.** The unused fallback key algorithms.                                                                      |
 
 `DeviceLists`
 
 | Parameter | Type     | Description                                                                                                                                                      |
-|------------|-----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+|-----------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | changed   | [string] | List of users who have updated their device identity or cross-signing keys, or who now share an encrypted room with the client since the previous sync response. |
 | left      | [string] | List of users with whom we do not share any encrypted rooms anymore since the previous sync response.                                                            |
 

content/client-server-api/modules/event_replacements.md

@@ -195,7 +195,7 @@ given event (for example, if an event is edited multiple times). These should
 be [aggregated](#aggregations-of-child-events) by the homeserver.
 
 The aggregation format of `m.replace` relationships gives the **most recent**
-replacement event, formatted [as normal](#room-event-format).
+valid replacement event, formatted [as normal](#room-event-format).
 
 The most recent event is determined by comparing `origin_server_ts`; if two or
 more replacement events have identical `origin_server_ts`, the event with the
@@ -268,6 +268,11 @@ Client authors are reminded to take note of the requirements for [Validity of
 replacement events](#validity-of-replacement-events), and to ignore any
 invalid replacement events that are received.
 
+Clients should render the content of the **most recent** valid replacement event. The
+most recent event is determined by comparing `origin_server_ts`; if two or more
+replacement events have identical `origin_server_ts`, the event with the
+lexicographically largest `event_id` is treated as more recent.
+
 ##### Permalinks
 
 When creating [links](/appendices/#uris) to events (also known as permalinks),
@@ -364,7 +369,7 @@ property under `m.new_content`.
 
 A particular constraint applies to events which replace a [reply](#rich-replies):
 in contrast to the original reply, there should be no `m.in_reply_to` property
-in the the `m.relates_to` object, since it would be redundant (see
+in the `m.relates_to` object, since it would be redundant (see
 [Applying `m.new_content`](#applying-mnew_content) above, which notes that the
 original event's `m.relates_to` is preserved), as well as being contrary to the
 spirit of the event relationships mechanism which expects only one "parent" per

content/client-server-api/modules/guest_access.md

@@ -63,7 +63,8 @@ for sending events:
 The following API endpoints are allowed to be accessed by guest accounts
 for their own account maintenance:
 
-* [PUT /profile/{userId}/displayname](#put_matrixclientv3profileuseriddisplayname)
+* [PUT /profile/{userId}/displayname](#put_matrixclientv3profileuseridkeyname). Guest users may only modify their display name; other profile fields may not be changed.
+* {{% added-in v="1.16" %}} [DELETE /profile/{userId}/displayname](#delete_matrixclientv3profileuseridkeyname). Again, guest users may delete their display name but not other profile fields.
 * [GET /devices](#get_matrixclientv3devices)
 * [GET /devices/{deviceId}](#get_matrixclientv3devicesdeviceid)
 * [PUT /devices/{deviceId}](#put_matrixclientv3devicesdeviceid)

content/client-server-api/modules/history_visibility.md

@@ -16,8 +16,8 @@ The four options for the `m.room.history_visibility` event are:
 
 -   `world_readable` - All events while this is the
     `m.room.history_visibility` value may be shared by any participating
-    homeserver with anyone, regardless of whether they have ever joined
-    the room.
+    homeserver with any authenticated user, regardless of whether they have
+    ever joined the room. This includes [guest users](#guest-access).
 -   `shared` - Previous events are always accessible to newly joined
     members. All events in the room are accessible, even those sent when
     the member was not a part of the room.
@@ -44,7 +44,7 @@ setting at that time was more restrictive.
 #### Client behaviour
 
 Clients may want to display a notice that events may be read by
-non-joined people if the history visibility is set to `world_readable`.
+non-joined users if the history visibility is set to `world_readable`.
 
 #### Server behaviour
 

content/client-server-api/modules/instant_messaging.md

@@ -84,6 +84,10 @@ Additionally, web clients should ensure that *all* `a` tags get a
 `rel="noopener"` to prevent the target page from referencing the
 client's tab/window.
 
+{{% added-in v="1.18" %}} Clients that support rendering numbered lists via the
+`ol` tag MUST also support the `start` attribute in order to prevent loss of
+meaning of a message due to the numbering of list items.
+
 Tags must not be nested more than 100 levels deep. Clients should only
 support the subset of tags they can render, falling back to other
 representations of the tags where possible. For example, a client may
@@ -119,7 +123,7 @@ Clients SHOULD verify the structure of incoming events to ensure that
 the expected keys exist and that they are of the right type. Clients can
 discard malformed events or display a placeholder message to the user.
 Redacted `m.room.message` events MUST be removed from the client. This
-can either be replaced with placeholder text (e.g. "\[REDACTED\]") or
+can either be replaced with placeholder text (e.g. "[REDACTED]") or
 the redacted message can be removed entirely from the messages view.
 
 Events which have attachments (e.g. `m.image`, `m.file`) SHOULD be

content/client-server-api/modules/invite_permission.md

@@ -0,0 +1,51 @@
+
+### Invite permission
+
+{{% added-in v="1.18" %}}
+
+Users may want to control who is allowed to invite them to new rooms. This module defines how
+clients and servers can implement invite permission.
+
+#### Account data
+
+{{% event event="m.invite_permission_config" %}}
+
+#### Client behaviour
+
+To reject invites from all users automatically, clients MAY add an [`m.invite_permission_config`](#minvite_permission_config)
+event in the user's [account data](#client-config) with the `default_action` property set to
+`block`. To stop rejecting all invites, the same event without the `default_action` property MUST be
+added to the account data.
+
+When the `default_action` field is unset, other parts of the specification might still have effects
+on invites seen by clients, like [ignoring users](#ignoring-users).
+
+Attempting to send an invite to a user that blocks invites will result in an error response with the
+`M_INVITE_BLOCKED` error code.
+
+#### Server behaviour
+
+When invites to a given user are blocked, the user's homeserver MUST respond to the following
+endpoints with an HTTP 403 status code, with the Matrix error code `M_INVITE_BLOCKED`, if the user
+is invited:
+
+* [`PUT /_matrix/federation/v1/invite/{roomId}/{eventId}`](/server-server-api/#put_matrixfederationv1inviteroomideventid)
+* [`PUT /_matrix/federation/v2/invite/{roomId}/{eventId}`](/server-server-api/#put_matrixfederationv2inviteroomideventid)
+* [`POST /_matrix/client/v3/rooms/{roomId}/invite`](#post_matrixclientv3roomsroomidinvite)
+* [`POST /_matrix/client/v3/createRoom`](#post_matrixclientv3createroom), due to a user in the
+  `invite` list. It is possible for one of the invited users to be rejected whilst the room creation
+  as a whole succeeds.
+* [`PUT /_matrix/client/v3/rooms/{roomId}/state/m.room.member/{stateKey}`](#put_matrixclientv3roomsroomidstateeventtypestatekey),
+  when the `membership` is set to `invite`.
+
+In addition, invite events for this user already in the database, or received over federation, MUST
+NOT be served over client synchronisation endpoints such as [`GET /sync`](#get_matrixclientv3sync).
+ 
+Servers MAY return any suppressed invite events over `GET /sync` if invite blocking is later
+disabled.
+ 
+Other endpoints, such as [`GET /rooms/{roomId}/state`](#get_matrixclientv3roomsroomidstate), are not
+affected by invite blocking: invite events are returned as normal.
+ 
+The Application Services API is also unaffected by invite blocking: invite events are sent over
+[`PUT /_matrix/app/v1/transactions/{txnId}`](/application-service-api/#put_matrixappv1transactionstxnid).

content/client-server-api/modules/mentions.md

@@ -3,6 +3,9 @@
 
 {{% changed-in v="1.7" %}}
 
+{{% changed-in v="1.17" %}}: the legacy push rules that looked for mentions in
+the `body` of the event were removed.
+
 This module allows users to "mention" other users and rooms within a room event.
 This is primarily used as an indicator that the recipient should receive a notification
 about the event.
@@ -38,19 +41,18 @@ encrypted as normal. To properly process mentions in encrypted rooms, events
 must be decrypted first. See [receiving notifications](#receiving-notifications).
 {{% /boxes/warning %}}
 
-Note that, for backwards compatibility, push rules such as [`.m.rule.contains_display_name`](#_m_rule_contains_display_name),
-[`.m.rule.contains_user_name`](#_m_rule_contains_user_name), and
-[`.m.rule.roomnotif`](#_m_rule_roomnotif) continue to  match if the `body` of
-the event contains the user's display name or ID. To avoid unintentional notifications,
-**it is recommended that clients include a `m.mentions` property on each event**.
-(If there are no mentions to include it can be an empty object.)
-
-{{% boxes/rationale %}}
+{{% boxes/note %}}
 In previous versions of the specification, mentioning users was done by
 including the user's display name or the localpart of their Matrix ID and room
 mentions were done by including the string "@room" in the plaintext `body` of
-the event. This was prone to confusing and buggy behaviour.
-{{% /boxes/rationale %}}
+the event. When the `m.mentions` field was introduced, those push rules were
+disabled if the `m.mentions` field was present.
+
+To avoid unintentional notifications with clients and servers that still use
+those push rules, **it is recommended that clients still include a `m.mentions`
+property on each event**. (If there are no mentions to include it can be an
+empty object.)
+{{% /boxes/note %}}
 
 #### Client behaviour
 

content/client-server-api/modules/moderation_policies.md

@@ -18,8 +18,9 @@ the entity making the decisions on filtering is best positioned to
 interpret the rules how it sees fit.
 
 Moderation policy lists are stored as room state events. There are no
-restrictions on how the rooms can be configured (they could be public,
-private, encrypted, etc).
+restrictions on how the rooms can be configured in terms of
+[join rules](#mroomjoin_rules), [history visibility](#room-history-visibility),
+encryption, etc.
 
 There are currently 3 kinds of entities which can be affected by rules:
 `user`, `server`, and `room`. All 3 are described with

content/client-server-api/modules/presence.md

@@ -68,5 +68,7 @@ will cause the server to automatically set their presence to `online`.
 
 #### Security considerations
 
-Presence information is shared with all users who share a room with the
-target user. In large public rooms this could be undesirable.
+Presence information is published to all users who share a room with the
+target user. If the target user is a member of a room with a `public`
+[join rule](#mroomjoin_rules), any other user in the federation is
+able to gain access to the target user's presence. This could be undesirable.

content/client-server-api/modules/push.md

@@ -1,7 +1,7 @@
 
 ### Push Notifications
 
-```
+```nohighlight
                                    +--------------------+  +-------------------+
                   Matrix HTTP      |                    |  |                   |
              Notification Protocol |   App Developer    |  |   Device Vendor   |
@@ -83,7 +83,7 @@ Push Ruleset
 :   A push ruleset *scopes a set of rules according to some criteria*. For
     example, some rules may only be applied for messages from a particular
     sender, a particular room, or by default. The push ruleset contains the
-    entire set of scopes and rules.
+    entire set of rules.
 
 #### Push Rules
 
@@ -91,10 +91,8 @@ A push rule is a single rule that states under what *conditions* an
 event should be passed onto a push gateway and *how* the notification
 should be presented. There are different "kinds" of push rules and each
 rule has an associated priority. Every push rule MUST have a `kind` and
-`rule_id`. The `rule_id` is a unique string within the kind of rule and
-its' scope: `rule_ids` do not need to be unique between rules of the
-same kind on different devices. Rules may have extra keys depending on
-the value of `kind`.
+`rule_id`. The `rule_id` is a unique string within the kind of rule.
+Rules may have extra keys depending on the value of `kind`.
 
 The different `kind`s of rule, in the order that they are checked, are:
 
@@ -382,6 +380,9 @@ The following `alt_aliases` values will NOT match:
 
 **`contains_display_name`**
 
+{{% changed-in v="1.17" %}}: this condition is deprecated and **should not be
+used in new push rules**.
+
 This matches messages where `content.body` contains the owner's display name in
 that room. This is a separate condition because display names may change and as such
 it would be hard to maintain a rule that matched the user's display name. This
@@ -413,6 +414,9 @@ Parameters:
 
 #### Predefined Rules
 
+{{% changed-in v="1.17" %}}: the legacy default push rules that looked for
+mentions in the `body` of the event were removed.
+
 Homeservers can specify "server-default rules". They operate at a lower
 priority than "user-defined rules", except for the `.m.rule.master` rule
 which has always a higher priority than any other rule. The `rule_id`
@@ -557,41 +561,6 @@ Definition:
 }
 ```
 
-<a id="_m_rule_contains_display_name"></a> **`.m.rule.contains_display_name`**
-
-{{% changed-in v="1.7" %}}
-
-As of `v1.7`, this rule is deprecated and **should only be enabled if the event
-does not have an [`m.mentions` property](#definition-mmentions)**.
-
-Matches any message whose content contains the user's current display name
-in the room in which it was sent.
-
-Definition:
-
-```json
-{
-    "rule_id": ".m.rule.contains_display_name",
-    "default": true,
-    "enabled": true,
-    "conditions": [
-        {
-            "kind": "contains_display_name"
-        }
-    ],
-    "actions": [
-        "notify",
-        {
-            "set_tweak": "sound",
-            "value": "default"
-        },
-        {
-            "set_tweak": "highlight"
-        }
-    ]
-}
-```
-
 <a id="_m_rule_is_room_mention"></a> **`.m.rule.is_room_mention`**
 
 {{% added-in v="1.7" %}}
@@ -626,44 +595,6 @@ Definition:
 }
 ```
 
-<a id="_m_rule_roomnotif"></a> **`.m.rule.roomnotif`**
-
-{{% changed-in v="1.7" %}}
-
-As of `v1.7`, this rule is deprecated and **should only be enabled if the event
-does not have an [`m.mentions` property](#definition-mmentions)**.
-
-Matches any message from a sender with the proper power level whose content
-contains the text `@room`, signifying the whole room should be notified of
-the event.
-
-Definition:
-
-```json
-{
-    "rule_id": ".m.rule.roomnotif",
-    "default": true,
-    "enabled": true,
-    "conditions": [
-        {
-            "kind": "event_match",
-            "key": "content.body",
-            "pattern": "@room"
-        },
-        {
-            "kind": "sender_notification_permission",
-            "key": "room"
-        }
-    ],
-    "actions": [
-        "notify",
-        {
-            "set_tweak": "highlight"
-        }
-    ]
-}
-```
-
 **<a id="mruletombstone"></a>`.m.rule.tombstone`**
 
 Matches any state event whose type is `m.room.tombstone`. This is
@@ -776,39 +707,6 @@ Definition:
 }
 ```
 
-##### Default Content Rules
-
-<a id="_m_rule_contains_user_name"></a> **`.m.rule.contains_user_name`**
-
-{{% changed-in v="1.7" %}}
-
-As of `v1.7`, this rule is deprecated and **should only be enabled if the event
-does not have an [`m.mentions` property](#definition-mmentions)**.
-
-Matches any message whose content contains the local part of the user's
-Matrix ID, separated by word boundaries.
-
-Definition (as a `content` rule):
-
-```json
-{
-    "rule_id": ".m.rule.contains_user_name",
-    "default": true,
-    "enabled": true,
-    "pattern": "[the local part of the user's Matrix ID]",
-    "actions": [
-        "notify",
-        {
-            "set_tweak": "sound",
-            "value": "default"
-        },
-        {
-            "set_tweak": "highlight"
-        }
-    ]
-}
-```
-
 ##### Default Underride Rules
 
 **`.m.rule.call`**

content/client-server-api/modules/receipts.md

@@ -214,7 +214,7 @@ before delivering them to clients.
 Some receipts are sent across federation as EDUs with type `m.receipt`. The
 format of the EDUs are:
 
-```
+```nohighlight
 {
     <room_id>: {
         <receipt_type>: {

content/client-server-api/modules/recent_emoji.md

@@ -0,0 +1,40 @@
+### Recently used emoji
+
+{{% added-in v="1.18" %}}
+
+This module enables clients to track a user's cumulated emoji usage across different
+devices. The data is stored in the [`m.recent_emoji`](#mrecent_emoji)
+global [account data](#client-config) and can, among other things, be used to
+generate recommendations in emoji pickers.
+
+#### Events
+
+{{% event event="m.recent_emoji" %}}
+
+#### Client behaviour
+
+What exactly constitutes trackable emoji usage is left as an implementation detail
+for clients. It is RECOMMENDED to include sending emoji in both messages and
+annotations.
+
+When an emoji is used, the sending client moves (or adds) it to the beginning of
+the `recent_emoji` array and increments (or initializes) its counter. This keeps
+the array ordered by last usage time which facilitates evaluating the data. How
+exactly the client evaluates and uses the collected data is deliberately left
+unspecified.
+
+To prevent excessive growth of the event as new emoji are being used, clients
+SHOULD limit the length of the `recent_emoji` array by dropping elements from
+its end. A RECOMMENDED maximum length is 100 emoji.
+
+To enable future extension, clients MUST tolerate and preserve array elements
+within `recent_emoji` regardless of whether they understand or support the
+contained `emoji` value. This means ignoring entries with unrecognised values
+of `emoji` when deciding what to display to the user while retaining them when
+modifying the array (unless the modification is for truncation).
+
+To prevent undefined behavior, clients SHOULD remove array elements that
+don't conform to the event schema such as elements with negative counters.
+
+
+

content/client-server-api/modules/rich_replies.md

@@ -20,9 +20,9 @@ Previously, a rich reply could only reference another `m.room.message` event.
 {{% boxes/note %}}
 {{% changed-in v="1.13" %}}
 In previous versions of the specification, rich replies could include a fallback
-representation of the original message message in the `body` (using a prefix
-sequence) and `formatted_body` (using a custom HTML element) for clients that do
-not support rich replies. This is no longer the case, but clients SHOULD still
+representation of the original message in the `body` (using a prefix sequence)
+and `formatted_body` (using a custom HTML element) for clients that do not
+support rich replies. This is no longer the case, but clients SHOULD still
 remove this fallback before rendering the event.
 
 To strip the fallback on the `body`, the client should iterate over each
@@ -64,11 +64,16 @@ as if it were to be under `m.relates_to` directly instead.
 
 #### Mentioning the replied to user
 
-In order to notify users of the reply, it may be desirable to include the `sender`
-of the replied to event and any users mentioned in that event. See
-[user and room mentions](#user-and-room-mentions) for additional information.
+{{% boxes/note %}}
+{{% changed-in v="1.16" %}}
+Clients SHOULD no longer propagate mentioned users in the replied to event.
+{{% /boxes/note %}}
 
-An example including mentioning the original sender and other users:
+In order to notify users of the reply, it MAY be desirable to include the `sender`
+of the replied to event. See [user and room mentions](#user-and-room-mentions) for
+additional information.
+
+An example including mentioning the original sender:
 
 ```json
 {
@@ -83,8 +88,6 @@ An example including mentioning the original sender and other users:
       "user_ids": [
         // The sender of $another_event.
         "@alice:example.org",
-        // Another Matrix ID copied from the m.mentions property of $another_event.
-        "@bob:example.org"
       ]
     }
   },

content/client-server-api/modules/room_previews.md

@@ -6,7 +6,7 @@ It is sometimes desirable to offer a preview of a room, where a user can
 This can be particularly effective when combined with [Guest Access](#guest-access).
 
 Previews are implemented via the `world_readable` [Room History
-Visibility](#room-history-visibility). setting, along with a special version of the [GET
+Visibility](#room-history-visibility) setting, along with a special version of the [GET
 /events](#get_matrixclientv3events) endpoint.
 
 #### Client behaviour

content/client-server-api/modules/room_upgrades.md

@@ -36,6 +36,17 @@ server:
     previous room, no `type` is specified on the new room's create event
     either.
 
+{{% boxes/note %}}
+{{% added-in v="1.16" %}} If both the new and old [room version](/rooms) support
+additional creators, the server will not transfer those additional creators automatically.
+They must be explicitly set during the `/upgrade` call.
+{{% /boxes/note %}}
+
+{{% boxes/note %}}
+{{% added-in v="1.16" %}} When upgrading to room version 12 or later, the `event_id` property inside
+`predecessor` MAY be omitted.
+{{% /boxes/note %}}
+
 3.  Replicates transferable state events to the new room. The exact
     details for what is transferred is left as an implementation detail,
     however the recommended state events to transfer are:

content/client-server-api/modules/search.md

@@ -26,9 +26,10 @@ on certain keys of certain event types.
 
 The supported keys to search over are:
 
--   `content.body` in `m.room.message`
--   `content.name` in `m.room.name`
--   `content.topic` in `m.room.topic`
+-   `content.body` in [`m.room.message`](/client-server-api/#mroommessage)
+-   `content.name` in [`m.room.name`](/client-server-api/#mroomname)
+-   In [`m.room.topic`](/client-server-api/#mroomtopic), `content.topic`
+    as well as the `body` of the `text/plain` representation in `content['m.topic']`.
 
 The search will *not* include rooms that are end to end encrypted.
 

content/client-server-api/modules/secrets.md

@@ -58,8 +58,8 @@ available on all their clients. Unless the user specifies otherwise,
 clients will try to use the default key to decrypt secrets.
 
 Clients that want to present a simplified interface to users by not supporting
-multiple keys should use the default key if one is specified. If not default
-key is specified, the client may behave as if there is no key is present at
+multiple keys should use the default key if one is specified. If no default
+key is specified, the client may behave as if no key is present at
 all. When such a client creates a key, it should mark that key as being the
 default key.
 
@@ -157,7 +157,7 @@ Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`:
 
 `org.example.some.secret`:
 
-```
+```nohighlight
 {
   "encrypted": {
     "key_id_1": {
@@ -177,7 +177,7 @@ and the key descriptions for the keys would be:
 
 `m.secret_storage.key.key_id_1`:
 
-```
+```nohighlight
 {
   "name": "Some key",
   "algorithm": "m.secret_storage.v1.aes-hmac-sha2",
@@ -187,7 +187,7 @@ and the key descriptions for the keys would be:
 
 `m.secret_storage.key.key_id_2`:
 
-```
+```nohighlight
 {
   "name": "Some other key",
   "algorithm": "m.secret_storage.v1.aes-hmac-sha2",
@@ -199,7 +199,7 @@ If `key_id_1` is the default key, then we also have:
 
 `m.secret_storage.default_key`:
 
-```
+```nohighlight
 {
   "key": "key_id_1"
 }
@@ -294,7 +294,7 @@ in the `iterations` parameter.
 
 Example:
 
-```
+```nohighlight
 {
     "passphrase": {
         "algorithm": "m.pbkdf2",

content/client-server-api/modules/spaces.md

@@ -2,8 +2,8 @@
 
 {{% added-in v="1.2" %}}
 
-Often used to group rooms of similar subject matter (such as a public "Official
-matrix.org rooms" space or personal "Work stuff" space), spaces are a way to
+Often used to group rooms of similar subject matter (such as an "Official
+matrix.org rooms" space or a "Work stuff" space), spaces are a way to
 organise rooms while being represented as rooms themselves.
 
 A space is defined by the [`m.space` room type](#types), making it known as a
@@ -18,11 +18,11 @@ In the default power level structure, this would be `100`. Clients might wish to
 go a step further and explicitly ignore notification counts on space-rooms.
 
 Membership of a space is defined and controlled by the existing mechanisms which
-govern a room: [`m.room.member`](#mroommember), [`m.room.history_visibility`](#mroomhistory_visibility),
-and [`m.room.join_rules`](#mroomjoin_rules). Public spaces are encouraged to have
-a similar setup to public rooms: `world_readable` history visibility, published
-canonical alias, and suitably public `join_rule`. Invites, including third-party
-invites, still work just as they do in normal rooms as well.
+govern a room: [`m.room.member`](/client-server-api#mroommember), [`m.room.history_visibility`](/client-server-api#mroomhistory_visibility),
+and [`m.room.join_rules`](/client-server-api#mroomjoin_rules). Canonical aliases and invites, including
+third-party invites, still work just as they do in normal rooms as well. Furthermore,
+spaces can also be published in the [room directory](/client-server-api#published-room-directory) to make them
+discoverable.
 
 All other aspects of regular rooms are additionally carried over, such as the
 ability to set arbitrary state events, hold room account data, etc. Spaces are
@@ -58,7 +58,7 @@ parent to the room. The `state_key` for the event is the child room's ID.
 
 For example, to achieve the following:
 
-```
+```nohighlight
 #space:example.org
     #general:example.org (!abcdefg:example.org)
     !private:example.org
@@ -87,10 +87,9 @@ the state of `#space:example.org` would consist of:
 }
 ```
 
-No state events in the child rooms themselves would be required (though they
-can also be present). This allows for users
-to define personal/private spaces to organise their own rooms without needing explicit
-permission from the room moderators/admins.
+No state events in the child rooms themselves would be required (though they can also
+be present). This allows for users to define spaces without needing explicit permission
+from the room moderators/admins.
 
 Child rooms can be removed from a space by omitting the `via` key of `content` on the
 relevant state event, such as through redaction or otherwise clearing the `content`.

content/client-server-api/modules/sso_login.md

@@ -6,9 +6,10 @@ allow users to log into applications via a single web-based
 authentication portal. Examples include OpenID Connect, "Central
 Authentication Service" (CAS) and SAML.
 
-This module allows a Matrix homeserver to delegate user authentication
-to an external authentication server supporting one of these protocols.
-In this process, there are three systems involved:
+This module allows a Matrix homeserver that supports the [legacy authentication
+API](#legacy-api) to delegate user authentication to an external authentication
+server supporting one of these protocols. In this process, there are three
+systems involved:
 
 -   A Matrix client, using the APIs defined in this specification, which
     is seeking to authenticate a user to a Matrix homeserver.
@@ -24,7 +25,7 @@ used to communicate with the authentication server. Different Matrix
 homeserver implementations might support different SSO protocols.
 
 Clients and homeservers implementing the SSO flow will need to consider
-both [login](#login) and [user-interactive authentication](#user-interactive-authentication-api). The flow is
+both [login](#legacy-login) and [user-interactive authentication](#user-interactive-authentication-api). The flow is
 similar in both cases, but there are slight differences.
 
 Typically, SSO systems require a single "callback" URI to be configured
@@ -66,7 +67,7 @@ opening an embedded web view.
 
 These steps are illustrated as follows:
 
-```
+```nohighlight
     Matrix Client                        Matrix Homeserver      Auth Server
         |                                       |                   |
         |-------------(0) GET /login----------->|                   |

content/client-server-api/modules/third_party_invites.md

@@ -5,8 +5,8 @@ This module adds in support for inviting new members to a room where
 their Matrix user ID is not known, instead addressing them by a third-party
 identifier such as an email address. There are two flows here; one
 if a Matrix user ID is known for the third-party identifier, and one if
-not. Either way, the client calls [`/invite`](#post_matrixclientv3roomsroomidinvite) with the details of the
-third-party identifier.
+not. Either way, the client calls [`/invite`](#thirdparty_post_matrixclientv3roomsroomidinvite)
+with the details of the third-party identifier.
 
 The homeserver asks the identity server whether a Matrix user ID is
 known for that identifier:
@@ -37,12 +37,14 @@ A client asks a server to invite a user by their third-party identifier.
 
 #### Server behaviour
 
-Upon receipt of an [`/invite`](#post_matrixclientv3roomsroomidinvite), the server is expected to look up the
-third-party identifier with the provided identity server. If the lookup
-yields a result for a Matrix User ID then the normal invite process can
-be initiated. This process ends up looking like this:
+Upon receipt of an [`/invite`](#thirdparty_post_matrixclientv3roomsroomidinvite),
+the server is expected to look up the third-party identifier with the provided
+identity server by making a call to [`/_matrix/identity/v2/lookup`](/identity-service-api/#post_matrixidentityv2lookup).
+If the lookup yields a result for a Matrix User ID then the normal [invite
+process](/server-server-api/#inviting-to-a-room) can be initiated. This process
+ends up looking like this:
 
-```
+```nohighlight
     +---------+                         +-------------+                                    +-----------------+
     | Client  |                         | Homeserver  |                                    | IdentityServer  |
     +---------+                         +-------------+                                    +-----------------+
@@ -66,12 +68,13 @@ be initiated. This process ends up looking like this:
         |                                     |                                                    |
 ```
 
-However, if the lookup does not yield a bound User ID, the homeserver
-must store the invite on the identity server and emit a valid
-`m.room.third_party_invite` event to the room. This process ends up
-looking like this:
+However, if the lookup does not yield a bound User ID, the homeserver must store
+the invite on the identity server with a call to
+[`/_matrix/identity/v2/store-invite`](/identity-service-api/#post_matrixidentityv2store-invite)
+and emit a valid [`m.room.third_party_invite`](#mroomthird_party_invite) event
+to the room. This process ends up looking like this:
 
-```
+```nohighlight
     +---------+                         +-------------+                                               +-----------------+
     | Client  |                         | Homeserver  |                                               | IdentityServer  |
     +---------+                         +-------------+                                               +-----------------+
@@ -101,15 +104,18 @@ looking like this:
         |                                     |                                                               |
 ```
 
-All homeservers MUST verify the signature in the event's
-`content.third_party_invite.signed` object.
+The third-party user will then need to verify their identity, which results in a
+request to [`/_matrix/federation/v1/3pid/onbind`](/server-server-api/#put_matrixfederationv13pidonbind)
+from the identity server to the homeserver that bound the third-party identifier
+to a user. The homeserver then exchanges the `m.room.third_party_invite` event
+in the room for a complete [`m.room.member`](#mroommember) event with
+`content.membership: invite` and a `content.third_party_invite` property for the
+user that has bound the third-party identifier. If the invitee is on a different
+homeserver than the inviting user, the invitee's homeserver makes a request to
+[`/_matrix/federation/v1/exchange_third_party_invite/{roomId}`](/server-server-api/#put_matrixfederationv1exchange_third_party_inviteroomid).
 
-The third-party user will then need to verify their identity, which
-results in a call from the identity server to the homeserver that bound
-the third-party identifier to a user. The homeserver then exchanges the
-`m.room.third_party_invite` event in the room for a complete
-`m.room.member` event for `membership: invite` for the user that has
-bound the third-party identifier.
+All homeservers MUST verify the signature in the `m.room.member` event's
+`content.third_party_invite.signed` object.
 
 If a homeserver is joining a room for the first time because of an
 `m.room.third_party_invite`, the server which is already participating
@@ -127,7 +133,7 @@ and an identity server IS, the full sequence for a third-party invite
 would look like the following. This diagram assumes H1 and H2 are
 residents of the room while H3 is attempting to join.
 
-```
+```nohighlight
     +-------+ +-----------------+         +-----+                                          +-----+           +-----+                      +-----+
     | UserA | | ThirdPartyUser  |         | H1  |                                          | H2  |           | H3  |                      | IS  |
     +-------+ +-----------------+         +-----+                                          +-----+           +-----+                      +-----+
@@ -193,8 +199,8 @@ at any time - the completion is not shown in the diagram.
 
 H1 MUST verify the request from H3 to ensure the `signed` property is
 correct as well as the `key_validity_url` as still being valid. This is
-done by making a request to the [identity server
-/isvalid](/identity-service-api/#get_matrixidentityv2pubkeyisvalid)
+done by making a request to the identity server's
+[`/pubkey/isvalid`](/identity-service-api/#get_matrixidentityv2pubkeyisvalid)
 endpoint, using the provided URL rather than constructing a new one. The
 query string and response for the provided URL must match the Identity
 Service Specification.

content/client-server-api/modules/threading.md

@@ -107,7 +107,7 @@ flag to `true`.
 ```
 
 {{% boxes/note %}}
-Clients which are acutely aware of threads (they do not render threads, but are otherwise
+Clients which are aware of threads (they do not render threads, but are otherwise
 aware of the feature existing in the spec) can treat rich replies to an event with a `rel_type`
 of `m.thread` as a threaded reply, for conversation continuity on the threaded client's side.
 
@@ -185,7 +185,7 @@ included under the `m.relations` property in `unsigned` for the thread root. For
 ```
 
 `latest_event` is the most recent event (topologically to the server) in the thread sent by an
-un-[ignored user](#ignoring-users).
+un-[ignored user](#ignoring-users). It should be serialized in the same form as the event itself.
 
 Note that, as in the example above, child events of the `latest_event` should
 themselves be aggregated and included under `m.relations` for that event. The

content/client-server-api/modules/voip_events.md

@@ -129,7 +129,7 @@ or not there have been any changes to the Matrix spec.
 
 A call is set up with message events exchanged as follows:
 
-```
+```nohighlight
     Caller                    Callee
     [Place Call]
     m.call.invite ----------->
@@ -144,7 +144,7 @@ A call is set up with message events exchanged as follows:
 
 Or a rejected call:
 
-```
+```nohighlight
     Caller                      Callee
     m.call.invite ------------>
     m.call.candidate --------->
@@ -202,11 +202,13 @@ specific user, and should be set to the Matrix user ID of that user. Invites
 without an `invitee` field are defined to be intended for any member of the
 room other than the sender of the event.
 
-Clients should consider an incoming call if they see a non-expired invite event where the `invitee` field is either
-absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their
-user's trust relationship with the callers and/or where the call was placed. As a starting point, it is
-suggested that clients ignore call invites from users in public rooms. It is strongly recommended that
-when clients do not ring for an incoming call invite, they still display the call invite in the room and
+Clients should consider an incoming call if they see a non-expired invite event
+where the `invitee` field is either absent or equal to their user's Matrix ID.
+They should, however, evaluate whether or not to ring based on their user's trust
+relationship with the callers and/or where the call was placed. As a starting
+point, it is RECOMMENDED that clients ignore call invites in rooms with a
+[join rule](#mroomjoin_rules) of `public`. When clients suppress ringing for an
+incoming call invite, they SHOULD still display the call invite in the room and
 annotate that it was ignored.
 
 ##### Glare

content/identity-service-api.md

@@ -2,16 +2,14 @@
 title: "Identity Service API"
 weight: 40
 type: docs
----
-
+description: |
   The Matrix client-server and server-server APIs are largely expressed in
-Matrix user identifiers. From time to time, it is useful to refer to
-users by other ("third-party") identifiers, or "3PID"s, e.g. their email
-address or phone number. This Identity Service Specification describes
-how mappings between third-party identifiers and Matrix user identifiers
-can be established, validated, and used. This description technically
-may apply to any 3PID, but in practice has only been applied
-specifically to email addresses and phone numbers.
+  Matrix user identifiers. Sometimes it is useful to refer to users by other
+  (“third-party”) identifiers such as email addresses or phone numbers. The
+  Identity Service API describes how mappings between 3PIDs and Matrix user
+  IDs can be established, validated, and used; in practice this has been
+  applied to email addresses and phone numbers.
+---
 
 ## General principles
 
@@ -72,54 +70,58 @@ the keys `error` and `errcode` MUST always be present.
 
 Some standard error codes are below:
 
-`M_NOT_FOUND`
-The resource requested could not be located.
-
-`M_MISSING_PARAMS`
-The request was missing one or more parameters.
-
-`M_INVALID_PARAM`
-The request contained one or more invalid parameters.
-
-`M_SESSION_NOT_VALIDATED`
-The session has not been validated.
-
-`M_NO_VALID_SESSION`
-A session could not be located for the given parameters.
-
-`M_SESSION_EXPIRED`
-The session has expired and must be renewed.
-
-`M_INVALID_EMAIL`
-The email address provided was not valid.
+<!-- Please keep the error codes below in alphabetical order -->
 
 `M_EMAIL_SEND_ERROR`
-There was an error sending an email. Typically seen when attempting to
+: There was an error sending an email. Typically seen when attempting to
 verify ownership of a given email address.
 
 `M_INVALID_ADDRESS`
-The provided third-party address was not valid.
+: The provided third-party address was not valid.
+
+`M_INVALID_EMAIL`
+: The email address provided was not valid.
+
+`M_INVALID_PARAM`
+: The request contained one or more invalid parameters.
+
+`M_MISSING_PARAMS`
+: The request was missing one or more parameters.
+
+`M_NO_VALID_SESSION`
+: A session could not be located for the given parameters.
+
+`M_NOT_FOUND`
+: The resource requested could not be located.
 
 `M_SEND_ERROR`
-There was an error sending a notification. Typically seen when
+: There was an error sending a notification. Typically seen when
 attempting to verify ownership of a given third-party address.
 
-`M_UNRECOGNIZED`
-The request contained an unrecognised value, such as an unknown token or
-medium.
+`M_SESSION_EXPIRED`
+: The session has expired and must be renewed.
 
-This is also used as the response if a server did not understand the request.
-This is expected to be returned with a 404 HTTP status code if the endpoint is
-not implemented or a 405 HTTP status code if the endpoint is implemented, but
-the incorrect HTTP method is used.
+`M_SESSION_NOT_VALIDATED`
+: The session has not been validated.
 
 `M_THREEPID_IN_USE`
-The third-party identifier is already in use by another user. Typically
+: The third-party identifier is already in use by another user. Typically
 this error will have an additional `mxid` property to indicate who owns
 the third-party identifier.
 
 `M_UNKNOWN`
-An unknown error has occurred.
+: An unknown error has occurred.
+
+`M_UNRECOGNIZED`
+: The request contained an unrecognised value, such as an unknown token or
+medium.
+
+: This is also used as the response if a server did not understand the request.
+This is expected to be returned with a 404 HTTP status code if the endpoint is
+not implemented or a 405 HTTP status code if the endpoint is implemented, but
+the incorrect HTTP method is used.
+
+<!-- Please keep the error codes above in alphabetical order -->
 
 ## Privacy
 

content/olm-megolm/_index.md

@@ -0,0 +1,10 @@
+---
+title: "Olm & Megolm"
+weight: 61
+type: docs
+---
+
+Matrix uses the Olm and Megolm cryptographic ratchets for [end-to-end encryption](../client-server-api/#end-to-end-encryption).
+
+-   [Olm: A Cryptographic Ratchet](/olm-megolm/olm/)
+-   [Megolm group ratchet](/olm-megolm/megolm/)

content/olm-megolm/megolm.md

@@ -0,0 +1,378 @@
+---
+title: "Megolm group ratchet"
+weight: 20
+type: docs
+---
+
+An AES-based cryptographic ratchet intended for group communications.
+
+## Background
+
+The Megolm ratchet is intended for encrypted messaging applications where there
+may be a large number of recipients of each message, thus precluding the use of
+peer-to-peer encryption systems such as [Olm][].
+
+It also allows a recipient to decrypt received messages multiple times. For
+instance, in client/server applications, a copy of the ciphertext can be stored
+on the (untrusted) server, while the client need only store the session keys.
+
+## Overview
+
+Each participant in a conversation uses their own outbound session for
+encrypting messages. A session consists of a ratchet and an [Ed25519][] keypair.
+
+Secrecy is provided by the ratchet, which can be wound forwards but not
+backwards, and is used to derive a distinct message key for each message.
+
+Authenticity is provided via Ed25519 signatures.
+
+The value of the ratchet, and the public part of the Ed25519 key, are shared
+with other participants in the conversation via secure peer-to-peer
+channels. Provided that peer-to-peer channel provides authenticity of the
+messages to the participants and deniability of the messages to third parties,
+the Megolm session will inherit those properties.
+
+## The Megolm ratchet algorithm
+
+The Megolm ratchet \(R_i\) consists of four parts, \(R_{i,j}\) for
+\(j \in {0,1,2,3}\). The length of each part depends on the hash function
+in use (256 bits for this version of Megolm).
+
+The ratchet is initialised with cryptographically-secure random data, and
+advanced as follows:
+
+\[
+\begin{aligned}
+R_{i,0} &=
+  \begin{cases}
+  H_0\left(R_{2^{24}(n-1),0}\right) &\text{if }\exists n | i = 2^{24}n\\
+  R_{i-1,0} &\text{otherwise}
+  \end{cases}\\
+R_{i,1} &=
+  \begin{cases}
+  H_1\left(R_{2^{24}(n-1),0}\right) &\text{if }\exists n | i = 2^{24}n\\
+  H_1\left(R_{2^{16}(m-1),1}\right) &\text{if }\exists m | i = 2^{16}m\\
+  R_{i-1,1} &\text{otherwise}
+  \end{cases}\\
+R_{i,2} &=
+  \begin{cases}
+  H_2\left(R_{2^{24}(n-1),0}\right) &\text{if }\exists n | i = 2^{24}n\\
+  H_2\left(R_{2^{16}(m-1),1}\right) &\text{if }\exists m | i = 2^{16}m\\
+  H_2\left(R_{2^8(p-1),2}\right) &\text{if }\exists p | i = 2^8p\\
+  R_{i-1,2} &\text{otherwise}
+  \end{cases}\\
+R_{i,3} &=
+  \begin{cases}
+  H_3\left(R_{2^{24}(n-1),0}\right) &\text{if }\exists n | i = 2^{24}n\\
+  H_3\left(R_{2^{16}(m-1),1}\right) &\text{if }\exists m | i = 2^{16}m\\
+  H_3\left(R_{2^8(p-1),2}\right) &\text{if }\exists p | i = 2^8p\\
+  H_3\left(R_{i-1,3}\right) &\text{otherwise}
+  \end{cases}
+\end{aligned}
+\]
+
+where \(H_0\), \(H_1\), \(H_2\), and \(H_3\) are different hash
+functions. In summary: every \(2^8\) iterations, \(R_{i,3}\) is
+reseeded from \(R_{i,2}\). Every \(2^{16}\) iterations, \(R_{i,2}\)
+and \(R_{i,3}\) are reseeded from \(R_{i,1}\). Every \(2^{24}\)
+iterations, \(R_{i,1}\), \(R_{i,2}\) and \(R_{i,3}\) are reseeded
+from \(R_{i,0}\).
+
+The complete ratchet value, \(R_{i}\), is hashed to generate the keys used
+to encrypt each message. This scheme allows the ratchet to be advanced an
+arbitrary amount forwards while needing at most 1020 hash computations.  A
+client can decrypt chat history onwards from the earliest value of the ratchet
+it is aware of, but cannot decrypt history from before that point without
+reversing the hash function.
+
+This allows a participant to share its ability to decrypt chat history with
+another from a point in the conversation onwards by giving a copy of the
+ratchet at that point in the conversation.
+
+
+## The Megolm protocol
+
+### Session setup
+
+Each participant in a conversation generates their own Megolm session. A
+session consists of three parts:
+
+* a 32 bit counter, \(i\).
+* an [Ed25519][] keypair, \(K\).
+* a ratchet, \(R_i\), which consists of four 256-bit values,
+  \(R_{i,j}\) for \(j \in {0,1,2,3}\).
+
+The counter \(i\) is initialised to \(0\). A new Ed25519 keypair is
+generated for \(K\). The ratchet is simply initialised with 1024 bits of
+cryptographically-secure random data.
+
+A single participant may use multiple sessions over the lifetime of a
+conversation. The public part of \(K\) is used as an identifier to
+discriminate between sessions.
+
+### Sharing session data
+
+To allow other participants in the conversation to decrypt messages, the
+session data is formatted as described in [Session-sharing format](#session-sharing-format). It is then
+shared with other participants in the conversation via a secure peer-to-peer
+channel (such as that provided by [Olm][]).
+
+When the session data is received from other participants, the recipient first
+checks that the signature matches the public key. They then store their own
+copy of the counter, ratchet, and public key.
+
+### Message encryption
+
+This version of Megolm uses [AES-256][] in [CBC][] mode with [PKCS#7][] padding and
+[HMAC-SHA-256][] (truncated to 64 bits). The 256 bit AES key, 256 bit HMAC key,
+and 128 bit AES IV are derived from the megolm ratchet \(R_i\):
+
+\[
+\begin{aligned}
+ \mathit{AES\_KEY}_{i}\;\parallel\;\mathit{HMAC\_KEY}_{i}\;\parallel\;\mathit{AES\_IV}_{i}
+    &= \operatorname{HKDF}\left(0,\,R_{i},\text{"MEGOLM\_KEYS"},\,80\right) \\
+\end{aligned}
+\]
+
+where \(\parallel\) represents string splitting, and
+\(\operatorname{HKDF}\left(\mathit{salt},\,\mathit{IKM},\,\mathit{info},\,L\right)\)
+refers to the [HMAC-based key
+derivation function][] using using [SHA-256][] as the hash function
+([HKDF-SHA-256][]) with a salt value of \(\mathit{salt}\), input key material of
+\(\mathit{IKM}\), context string \(\mathit{info}\), and output keying material length of
+\(L\) bytes.
+
+The plain-text is encrypted with AES-256, using the key \(\mathit{AES\_KEY}_{i}\)
+and the IV \(\mathit{AES\_IV}_{i}\) to give the cipher-text, \(X_{i}\).
+
+The ratchet index \(i\), and the cipher-text \(X_{i}\), are then packed
+into a message as described in [Message format](#message-format). Then the entire message
+(including the version bytes and all payload bytes) are passed through
+HMAC-SHA-256. The first 8 bytes of the MAC are appended to the message.
+
+Finally, the authenticated message is signed using the Ed25519 keypair; the 64
+byte signature is appended to the message.
+
+The complete signed message, together with the public part of \(K\) (acting
+as a session identifier), can then be sent over an insecure channel. The
+message can then be authenticated and decrypted only by recipients who have
+received the session data.
+
+### Advancing the ratchet
+
+After each message is encrypted, the ratchet is advanced. This is done as
+described in [The Megolm ratchet algorithm](#the-megolm-ratchet-algorithm), using the following definitions:
+
+\[
+\begin{aligned}
+    H_0(A) &\equiv \operatorname{HMAC}(A,\text{``\char`\\x00"}) \\
+    H_1(A) &\equiv \operatorname{HMAC}(A,\text{``\char`\\x01"}) \\
+    H_2(A) &\equiv \operatorname{HMAC}(A,\text{``\char`\\x02"}) \\
+    H_3(A) &\equiv \operatorname{HMAC}(A,\text{``\char`\\x03"}) \\
+\end{aligned}
+\]
+
+where \(\operatorname{HMAC}(A, T)\) is the HMAC-SHA-256 of ``T``, using ``A`` as the
+key.
+
+For outbound sessions, the updated ratchet and counter are stored in the
+session.
+
+In order to maintain the ability to decrypt conversation history, inbound
+sessions should store a copy of their earliest known ratchet value (unless they
+explicitly want to drop the ability to decrypt that history - see [Partial
+Forward Secrecy](#partial-forward-secrecy)). They may also choose to cache calculated ratchet values,
+but the decision of which ratchet states to cache is left to the application.
+
+## Data exchange formats
+
+### Session sharing format
+
+This format is used for the initial sharing of a Megolm session with other
+group participants who need to be able to read messages encrypted by this
+session.
+
+The session sharing format is as follows:
+
+```nohighlight
++---+----+--------+--------+--------+--------+------+-----------+
+| V | i  | R(i,0) | R(i,1) | R(i,2) | R(i,3) | Kpub | Signature |
++---+----+--------+--------+--------+--------+------+-----------+
+0   1    5        37       69      101      133    165         229   bytes
+```
+
+The version byte, ``V``, is ``"\x02"``.
+
+This is followed by the ratchet index, \(i\), which is encoded as a
+big-endian 32-bit integer; the ratchet values \(R_{i,j}\); and the public
+part of the Ed25519 keypair \(K\).
+
+The data is then signed using the Ed25519 keypair, and the 64-byte signature is
+appended.
+
+### Session export format
+
+Once the session is initially shared with the group participants, each
+participant needs to retain a copy of the session if they want to maintain
+their ability to decrypt messages encrypted with that session.
+
+For forward-secrecy purposes, a participant may choose to store a ratcheted
+version of the session. But since the ratchet index is covered by the
+signature, this would invalidate the signature. So we define a similar format,
+called the *session export format*, which is identical to the [session sharing
+format](#session-sharing-format) except for dropping the signature.
+
+The Megolm session export format is thus as follows:
+
+```nohighlight
++---+----+--------+--------+--------+--------+------+
+| V | i  | R(i,0) | R(i,1) | R(i,2) | R(i,3) | Kpub |
++---+----+--------+--------+--------+--------+------+
+0   1    5        37       69      101      133    165   bytes
+```
+
+The version byte, ``V``, is ``"\x01"``.
+
+This is followed by the ratchet index, \(i\), which is encoded as a
+big-endian 32-bit integer; the ratchet values \(R_{i,j}\); and the public
+part of the Ed25519 keypair \(K\).
+
+### Message format
+
+Megolm messages consist of a one byte version, followed by a variable length
+payload, a fixed length message authentication code, and a fixed length
+signature.
+
+```nohighlight
++---+------------------------------------+-----------+------------------+
+| V | Payload Bytes                      | MAC Bytes | Signature Bytes  |
++---+------------------------------------+-----------+------------------+
+0   1                                    N          N+8                N+72   bytes
+```
+
+The version byte, ``V``, is ``"\x03"``.
+
+The payload uses a format based on the [Protocol Buffers encoding][]. It
+consists of the following key-value pairs:
+
+**Name**|**Tag**|**Type**|**Meaning**
+:-----:|:-----:|:-----:|:-----:
+Message-Index|0x08|Integer|The index of the ratchet, i
+Cipher-Text|0x12|String|The cipher-text, Xi, of the message
+
+Within the payload, integers are encoded using a variable length encoding. Each
+integer is encoded as a sequence of bytes with the high bit set followed by a
+byte with the high bit clear. The seven low bits of each byte store the bits of
+the integer. The least significant bits are stored in the first byte.
+
+Strings are encoded as a variable-length integer followed by the string itself.
+
+Each key-value pair is encoded as a variable-length integer giving the tag,
+followed by a string or variable-length integer giving the value.
+
+The payload is followed by the MAC. The length of the MAC is determined by the
+authenticated encryption algorithm being used (8 bytes in this version of the
+protocol). The MAC protects all of the bytes preceding the MAC.
+
+The length of the signature is determined by the signing algorithm being used
+(64 bytes in this version of the protocol). The signature covers all of the
+bytes preceding the signature.
+
+## Limitations
+
+### Message Replays
+
+A message can be decrypted successfully multiple times. This means that an
+attacker can re-send a copy of an old message, and the recipient will treat it
+as a new message.
+
+To mitigate this it is recommended that applications track the ratchet indices
+they have received and that they reject messages with a ratchet index that
+they have already decrypted.
+
+### Lack of Transcript Consistency
+
+In a group conversation, there is no guarantee that all recipients have
+received the same messages. For example, if Alice is in a conversation with Bob
+and Charlie, she could send different messages to Bob and Charlie, or could
+send some messages to Bob but not Charlie, or vice versa.
+
+Solving this is, in general, a hard problem, particularly in a protocol which
+does not guarantee in-order message delivery. For now it remains the subject of
+future research.
+
+### Lack of Backward Secrecy
+
+[Backward secrecy](https://intensecrypto.org/public/lec_08_hash_functions_part2.html#sec-forward-and-backward-secrecy)
+(also called 'future secrecy' or 'post-compromise security') is the property
+that if current private keys are compromised, an attacker cannot decrypt
+future messages in a given session.  In other words, when looking
+**backwards** in time at a compromise which has already happened, **current**
+messages are still secret.
+
+By itself, Megolm does not possess this property: once the key to a Megolm
+session is compromised, the attacker can decrypt any message that was
+encrypted using a key derived from the compromised or subsequent ratchet
+values.
+
+In order to mitigate this, the application should ensure that Megolm sessions
+are not used indefinitely. Instead it should periodically start a new session,
+with new keys shared over a secure channel.
+
+<!-- TODO: Can we recommend sensible lifetimes for Megolm sessions? Probably
+   depends how paranoid we're feeling, but some guidelines might be useful. -->
+
+### Partial Forward Secrecy
+
+[Forward secrecy](https://intensecrypto.org/public/lec_08_hash_functions_part2.html#sec-forward-and-backward-secrecy)
+(also called 'perfect forward secrecy') is the property that if the current
+private keys are compromised, an attacker cannot decrypt *past* messages in
+a given session. In other words, when looking **forwards** in time towards a
+potential future compromise, **current** messages will be secret.
+
+In Megolm, each recipient maintains a record of the ratchet value which allows
+them to decrypt any messages sent in the session after the corresponding point
+in the conversation. If this value is compromised, an attacker can similarly
+decrypt past messages which were encrypted by a key derived from the
+compromised or subsequent ratchet values. This gives 'partial' forward
+secrecy.
+
+To mitigate this issue, the application should offer the user the option to
+discard historical conversations, by winding forward any stored ratchet values,
+or discarding sessions altogether.
+
+### Dependency on secure channel for key exchange
+
+The design of the Megolm ratchet relies on the availability of a secure
+peer-to-peer channel for the exchange of session keys. Any vulnerabilities in
+the underlying channel are likely to be amplified when applied to Megolm
+session setup.
+
+For example, if the peer-to-peer channel is vulnerable to an unknown key-share
+attack, the entire Megolm session become similarly vulnerable. For example:
+Alice starts a group chat with Eve, and shares the session keys with Eve. Eve
+uses the unknown key-share attack to forward the session keys to Bob, who
+believes Alice is starting the session with him. Eve then forwards messages
+from the Megolm session to Bob, who again believes they are coming from
+Alice. Provided the peer-to-peer channel is not vulnerable to this attack, Bob
+will realise that the key-sharing message was forwarded by Eve, and can treat
+the Megolm session as a forgery.
+
+A second example: if the peer-to-peer channel is vulnerable to a replay
+attack, this can be extended to entire Megolm sessions.
+
+## License
+
+The Megolm specification (this document) is licensed under the Apache License,
+Version 2.0 http://www.apache.org/licenses/LICENSE-2.0.
+
+[Ed25519]: http://ed25519.cr.yp.to/
+[HMAC-based key derivation function]: https://tools.ietf.org/html/rfc5869
+[HKDF-SHA-256]: https://tools.ietf.org/html/rfc5869
+[HMAC-SHA-256]: https://tools.ietf.org/html/rfc2104
+[SHA-256]: https://tools.ietf.org/html/rfc6234
+[AES-256]: http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf
+[CBC]: http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf
+[PKCS#7]: https://tools.ietf.org/html/rfc2315
+[Olm]: https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/olm.md
+[Protocol Buffers encoding]: https://developers.google.com/protocol-buffers/docs/encoding

content/olm-megolm/olm.md

@@ -0,0 +1,334 @@
+---
+title: "Olm: A Cryptographic Ratchet"
+weight: 10
+type: docs
+---
+
+An implementation of the double cryptographic ratchet described by
+https://whispersystems.org/docs/specifications/doubleratchet/.
+
+## Notation
+
+This document uses \(\parallel\) to represent string concatenation. When
+\(\parallel\) appears on the right hand side of an \(=\) it means that
+the inputs are concatenated. When \(\parallel\) appears on the left hand
+side of an \(=\) it means that the output is split.
+
+When this document uses \(\operatorname{ECDH}\left(K_A,K_B\right)\) it means
+that each party computes a Diffie-Hellman agreement using their private key 
+and the remote party's public key.
+So party \(A\) computes \(\operatorname{ECDH}\left(K_B^{public},K_A^{private}\right)\)
+and party \(B\) computes \(\operatorname{ECDH}\left(K_A^{public},K_B^{private}\right)\).
+
+Where this document uses \(\operatorname{HKDF}\left(salt,IKM,info,L\right)\) it
+refers to the [HMAC-based key derivation function][] with a salt value of
+\(salt\), input key material of \(IKM\), context string \(info\),
+and output keying material length of \(L\) bytes.
+
+## The Olm Algorithm
+
+### Initial setup
+
+The setup takes four [Curve25519][] inputs: Identity keys for Alice and Bob,
+\(I_A\) and \(I_B\), and one-time keys for Alice and Bob,
+\(E_A\) and \(E_B\). A shared secret, \(S\), is generated using
+[Triple Diffie-Hellman][]. The initial 256 bit root key, \(R_0\), and 256
+bit chain key, \(C_{0,0}\), are derived from the shared secret using an
+HMAC-based Key Derivation Function using [SHA-256][] as the hash function
+([HKDF-SHA-256][]) with default salt and ``"OLM_ROOT"`` as the info.
+
+\[
+\begin{aligned}
+    S&=\operatorname{ECDH}\left(I_A,E_B\right)\;\parallel\;
+       \operatorname{ECDH}\left(E_A,I_B\right)\;\parallel\;
+       \operatorname{ECDH}\left(E_A,E_B\right)\\
+    R_0\;\parallel\;C_{0,0}&=
+        \operatorname{HKDF}\left(0,S,\text{``OLM\_ROOT"},64\right)
+\end{aligned}
+\]
+
+### Advancing the root key
+
+Advancing a root key takes the previous root key, \(R_{i-1}\), and two
+Curve25519 inputs: the previous ratchet key, \(T_{i-1}\), and the current
+ratchet key \(T_i\). The even ratchet keys are generated by Alice.
+The odd ratchet keys are generated by Bob. A shared secret is generated
+using Diffie-Hellman on the ratchet keys. The next root key, \(R_i\), and
+chain key, \(C_{i,0}\), are derived from the shared secret using
+[HKDF-SHA-256][] using \(R_{i-1}\) as the salt and ``"OLM_RATCHET"`` as the
+info.
+
+\[
+\begin{aligned}
+    R_i\;\parallel\;C_{i,0}&=
+        \operatorname{HKDF}\left(
+            R_{i-1},
+            \operatorname{ECDH}\left(T_{i-1},T_i\right),
+            \text{``OLM\_RATCHET"},
+            64
+        \right)
+\end{aligned}
+\]
+
+### Advancing the chain key
+
+Advancing a chain key takes the previous chain key, \(C_{i,j-1}\). The next
+chain key, \(C_{i,j}\), is the [HMAC-SHA-256][] of ``"\x02"`` using the
+previous chain key as the key.
+
+\[
+\begin{aligned}
+    C_{i,j}&=\operatorname{HMAC}\left(C_{i,j-1},\text{``\char`\\x02"}\right)
+\end{aligned}
+\]
+
+### Creating a message key
+
+Creating a message key takes the current chain key, \(C_{i,j}\). The
+message key, \(M_{i,j}\), is the [HMAC-SHA-256][] of ``"\x01"`` using the
+current chain key as the key. The message keys where \(i\) is even are used
+by Alice to encrypt messages. The message keys where \(i\) is odd are used
+by Bob to encrypt messages.
+
+\[
+\begin{aligned}
+    M_{i,j}&=\operatorname{HMAC}\left(C_{i,j},\text{``\char`\\x01"}\right)
+\end{aligned}
+\]
+
+## The Olm Protocol
+
+### Creating an outbound session
+
+Bob publishes the public parts of his identity key, \(I_B\), and some
+single-use one-time keys \(E_B\).
+
+Alice downloads Bob's identity key, \(I_B\), and a one-time key,
+\(E_B\). She generates a new single-use key, \(E_A\), and computes a
+root key, \(R_0\), and a chain key \(C_{0,0}\). She also generates a
+new ratchet key \(T_0\).
+
+### Sending the first pre-key messages
+
+Alice computes a message key, \(M_{0,j}\), and a new chain key,
+\(C_{0,j+1}\), using the current chain key. She replaces the current chain
+key with the new one.
+
+Alice encrypts her plain-text with the message key, \(M_{0,j}\), using an
+authenticated encryption scheme (see below) to get a cipher-text,
+\(X_{0,j}\).
+
+She then sends the following to Bob:
+ * The public part of her identity key, \(I_A\)
+ * The public part of her single-use key, \(E_A\)
+ * The public part of Bob's single-use key, \(E_B\)
+ * The current chain index, \(j\)
+ * The public part of her ratchet key, \(T_0\)
+ * The cipher-text, \(X_{0,j}\)
+
+Alice will continue to send pre-key messages until she receives a message from
+Bob.
+
+### Creating an inbound session from a pre-key message
+
+Bob receives a pre-key message as above.
+
+Bob looks up the private part of his single-use key, \(E_B\). He can now
+compute the root key, \(R_0\), and the chain key, \(C_{0,0}\), from
+\(I_A\), \(E_A\), \(I_B\), and \(E_B\).
+
+Bob then advances the chain key \(j\) times, to compute the chain key used
+by the message, \(C_{0,j}\). He now creates the
+message key, \(M_{0,j}\), and attempts to decrypt the cipher-text,
+\(X_{0,j}\). If the cipher-text's authentication is correct then Bob can
+discard the private part of his single-use one-time key, \(E_B\).
+
+Bob stores Alice's initial ratchet key, \(T_0\), until he wants to
+send a message.
+
+### Sending normal messages
+
+Once a message has been received from the other side, a session is considered
+established, and a more compact form is used.
+
+To send a message, the user checks if they have a sender chain key,
+\(C_{i,j}\). Alice uses chain keys where \(i\) is even. Bob uses chain
+keys where \(i\) is odd. If the chain key doesn't exist then a new ratchet
+key \(T_i\) is generated and a new root key \(R_i\) and chain key
+\(C_{i,0}\) are computed using \(R_{i-1}\), \(T_{i-1}\) and
+\(T_i\).
+
+A message key,
+\(M_{i,j}\) is computed from the current chain key, \(C_{i,j}\), and
+the chain key is replaced with the next chain key, \(C_{i,j+1}\). The
+plain-text is encrypted with \(M_{i,j}\), using an authenticated encryption
+scheme (see below) to get a cipher-text, \(X_{i,j}\).
+
+The user then sends the following to the recipient:
+ * The current chain index, \(j\)
+ * The public part of the current ratchet key, \(T_i\)
+ * The cipher-text, \(X_{i,j}\)
+
+### Receiving messages
+
+The user receives a message as above with the sender's current chain index, \(j\),
+the sender's ratchet key, \(T_i\), and the cipher-text, \(X_{i,j}\).
+
+The user checks if they have a receiver chain with the correct
+\(i\) by comparing the ratchet key, \(T_i\). If the chain doesn't exist
+then they compute a new root key, \(R_i\), and a new receiver chain, with
+chain key \(C_{i,0}\), using \(R_{i-1}\), \(T_{i-1}\) and
+\(T_i\).
+
+If the \(j\) of the message is less than
+the current chain index on the receiver then the message may only be decrypted
+if the receiver has stored a copy of the message key \(M_{i,j}\). Otherwise
+the receiver computes the chain key, \(C_{i,j}\). The receiver computes the
+message key, \(M_{i,j}\), from the chain key and attempts to decrypt the
+cipher-text, \(X_{i,j}\).
+
+If the decryption succeeds the receiver updates the chain key for \(T_i\)
+with \(C_{i,j+1}\) and stores the message keys that were skipped in the
+process so that they can decode out of order messages. If the receiver created
+a new receiver chain then they discard their current sender chain so that
+they will create a new chain when they next send a message.
+
+## The Olm Message Format
+
+Olm uses two types of messages. The underlying transport protocol must provide
+a means for recipients to distinguish between them.
+
+### Normal Messages
+
+Olm messages start with a one byte version followed by a variable length
+payload followed by a fixed length message authentication code.
+
+```nohighlight
+ +--------------+------------------------------------+-----------+
+ | Version Byte | Payload Bytes                      | MAC Bytes |
+ +--------------+------------------------------------+-----------+
+```
+
+The version byte is ``"\x03"``.
+
+The payload consists of key-value pairs where the keys are integers and the
+values are integers and strings. The keys are encoded as a variable length
+integer tag where the 3 lowest bits indicates the type of the value:
+0 for integers, 2 for strings. If the value is an integer then the tag is
+followed by the value encoded as a variable length integer. If the value is
+a string then the tag is followed by the length of the string encoded as
+a variable length integer followed by the string itself.
+
+Olm uses a variable length encoding for integers. Each integer is encoded as a
+sequence of bytes with the high bit set followed by a byte with the high bit
+clear. The seven low bits of each byte store the bits of the integer. The least
+significant bits are stored in the first byte.
+
+**Name**|**Tag**|**Type**|**Meaning**
+:-----:|:-----:|:-----:|:-----:
+Ratchet-Key|0x0A|String|The public part of the ratchet key, Ti, of the message
+Chain-Index|0x10|Integer|The chain index, j, of the message
+Cipher-Text|0x22|String|The cipher-text, Xi, j, of the message
+
+The length of the MAC is determined by the authenticated encryption algorithm
+being used. (Olm version 1 uses [HMAC-SHA-256][], truncated to 8 bytes). The
+MAC protects all of the bytes preceding the MAC.
+
+### Pre-Key Messages
+
+Olm pre-key messages start with a one byte version followed by a variable
+length payload.
+
+```nohighlight
+ +--------------+------------------------------------+
+ | Version Byte | Payload Bytes                      |
+ +--------------+------------------------------------+
+```
+
+The version byte is ``"\x03"``.
+
+The payload uses the same key-value format as for normal messages.
+
+**Name**|**Tag**|**Type**|**Meaning**
+:-----:|:-----:|:-----:|:-----:
+One-Time-Key|0x0A|String|The public part of Bob's single-use key, Eb.
+Base-Key|0x12|String|The public part of Alice's single-use key, Ea.
+Identity-Key|0x1A|String|The public part of Alice's identity key, Ia.
+Message|0x22|String|An embedded Olm message with its own version and MAC.
+
+## Olm Authenticated Encryption
+
+### Version 1
+
+Version 1 of Olm uses [AES-256][] in [CBC][] mode with [PKCS#7][] padding for
+encryption and [HMAC-SHA-256][] (truncated to 64 bits) for authentication.  The
+256 bit AES key, 256 bit HMAC key, and 128 bit AES IV are derived from the
+message key using [HKDF-SHA-256][] using the default salt and an info of
+``"OLM_KEYS"``.
+
+\[
+\begin{aligned}
+    AES\_KEY_{i,j}\;\parallel\;HMAC\_KEY_{i,j}\;\parallel\;AES\_IV_{i,j}
+    &= \operatorname{HKDF}\left(0,M_{i,j},\text{``OLM\_KEYS"},80\right)
+\end{aligned}
+\]
+
+The plain-text is encrypted with AES-256, using the key \(AES\_KEY_{i,j}\)
+and the IV \(AES\_IV_{i,j}\) to give the cipher-text, \(X_{i,j}\).
+
+Then the entire message (including the Version Byte and all Payload Bytes) are
+passed through [HMAC-SHA-256][]. The first 8 bytes of the MAC are appended to the message.
+
+## Message authentication concerns
+
+To avoid unknown key-share attacks, the application must include identifying
+data for the sending and receiving user in the plain-text of (at least) the
+pre-key messages. Such data could be a user ID, a telephone number;
+alternatively it could be the public part of a keypair which the relevant user
+has proven ownership of.
+
+### Example attacks
+
+1. Alice publishes her public [Curve25519][] identity key, \(I_A\). Eve
+   publishes the same identity key, claiming it as her own. Bob downloads
+   Eve's keys, and associates \(I_A\) with Eve. Alice sends a message to
+   Bob; Eve intercepts it before forwarding it to Bob. Bob believes the
+   message came from Eve rather than Alice.
+
+   This is prevented if Alice includes her user ID in the plain-text of the
+   pre-key message, so that Bob can see that the message was sent by Alice
+   originally.
+
+2. Bob publishes his public [Curve25519][] identity key, \(I_B\). Eve
+   publishes the same identity key, claiming it as her own. Alice downloads
+   Eve's keys, and associates \(I_B\) with Eve. Alice sends a message to
+   Eve; Eve cannot decrypt it, but forwards it to Bob. Bob believes the
+   Alice sent the message to him, whereas Alice intended it to go to Eve.
+
+   This is prevented by Alice including the user ID of the intended recpient
+   (Eve) in the plain-text of the pre-key message. Bob can now tell that the
+   message was meant for Eve rather than him.
+
+## IPR
+
+The Olm specification (this document) is hereby placed in the public domain.
+
+## Feedback
+
+Can be sent to olm at matrix.org.
+
+## Acknowledgements
+
+The ratchet that Olm implements was designed by Trevor Perrin and Moxie
+Marlinspike - details at https://whispersystems.org/docs/specifications/doubleratchet/. Olm is
+an entirely new implementation written by the Matrix.org team.
+
+[Curve25519]: http://cr.yp.to/ecdh.html
+[Triple Diffie-Hellman]: https://whispersystems.org/blog/simplifying-otr-deniability/
+[HMAC-based key derivation function]: https://tools.ietf.org/html/rfc5869
+[HKDF-SHA-256]: https://tools.ietf.org/html/rfc5869
+[HMAC-SHA-256]: https://tools.ietf.org/html/rfc2104
+[SHA-256]: https://tools.ietf.org/html/rfc6234
+[AES-256]: http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf
+[CBC]: http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf
+[PKCS#7]: https://tools.ietf.org/html/rfc2315

content/proposals.md

@@ -1,6 +1,6 @@
 ---
 title: "Spec Change Proposals"
-weight: 60
+weight: 62
 type: docs
 ---
 
@@ -185,6 +185,10 @@ is as follows:
     -   Take care in creating your proposal. Specify your intended
         changes, and give reasoning to back them up. Changes without
         justification will likely be poorly received by the community.
+    -   At the time of creating your draft you will not yet know the PR number, so you
+        should use a placeholder number to name your file and edit that
+        after PR submission. The suggested steps are described in
+        detail [in the proposals guide](https://github.com/matrix-org/matrix-spec-proposals#1-writing-the-proposal).
 -   Fork and make a PR to the
     [matrix-spec-proposals](https://github.com/matrix-org/matrix-spec-proposals) repository.
     The ID of your PR will become the MSC ID for the lifetime of your
@@ -277,7 +281,7 @@ corresponding labels for each stage on the
 [matrix-spec-proposals](https://github.com/matrix-org/matrix-spec-proposals)
 pull request trackers.
 
-```
+```nohighlight
                            +                          +
          Proposals         |          Spec PRs        |  Additional States
          +-------+         |          +------+        |  +---------------+
@@ -404,41 +408,9 @@ development or testing data.
 that a particular MSC works) do not have to follow this process.
 
 1.  Have an idea for a feature.
-1.  Implement the feature using unstable endpoints, vendor prefixes, and
-    unstable feature flags as appropriate.
-    -   When using unstable endpoints, they MUST include a vendor
-        prefix. For example:
-        `/_matrix/client/unstable/com.example/login`. Vendor prefixes
-        throughout Matrix always use the Java package naming convention.
-        The MSC for the feature should identify which preferred vendor
-        prefix is to be used by early adopters.
-    -   Note that unstable namespaces do not automatically inherit
-        endpoints from stable namespaces: for example, the fact that
-        `/_matrix/client/r0/sync` exists does not imply that
-        `/_matrix/client/unstable/com.example/sync` exists.
-    -   If the client needs to be sure the server supports the feature,
-        an unstable feature flag that MUST be vendor prefixed is to be
-        used. This kind of flag shows up in the `unstable_features`
-        section of `/versions` as, for example, `com.example.new_login`.
-        The MSC for the feature should identify which preferred feature
-        flag is to be used by early adopters.
-    -   When using this approach correctly, the implementation can
-        ship/release the feature at any time, so long as the
-        implementation is able to accept the technical debt that results
-        from needing to provide adequate backwards and forwards
-        compatibility. The implementation MUST support the flag (and
-        server-side implementation) disappearing and be generally safe
-        for users. Note that implementations early in the MSC review
-        process may also be required to provide backwards compatibility
-        with earlier editions of the proposal.
-    -   If the implementation cannot support the technical debt (or if
-        it's impossible to provide forwards/backwards compatibility -
-        e.g. a user authentication change which can't be safely rolled
-        back), the implementation should not attempt to implement the
-        feature and should instead wait for a spec release.
-    -   If at any point after early release, the idea changes in a
-        backwards-incompatible way, the feature flag should also change
-        so that implementations can adapt as needed.
+1.  Implement the feature using [unstable endpoints, vendor prefixes, and
+    unstable feature flags](#unstable-endpoints-features-and-vendor-prefixes)
+    as appropriate.
 1.  In parallel, or ahead of implementation, open an MSC and solicit
     review per above.
 1.  Before FCP can be called, the Spec Core Team will require evidence
@@ -448,10 +420,7 @@ that a particular MSC works) do not have to follow this process.
     forwards/backwards compatibility concerns mentioned here.
 1.  The FCP process is completed, and assuming nothing is flagged the
     MSC lands.
-1.  Implementations can now switch to using stable prefixes
-    (for example, for an endpoint, moving from
-    `/unstable/org.matrix.mscxxxx/frobnicate`
-    to `/v1/frobnicate`), assuming that the change
+1.  Implementations can now switch to using stable prefixes, assuming that the change
     is backwards compatible with older implementations. In the rare occasion
     where backwards compatibility is not possible without a new spec release,
     implementations should continue to use unstable prefixes.
@@ -467,13 +436,6 @@ that a particular MSC works) do not have to follow this process.
     started supporting the new spec release, some noise should be raised
     in the general direction of the implementation.
 
-{{% boxes/note %}}
-MSCs MUST still describe what the stable endpoints/feature looks like
-with a note towards the bottom for what the unstable feature
-flag/prefixes are. For example, an MSC would propose `/_matrix/client/r0/new/endpoint`, not `/_matrix/client/unstable/
-com.example/new/endpoint`.
-{{% /boxes/note %}}
-
 In summary:
 
 -   Implementations MUST NOT use stable endpoints before the MSC has
@@ -485,14 +447,126 @@ In summary:
 -   Implementations SHOULD be wary of the technical debt they are
     incurring by moving faster than the spec.
 -   The vendor prefix is chosen by the developer of the feature, using
-    the Java package naming convention. The foundation's preferred
-    vendor prefix is `org.matrix`.
+    the Java package naming convention.
 -   The vendor prefixes, unstable feature flags, and unstable endpoints
     should be included in the MSC, though the MSC MUST be written in a
     way that proposes new stable endpoints. Typically this is solved by
     a small table at the bottom mapping the various values from stable
     to unstable.
 
+#### Unstable endpoints, features and vendor prefixes
+
+Unstable endpoints MUST use `/unstable` as the endpoint version and a
+vendor prefix in Java package naming format. For example:
+`/_matrix/client/unstable/com.example.mscxxxx/login`.
+
+{{% boxes/note %}}
+Proposal authors operating with a Matrix.org Foundation mandate SHOULD use
+a vendor prefix within the `org.matrix` namespace. This namespace is otherwise
+restricted. Authors who don't own a domain MAY use the `io.github` namespace
+instead.
+{{% /boxes/note %}}
+
+Note that unstable namespaces do not automatically inherit endpoints from
+stable namespaces: for example, the fact that `/_matrix/client/v3/sync`
+exists does not imply that `/_matrix/client/unstable/com.example.mscxxxx/sync`
+exists.
+
+Vendor prefixes MUST also be used for:
+
+-   New parameters on existing endpoints. For example:
+    `/_matrix/client/v3/publicRooms?com.example.mscxxxx.ordered_by=member_count`.
+-   New properties in existing JSON objects. For example:
+
+    ```json
+    {
+      "avatar_url": "mxc://matrix.org/SDGdghriugerRg",
+      "displayname": "Alice Margatroid",
+      "com.example.mscxxxx.phone": [{
+        "type": "landline",
+        "number": "+1-206-555-7000"
+      }],
+      ...
+    }
+    ```
+
+-   New values for existing parameters or properties. For example:
+
+    ```json
+    {
+      "errcode": "COM.EXAMPLE.MSCXXXX.M_INVALID_EMAIL",
+      "error": "The email address you provided is invalid."
+    }
+    ```
+
+If the client needs to be sure the server supports the feature, an
+unstable feature flag that MUST also be vendor prefixed is to be used.
+This flag shows up in the `unstable_features` section of
+[`/_matrix/client/versions`](/client-server-api/#get_matrixclientversions)
+as, for example, `com.example.mscxxxx.new_login`.
+
+{{% boxes/note %}}
+MSCs MUST still describe what the stable endpoints/feature looks like
+with a note towards the bottom for what the unstable feature
+flag/prefixes are. For example, an MSC would propose `/_matrix/client/v1/new/endpoint`,
+not `/_matrix/client/unstable/com.example.mscxxxx/new/endpoint`.
+{{% /boxes/note %}}
+
+When using this approach correctly, the implementation can release
+the feature at any time, so long as the implementation is able to
+accept the technical debt that results from needing to provide
+adequate backwards and forwards compatibility. The implementation
+MUST support the flag (and server-side implementation) disappearing
+and be generally safe for users. Note that implementations early in
+the MSC review process may also be required to provide backwards
+compatibility with earlier editions of the proposal.
+
+If the implementation cannot support the technical debt (or if it's
+impossible to provide forwards/backwards compatibility - e.g. a user
+authentication change which can't be safely rolled back), the
+implementation should not attempt to implement the feature and should
+instead wait for a spec release.
+
+If at any point after early release, the idea changes in a
+backwards-incompatible way, the feature flag should also change so
+that implementations can adapt as needed.
+
+### Placeholder MSCs
+
+Some proposals may contain security-sensitive or private context which can't be
+publicly disclosed until a later stage in the idea or solution process. Typically,
+the initial idea is validated using some amount of implementation or experimentation
+and may require an MSC number to make that implementation easier.
+
+Placeholder MSCs are used to represent proposals in a state where implementation
+is ongoing, but the MSC details can't yet be disclosed. Authors which feel as
+though their MSC could be highly sensitive MUST get in contact with the Spec Core
+Team or [Security Team](https://matrix.org/security-disclosure-policy/) prior to
+opening their MSC. If either team determines that a placeholder MSC is required,
+it may be opened as such.
+
+There are a few expectations attached to placeholder MSCs:
+
+* They have a title which marks them WIP, and are in the "draft" state.
+* They have the following labels: `[proposal-placeholder, action-required, needs-implementation]`.
+  * Notably, *not* `proposal`.
+* They are relatively short-lived (ideally less than 6-12 months in placeholder).
+* They propose solutions which are reasonably likely to be accepted. If a placeholder
+  needs to be closed because the idea won't work, isn't needed, etc, then the MSC's
+  content MUST be published ahead of that closure.
+  * Note: the MSC's publication (and therefore closure) may be delayed until an
+    appropriate point in the security disclosure cycle. For example, an alternative
+    MSC being published, or a stream of work being completed.
+* When they are updated to receive real content, the following happens:
+  1. The Spec Core Team or the author leaves a comment to cause a notification
+     that the MSC has been replaced with real content.
+  2. The `proposal` label (or its equivalent) is added to trigger chat notifications
+     in the public Matrix rooms. The `proposal-placeholder` and `action-required`
+     labels should be removed at this stage as well. Other labels are removed/applied
+     per normal process.
+* The Spec Core Team is aware of the intended MSC's title and purpose. This is
+  especially important if the Security Team approved the use of a placeholder MSC.
+
 ## Proposal Tracking
 
 This is a living document generated from the list of proposals on the
@@ -515,7 +589,7 @@ resolve to the desired MSC, whether it started as an issue or a PR.
 Other metadata:
 
 -   The MSC number is taken from the GitHub Pull Request ID. This is
-    carried for the lifetime of the proposal. These IDs do not necessary
+    carried for the lifetime of the proposal. These IDs do not necessarily
     represent a chronological order.
 -   The GitHub PR title will act as the MSC's title.
 -   Please link to the spec PR (if any) by adding a "PRs: \#1234" line

content/push-gateway-api.md

@@ -2,19 +2,18 @@
 title: "Push Gateway API"
 weight: 50
 type: docs
+description: |
+  Clients may want to receive push notifications when events are received at the
+  homeserver. This is managed by a distinct entity called the Push Gateway.
 ---
 
-Clients may want to receive push notifications when events are received
-at the homeserver. This is managed by a distinct entity called the Push
-Gateway.
-
 ## Overview
 
 A client's homeserver forwards information about received events to the
 push gateway. The gateway then submits a push notification to the push
 notification provider (e.g. APNS, GCM).
 
-```
+```nohighlight
                                    +--------------------+  +-------------------+
                   Matrix HTTP      |                    |  |                   |
              Notification Protocol |   App Developer    |  |   Device Vendor   |

content/rooms/_index.md

@@ -36,11 +36,12 @@ Alternatively, consider flipping the column/row organization to be features
 up top and versions on the left.
 -->
 
-| Feature \ Version | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 |
-|-------------------|---|---|---|---|---|---|---|---|---|----|----|
-| **Knocking**      | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ | ✔ | ✔ | ✔ | ✔ |
-| **Restricted join rules** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ | ✔ | ✔ | ✔ |
-| **`knock_restricted` join rule** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ | ✔ |
+| Feature \ Version | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+|-------------------|---|---|---|---|---|---|---|---|---|----|----|----|
+| **Knocking**      | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
+| **Restricted join rules** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ | ✔ | ✔ | ✔ | ✔ |
+| **`knock_restricted` join rule** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ | ✔ | ✔ |
+| **Additional room creators** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✔ |
 
 ## Complete list of room versions
 
@@ -52,7 +53,7 @@ stable and unstable periodically for a variety of reasons, including
 discovered security vulnerabilities and age.
 
 Clients should not ask room administrators to upgrade their rooms if the
-room is running a stable version. Servers SHOULD use **room version 11** as
+room is running a stable version. Servers SHOULD use **room version 12** as
 the default room version when creating new rooms.
 
 The available room versions are:
@@ -76,6 +77,9 @@ The available room versions are:
 -   [Version 10](/rooms/v10) - **Stable**. Enforces integer-only power levels
     and adds `knock_restricted` join rule.
 -   [Version 11](/rooms/v11) - **Stable**. Clarifies the redaction algorithm.
+-   [Version 12](/rooms/v12) - **Stable**. Changes room IDs to be hashes of the
+    create event, formalizes room creators with infinite power level, and iterates
+    on state resolution.
 
 ## Room version grammar
 

content/rooms/fragments/v1-auth-rules.md

@@ -30,10 +30,14 @@ The rules are as follows:
         specified by the [auth events
         selection](/server-server-api#auth-events-selection)
         algorithm described in the server specification, reject.
+
+        **Note**: This room version requires an `m.room.create` event to be selected.
     3.  If there are entries which were themselves rejected under the [checks
         performed on receipt of a
         PDU](/server-server-api/#checks-performed-on-receipt-of-a-pdu), reject.
     4. If there is no `m.room.create` event among the entries, reject.
+    5. If any event in `auth_events` has a `room_id` which does not match that of
+       the event being authorised, reject.
 3. If the `content` of the `m.room.create` event in the room state has the
    property `m.federate` set to `false`, and the `sender` domain of the event
    does not match the `sender` domain of the create event, reject.