From 3ee9a7d654050219f09487ce7b6bd9584827a6c2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Feb 2020 21:03:11 -0700 Subject: [PATCH 1/6] Attempt to convert MSC2324 to reality We've been using this in practice for a while now, so we should document it more officially. MSC: https://github.com/matrix-org/matrix-doc/pull/2324 --- specification/proposals_intro.rst | 102 +++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 82a4225b..3f70345f 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -230,7 +230,8 @@ follows: comments and in relevant rooms on Matrix. Discussion outside of GitHub should be summarised in a comment on the PR. - When a member of the Spec Core Team believes that no new discussion points are - being made, they will propose a motion for a final comment period (FCP), + being made, and the proposal has suitable evidence of working (see `implementing a + proposal`_ below), they will propose a motion for a final comment period (FCP), along with a *disposition* of either merge, close or postpone. This FCP is provided to allow a short period of time for any invested party to provide a final objection before a major decision is made. If sufficient reasoning is @@ -341,6 +342,105 @@ Closed proposal-closed A proposal which Obsolete obsolete A proposal which has been made obsolete by another proposal or decision elsewhere. =============================== ============================= ==================================== +Implementing a proposal +----------------------- + +As part of the proposal process the spec core team will require evidence of the MSC +working in order for it to move into FCP. This can usually be a branch/pull request +to whichever implementation of choice that proves the MSC works in practice, though +in some cases the MSC itself will be small enough to be considered proven. Where it's +unclear if a MSC will require an implementation proof, ask in `#matrix-spec:matrix.org +`_. + +Early adoption of a MSC/idea +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To help facilitate early releases of software dependent on a spec release, implementations +are required to use the following process to ensure that the official Matrix namespace +is not cluttered with development or testing data. + +.. Note:: + Proof of concept implementations that are solely left on a branch/unreleased and for + demonstrating that a particular MSC works do not have to follow this. Note that the + ``develop`` branch adopted by some projects is still subject to this process. + +The process for an implementation willing to be ahead of a spec release is: + +1. Have an idea for a feature. +2. Implement the feature using unstable endpoints, vendor prefixes, and unstable + feature flags as appropriate. + + * When using unstable endpoints, they MUST include a vendor prefix. For example: + `/_matrix/client/unstable/com.example/login`. Vendor prefixes throughout Matrix + always use the Java package naming convention. The MSC for the feature should + identify which preferred vendor prefix is to be used by early adopters. + * Unstable endpoints **do not** inherit from stable (e.g. `/r0`) APIs. Implementations + cannot assume that a particular endpoint will exist in the unstable namespace + even if the server advertises support for the feature. + * If the client needs to be sure the server supports the feature, an unstable + feature flag that MUST be vendor prefixed is to be used. This kind of flag shows + up in the ``unstable_features`` section of ``/versions`` as, for example, + ``com.example.new_login``. The MSC for the feature should identify which preferred + feature flag is to be used by early adopters. + * When using this approach correctly, the implementation can ship/release the + feature at any time, so long as the implementation is able to accept the technical + debt that results from needing to provide adequate backwards and forwards + compatibility. The implementation MUST support the flag disappearing and be + generally safe for users. Note that implementations early in the MSC review + process may also be required to provide backwards compatibility with earlier + editions of the proposal. + * If the implementation cannot support the technical debt (or if it's impossible + to provide forwards/backwards compatibility - e.g. a user authentication change + which can't be safely rolled back), the implementation should not attempt to + implement the feature and instead wait for a spec release. + * If at any point the idea changes, the feature flag should also change so that + implementations can adapt as needed. + +3. In parallel, or ahead of implementation, open an MSC and solicit review per above. +4. Before FCP can be called, the Spec Core Team will require evidence of the MSC + working as proposed. A typical example of this is an implementation of the MSC, + though the implementation does not need to be shipped anywhere and can therefore + avoid the forwards/backwards compatibility concerns mentioned here. +5. FCP is gone through, and assuming nothing is flagged the MSC lands. +6. A spec PR is written to incorporate the changes into Matrix. +7. A spec release happens. +8. Implementations switch to using stable prefixes (e.g.: ``/r0``) if the server + supports the specification version released. If the server doesn't advertise the + specification version, but does have the feature flag, unstable prefixes should + still be used. +9. A transition period of about 2 months starts immediately after the spec release, + before implementations start to loudly encourage other implementations to switch + to stable endpoints. For example, a server implementation should start asking + client implementations to support the stable endpoints 2 months after the spec + release, if they haven't already. The same applies in the reverse: if clients + cannot switch to stable prefixes because server implementations haven't started + supporting the new spec release, some noise should be raised in the general direction + of the implementation. + + * Please be considerate when being noisy to implementations. A gentle reminder + in their issue tracker is generally good enough. + +.. Note:: + MSCs MUST still describe what the stable endpoints/feature looks like with a note + towards the bottom for what the unstable feature flag/prefixes are. For example, + a MSC would propose `/_matrix/client/r0/new/endpoint`, not `/_matrix/client/unstable/ + com.example/new/endpoint`. + +In summary: + +* Implementations MUST NOT use stable endpoints before the MSC is in the spec. This + includes NOT using stable endpoints before a spec release happens, but after FCP has + passed. +* Implementations are able to ship features that are exposed to users by default before + an MSC has been merged to the spec, provided they follow the process above. +* Implementations SHOULD be wary of the technical debt they are incurring by moving faster + than the spec. +* The vendor prefix is chosen by the developer of the feature, using the Java package + naming convention. The foundation's preferred vendor prefix is `org.matrix`. +* The vendor prefixes, unstable feature flags, and unstable endpoints should be included + in the MSC, though the MSC MUST be written in a way that proposes new stable endpoints. + Typically this is solved by a small table at the bottom mapping the various values + from stable to unstable. Proposal Tracking ----------------- From da795e582478da464defd8f68d2635ea1c41eabc Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 5 Mar 2020 17:24:29 -0700 Subject: [PATCH 2/6] Apply suggestions from code review Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/proposals_intro.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 3f70345f..c3b312ce 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -385,14 +385,14 @@ The process for an implementation willing to be ahead of a spec release is: * When using this approach correctly, the implementation can ship/release the feature at any time, so long as the implementation is able to accept the technical debt that results from needing to provide adequate backwards and forwards - compatibility. The implementation MUST support the flag disappearing and be + compatibility. The implementation MUST support the flag (and server-side implementation) disappearing and be generally safe for users. Note that implementations early in the MSC review process may also be required to provide backwards compatibility with earlier editions of the proposal. * If the implementation cannot support the technical debt (or if it's impossible to provide forwards/backwards compatibility - e.g. a user authentication change which can't be safely rolled back), the implementation should not attempt to - implement the feature and instead wait for a spec release. + implement the feature and should instead wait for a spec release. * If at any point the idea changes, the feature flag should also change so that implementations can adapt as needed. @@ -429,7 +429,7 @@ The process for an implementation willing to be ahead of a spec release is: In summary: * Implementations MUST NOT use stable endpoints before the MSC is in the spec. This - includes NOT using stable endpoints before a spec release happens, but after FCP has + includes NOT using stable endpoints in the period between completion of FCP and release of the spec. passed. * Implementations are able to ship features that are exposed to users by default before an MSC has been merged to the spec, provided they follow the process above. From 79b2e5263d296a735d3346bf4eb8882841671fb6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sat, 7 Mar 2020 14:05:05 -0700 Subject: [PATCH 3/6] Update specification/proposals_intro.rst Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/proposals_intro.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index c3b312ce..5ee82dc3 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -393,7 +393,7 @@ The process for an implementation willing to be ahead of a spec release is: to provide forwards/backwards compatibility - e.g. a user authentication change which can't be safely rolled back), the implementation should not attempt to implement the feature and should instead wait for a spec release. - * If at any point the idea changes, the feature flag should also change so that + * If at any point after early release, the idea changes in a backwards-incompatible way, the feature flag should also change so that implementations can adapt as needed. 3. In parallel, or ahead of implementation, open an MSC and solicit review per above. From 54102b8d36d31ade99ad1b8023849d1a54a5dcc3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 13:50:17 -0600 Subject: [PATCH 4/6] Clarify wording per review suggestions --- specification/proposals_intro.rst | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index e6cebcf4..7c8e4efd 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -382,17 +382,16 @@ in some cases the MSC itself will be small enough to be considered proven. Where unclear if a MSC will require an implementation proof, ask in `#matrix-spec:matrix.org `_. -Early adoption of a MSC/idea -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Early release of a MSC/idea +~~~~~~~~~~~~~~~~~~~~~~~~~~~ To help facilitate early releases of software dependent on a spec release, implementations are required to use the following process to ensure that the official Matrix namespace is not cluttered with development or testing data. .. Note:: - Proof of concept implementations that are solely left on a branch/unreleased and for - demonstrating that a particular MSC works do not have to follow this. Note that the - ``develop`` branch adopted by some projects is still subject to this process. + Proof of concept implementations that are solely left on a branch/unreleased for + demonstrating that a particular MSC works do not have to follow this. The process for an implementation willing to be ahead of a spec release is: @@ -404,9 +403,9 @@ The process for an implementation willing to be ahead of a spec release is: `/_matrix/client/unstable/com.example/login`. Vendor prefixes throughout Matrix always use the Java package naming convention. The MSC for the feature should identify which preferred vendor prefix is to be used by early adopters. - * Unstable endpoints **do not** inherit from stable (e.g. `/r0`) APIs. Implementations - cannot assume that a particular endpoint will exist in the unstable namespace - even if the server advertises support for the feature. + * Note that unstable namespaces do not automatically inherit endpoints from stable + namespaces: for example, the fact that ``/_matrix/client/r0/sync`` exists does + not imply that ``/_matrix/client/unstable/com.example/sync`` exists. * If the client needs to be sure the server supports the feature, an unstable feature flag that MUST be vendor prefixed is to be used. This kind of flag shows up in the ``unstable_features`` section of ``/versions`` as, for example, @@ -431,7 +430,7 @@ The process for an implementation willing to be ahead of a spec release is: working as proposed. A typical example of this is an implementation of the MSC, though the implementation does not need to be shipped anywhere and can therefore avoid the forwards/backwards compatibility concerns mentioned here. -5. FCP is gone through, and assuming nothing is flagged the MSC lands. +5. The FCP process is completed, and assuming nothing is flagged the MSC lands. 6. A spec PR is written to incorporate the changes into Matrix. 7. A spec release happens. 8. Implementations switch to using stable prefixes (e.g.: ``/r0``) if the server From 04e8244dbc3ffb7d805bd6c97bf7271c9a472162 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 17:44:40 -0600 Subject: [PATCH 5/6] Clarify various bits of wording Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/proposals_intro.rst | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 7c8e4efd..2e6f4389 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -390,17 +390,15 @@ are required to use the following process to ensure that the official Matrix nam is not cluttered with development or testing data. .. Note:: - Proof of concept implementations that are solely left on a branch/unreleased for - demonstrating that a particular MSC works do not have to follow this. - -The process for an implementation willing to be ahead of a spec release is: + Unreleased implementations (including proofs-of-concept demonstrating that a + particular MSC works) do not have to follow this process. 1. Have an idea for a feature. 2. Implement the feature using unstable endpoints, vendor prefixes, and unstable feature flags as appropriate. * When using unstable endpoints, they MUST include a vendor prefix. For example: - `/_matrix/client/unstable/com.example/login`. Vendor prefixes throughout Matrix + ``/_matrix/client/unstable/com.example/login``. Vendor prefixes throughout Matrix always use the Java package naming convention. The MSC for the feature should identify which preferred vendor prefix is to be used by early adopters. * Note that unstable namespaces do not automatically inherit endpoints from stable From 8220de58bddeb7653dd917cf2831f543698dc50a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 17:48:13 -0600 Subject: [PATCH 6/6] Drop loudly and instead encourage --- specification/proposals_intro.rst | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 2e6f4389..6d8dc8a9 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -436,7 +436,7 @@ is not cluttered with development or testing data. specification version, but does have the feature flag, unstable prefixes should still be used. 9. A transition period of about 2 months starts immediately after the spec release, - before implementations start to loudly encourage other implementations to switch + before implementations start to encourage other implementations to switch to stable endpoints. For example, a server implementation should start asking client implementations to support the stable endpoints 2 months after the spec release, if they haven't already. The same applies in the reverse: if clients @@ -444,9 +444,6 @@ is not cluttered with development or testing data. supporting the new spec release, some noise should be raised in the general direction of the implementation. - * Please be considerate when being noisy to implementations. A gentle reminder - in their issue tracker is generally good enough. - .. Note:: MSCs MUST still describe what the stable endpoints/feature looks like with a note towards the bottom for what the unstable feature flag/prefixes are. For example,